* via-rhine interrupts
From: Jakub Ružička @ 2010-07-29 11:03 UTC (permalink / raw)
To: netdev
[-- Attachment #1: Type: text/plain, Size: 735 bytes --]
Hello,
the via-rhine driver powered cards generate a really big number of
interrupts, almost one per packet (11429 interrupts for 8210 incoming
and 2475 outgoing packets per second on full 100 Mbps load). This is
observed on multiple different machines (embbed and desktop) and
kernels (2.6.25 with and without NAPI, 2.6.30, 2.6.32 and 2.6.33). Do
you have any idea why isn't the polling used or what can I try to find
out what's wrong?
I have tested sending to/from the machines with nc and scp, measured
interrups and load with atop. Few of these measurements on an embbed
device (where the interrupt handling is a problem) are attached.
I'm not a subscriber, please Cc me if needed.
Thankfully
Jakub Ružička
[-- Attachment #2: atop_on_2.6.25.20_no_NAPI --]
[-- Type: application/octet-stream, Size: 1013 bytes --]
PRC | sys 0.06s | user 0.00s | #proc 44 | #zombie 0 | #exit 0/s
CPU | sys 0% | user 0% | irq 45% | idle 54% | wait 0% |
CPL | avg1 0.00 | avg5 0.00 | avg15 0.00 | csw 13/s | intr 10757/s
MEM | tot 247.4M | free 183.1M | cache 38.4M | buff 4.8M | slab 6.2M
SWP | tot 0.0M | free 0.0M | | vmcom 46.0M | vmlim 123.7M
PAG | scan 0/s | stall 0/s | | swin 0/s | swout 0/s
DSK | sda | busy 0% | read 0/s | write 0/s | avio 0 ms
NET | transport | tcpi 1/s | tcpo 1/s | udpi 0/s | udpo 0/s
NET | network | ipi 1/s | ipo 1/s | ipfrw 0/s | deliv 1/s
NET | eth2 98% | pcki 8112/s | pcko 0/s | si 98 Mbps | so 0 Kbps
NET | eth1 2% | pcki 3838/s | pcko 0/s | si 2032 Kbps | so 0 Kbps
NET | eth0 0% | pcki 7/s | pcko 1/s | si 3 Kbps | so 1 Kbps
NET | lo ---- | pcki 0/s | pcko 0/s | si 0 Kbps | so 0 Kbps
[-- Attachment #3: atop_on_2.6.25.20_with_NAPI --]
[-- Type: application/octet-stream, Size: 1014 bytes --]
PRC | sys 0.08s | user 0.00s | #proc 42 | #zombie 0 | #exit 0/s
CPU | sys 1% | user 2% | irq 47% | idle 50% | wait 0% |
CPL | avg1 0.06 | avg5 0.23 | avg15 0.12 | csw 12/s | intr 11902/s
MEM | tot 247.4M | free 189.1M | cache 35.5M | buff 2.8M | slab 6.2M
SWP | tot 0.0M | free 0.0M | | vmcom 42.0M | vmlim 123.7M
PAG | scan 0/s | stall 0/s | | swin 0/s | swout 0/s
DSK | sda | busy 0% | read 0/s | write 2/s | avio 2 ms
NET | transport | tcpi 1/s | tcpo 1/s | udpi 0/s | udpo 0/s
NET | network | ipi 1/s | ipo 1/s | ipfrw 0/s | deliv 1/s
NET | eth2 98% | pcki 8133/s | pcko 0/s | si 98 Mbps | so 0 Kbps
NET | eth1 2% | pcki 3983/s | pcko 0/s | si 2104 Kbps | so 0 Kbps
NET | eth0 0% | pcki 3/s | pcko 1/s | si 1 Kbps | so 1 Kbps
NET | lo ---- | pcki 0/s | pcko 0/s | si 0 Kbps | so 0 Kbps
[-- Attachment #4: atop_on_2.6.32 --]
[-- Type: application/octet-stream, Size: 1014 bytes --]
PRC | sys 0.16s | user 0.01s | #proc 50 | #zombie 0 | #exit 0/s
CPU | sys 3% | user 0% | irq 69% | idle 28% | wait 0%
CPL | avg1 0.03 | avg5 0.26 | avg15 0.15 | csw 11/s | intr 11342/s
MEM | tot 243.2M | free 180.1M | cache 36.8M | buff 2.8M | slab 6.8M
SWP | tot 0.0M | free 0.0M | | vmcom 45.8M | vmlim 121.6M
PAG | scan 0/s | stall 0/s | | swin 0/s | swout 0/s
DSK | sda | busy 0% | read 0/s | write 1/s | avio 8 ms
NET | transport | tcpi 1/s | tcpo 1/s | udpi 0/s | udpo 0/s
NET | network | ipi 1/s | ipo 1/s | ipfrw 0/s | deliv 1/s
NET | eth2 98% | pcki 8126/s | pcko 0/s | si 98 Mbps | so 0 Kbps
NET | eth1 2% | pcki 3887/s | pcko 0/s | si 2057 Kbps | so 0 Kbps
NET | eth0 0% | pcki 1/s | pcko 1/s | si 0 Kbps | so 1 Kbps
NET | lo ---- | pcki 0/s | pcko 0/s | si 0 Kbps | so 0 Kbps
^ permalink raw reply
* Re: [BUG] bridge leaks 3 references on lo per up&down
From: Herbert Xu @ 2010-07-29 11:12 UTC (permalink / raw)
To: David Lamparter; +Cc: netdev, Stephen Hemminger, bridge
In-Reply-To: <20100728120336.GD13413@jupiter.n2.diac24.net>
On Wed, Jul 28, 2010 at 02:03:36PM +0200, David Lamparter wrote:
>
> Herbert, you implemented bridge multicast support, maybe you can figure
> out where these references are taken?
OK this patch should fix it.
However, it also means that the IPv6 membership report parsing is
broken and we still need look into it.
bridge: Fix skb leak when multicast parsing fails on TX
On the bridge TX path we're leaking an skb when br_multicast_rcv
returns an error.
Reported-by: David Lamparter <equinox@diac24.net>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/net/bridge/br_device.c b/net/bridge/br_device.c
index 753fc42..f49f20b 100644
--- a/net/bridge/br_device.c
+++ b/net/bridge/br_device.c
@@ -47,8 +47,10 @@ netdev_tx_t br_dev_xmit(struct sk_buff *skb, struct net_device *dev)
skb_pull(skb, ETH_HLEN);
if (is_multicast_ether_addr(dest)) {
- if (br_multicast_rcv(br, NULL, skb))
+ if (br_multicast_rcv(br, NULL, skb)) {
+ kfree_skb(skb);
goto out;
+ }
mdst = br_mdb_get(br, skb);
if (mdst || BR_INPUT_SKB_CB_MROUTERS_ONLY(skb))
Thanks,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related
* [RFC PATCH v8 00/16] Provide a zero-copy method on KVM virtio-net.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike
We provide an zero-copy method which driver side may get external
buffers to DMA. Here external means driver don't use kernel space
to allocate skb buffers. Currently the external buffer can be from
guest virtio-net driver.
The idea is simple, just to pin the guest VM user space and then
let host NIC driver has the chance to directly DMA to it.
The patches are based on vhost-net backend driver. We add a device
which provides proto_ops as sendmsg/recvmsg to vhost-net to
send/recv directly to/from the NIC driver. KVM guest who use the
vhost-net backend may bind any ethX interface in the host side to
get copyless data transfer thru guest virtio-net frontend.
patch 01-10: net core and kernel changes.
patch 11-13: new device as interface to mantpulate external buffers.
patch 14: for vhost-net.
patch 15: An example on modifying NIC driver to using napi_gro_frags().
patch 16: An example how to get guest buffers based on driver
who using napi_gro_frags().
The guest virtio-net driver submits multiple requests thru vhost-net
backend driver to the kernel. And the requests are queued and then
completed after corresponding actions in h/w are done.
For read, user space buffers are dispensed to NIC driver for rx when
a page constructor API is invoked. Means NICs can allocate user buffers
from a page constructor. We add a hook in netif_receive_skb() function
to intercept the incoming packets, and notify the zero-copy device.
For write, the zero-copy deivce may allocates a new host skb and puts
payload on the skb_shinfo(skb)->frags, and copied the header to skb->data.
The request remains pending until the skb is transmitted by h/w.
We provide multiple submits and asynchronous notifiicaton to
vhost-net too.
Our goal is to improve the bandwidth and reduce the CPU usage.
Exact performance data will be provided later.
What we have not done yet:
Performance tuning
what we have done in v1:
polish the RCU usage
deal with write logging in asynchroush mode in vhost
add notifier block for mp device
rename page_ctor to mp_port in netdevice.h to make it looks generic
add mp_dev_change_flags() for mp device to change NIC state
add CONIFG_VHOST_MPASSTHRU to limit the usage when module is not load
a small fix for missing dev_put when fail
using dynamic minor instead of static minor number
a __KERNEL__ protect to mp_get_sock()
what we have done in v2:
remove most of the RCU usage, since the ctor pointer is only
changed by BIND/UNBIND ioctl, and during that time, NIC will be
stopped to get good cleanup(all outstanding requests are finished),
so the ctor pointer cannot be raced into wrong situation.
Remove the struct vhost_notifier with struct kiocb.
Let vhost-net backend to alloc/free the kiocb and transfer them
via sendmsg/recvmsg.
use get_user_pages_fast() and set_page_dirty_lock() when read.
Add some comments for netdev_mp_port_prep() and handle_mpassthru().
what we have done in v3:
the async write logging is rewritten
a drafted synchronous write function for qemu live migration
a limit for locked pages from get_user_pages_fast() to prevent Dos
by using RLIMIT_MEMLOCK
what we have done in v4:
add iocb completion callback from vhost-net to queue iocb in mp device
replace vq->receiver by mp_sock_data_ready()
remove stuff in mp device which access structures from vhost-net
modify skb_reserve() to ignore host NIC driver reserved space
rebase to the latest vhost tree
split large patches into small pieces, especially for net core part.
what we have done in v5:
address Arnd Bergmann's comments
-remove IFF_MPASSTHRU_EXCL flag in mp device
-Add CONFIG_COMPAT macro
-remove mp_release ops
move dev_is_mpassthru() as inline func
fix a bug in memory relinquish
Apply to current git (2.6.34-rc6) tree.
what we have done in v6:
move create_iocb() out of page_dtor which may happen in interrupt context
-This remove the potential issues which lock called in interrupt context
make the cache used by mp, vhost as static, and created/destoryed during
modules init/exit functions.
-This makes multiple mp guest created at the same time.
what we have done in v7:
some cleanup prepared to suppprt PS mode
what we have done in v8
discarding the modifications to point skb->data to guest buffer directly.
Add code to modify driver to support napi_gro_frags() with Herbert's comments.
To support PS mode.
Add mergeable buffer support in mp device.
Add GSO/GRO support in mp deice.
Address comments from Eric Dumazet about cache line and rcu usage.
^ permalink raw reply
* [RFC PATCH v8 01/16] Add a new structure for skb buffer from external.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-1-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/skbuff.h | 9 +++++++++
1 files changed, 9 insertions(+), 0 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 124f90c..74af06c 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -203,6 +203,15 @@ struct skb_shared_info {
void * destructor_arg;
};
+/* The structure is for a skb which pages may point to
+ * an external buffer, which is not allocated from kernel space.
+ * It also contains a destructor for itself.
+ */
+struct skb_ext_page {
+ struct page *page;
+ void (*dtor)(struct skb_ext_page *);
+};
+
/* We divide dataref into two halves. The higher 16 bits hold references
* to the payload part of skb->data. The lower 16 bits hold references to
* the entire skb->data. A clone of a headerless skb holds the length of
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 02/16] Add a new struct for device to manipulate external buffer.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-2-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/netdevice.h | 22 +++++++++++++++++++++-
1 files changed, 21 insertions(+), 1 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index fa8b476..ba582e1 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -530,6 +530,25 @@ struct netdev_queue {
unsigned long tx_dropped;
} ____cacheline_aligned_in_smp;
+/* Add a structure in structure net_device, the new field is
+ * named as mp_port. It's for mediate passthru (zero-copy).
+ * It contains the capability for the net device driver,
+ * a socket, and an external buffer creator, external means
+ * skb buffer belongs to the device may not be allocated from
+ * kernel space.
+ */
+struct mpassthru_port {
+ int hdr_len;
+ int data_len;
+ int npages;
+ unsigned flags;
+ struct socket *sock;
+ int vnet_hlen;
+ struct skb_ext_page *(*ctor)(struct mpassthru_port *,
+ struct sk_buff *, int);
+ struct skb_ext_page *(*hash)(struct net_device *,
+ struct page *);
+};
/*
* This structure defines the management hooks for network devices.
@@ -952,7 +971,8 @@ struct net_device {
struct macvlan_port *macvlan_port;
/* GARP */
struct garp_port *garp_port;
-
+ /* mpassthru */
+ struct mpassthru_port *mp_port;
/* class/net/name entry */
struct device dev;
/* space for optional device, statistics, and wireless sysfs groups */
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 05/16] Add a function to indicate if device use external buffer.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-5-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/netdevice.h | 5 +++++
1 files changed, 5 insertions(+), 0 deletions(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index 5f192de..23d6ec0 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1602,6 +1602,11 @@ extern gro_result_t napi_gro_frags(struct napi_struct *napi);
extern int netdev_mp_port_prep(struct net_device *dev,
struct mpassthru_port *port);
+static inline bool dev_is_mpassthru(struct net_device *dev)
+{
+ return dev && dev->mp_port;
+}
+
static inline void napi_free_frags(struct napi_struct *napi)
{
kfree_skb(napi->skb);
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 06/16] Use callback to deal with skb_release_data() specially.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-6-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
If buffer is external, then use the callback to destruct
buffers.
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/skbuff.h | 3 ++-
net/core/skbuff.c | 8 ++++++++
2 files changed, 10 insertions(+), 1 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index 74af06c..ab29675 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -197,10 +197,11 @@ struct skb_shared_info {
union skb_shared_tx tx_flags;
struct sk_buff *frag_list;
struct skb_shared_hwtstamps hwtstamps;
- skb_frag_t frags[MAX_SKB_FRAGS];
/* Intermediate layers must ensure that destructor_arg
* remains valid until skb destructor */
void * destructor_arg;
+
+ skb_frag_t frags[MAX_SKB_FRAGS];
};
/* The structure is for a skb which pages may point to
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 93c4e06..117d82b 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -217,6 +217,7 @@ struct sk_buff *__alloc_skb(unsigned int size, gfp_t gfp_mask,
shinfo->gso_type = 0;
shinfo->ip6_frag_id = 0;
shinfo->tx_flags.flags = 0;
+ shinfo->destructor_arg = NULL;
skb_frag_list_init(skb);
memset(&shinfo->hwtstamps, 0, sizeof(shinfo->hwtstamps));
@@ -350,6 +351,13 @@ static void skb_release_data(struct sk_buff *skb)
if (skb_has_frags(skb))
skb_drop_fraglist(skb);
+ if (skb->dev && dev_is_mpassthru(skb->dev)) {
+ struct skb_ext_page *ext_page =
+ skb_shinfo(skb)->destructor_arg;
+ if (ext_page && ext_page->dtor)
+ ext_page->dtor(ext_page);
+ }
+
kfree(skb->head);
}
}
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 08/16] Modify netdev_free_page() to release external buffer
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-8-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
Currently, it can get external buffers from mp device.
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/skbuff.h | 4 +++-
net/core/skbuff.c | 24 ++++++++++++++++++++++++
2 files changed, 27 insertions(+), 1 deletions(-)
diff --git a/include/linux/skbuff.h b/include/linux/skbuff.h
index ab29675..3d7f70e 100644
--- a/include/linux/skbuff.h
+++ b/include/linux/skbuff.h
@@ -1512,9 +1512,11 @@ static inline struct page *netdev_alloc_page(struct net_device *dev)
return __netdev_alloc_page(dev, GFP_ATOMIC);
}
+extern void __netdev_free_page(struct net_device *dev, struct page *page);
+
static inline void netdev_free_page(struct net_device *dev, struct page *page)
{
- __free_page(page);
+ __netdev_free_page(dev, page);
}
/**
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index 1a61e2b..bbf4707 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -306,6 +306,30 @@ struct page *__netdev_alloc_page(struct net_device *dev, gfp_t gfp_mask)
}
EXPORT_SYMBOL(__netdev_alloc_page);
+void netdev_free_ext_page(struct net_device *dev, struct page *page)
+{
+ struct skb_ext_page *ext_page = NULL;
+ if (dev_is_mpassthru(dev) && dev->mp_port->hash) {
+ ext_page = dev->mp_port->hash(dev, page);
+ if (ext_page)
+ ext_page->dtor(ext_page);
+ else
+ __free_page(page);
+ }
+}
+EXPORT_SYMBOL(netdev_free_ext_page);
+
+void __netdev_free_page(struct net_device *dev, struct page *page)
+{
+ if (dev_is_mpassthru(dev)) {
+ netdev_free_ext_page(dev, page);
+ return;
+ }
+
+ __free_page(page);
+}
+EXPORT_SYMBOL(__netdev_free_page);
+
void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off,
int size)
{
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 11/16] Add header file for mp device.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-11-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
include/linux/mpassthru.h | 25 +++++++++++++++++++++++++
1 files changed, 25 insertions(+), 0 deletions(-)
create mode 100644 include/linux/mpassthru.h
diff --git a/include/linux/mpassthru.h b/include/linux/mpassthru.h
new file mode 100644
index 0000000..ba8f320
--- /dev/null
+++ b/include/linux/mpassthru.h
@@ -0,0 +1,25 @@
+#ifndef __MPASSTHRU_H
+#define __MPASSTHRU_H
+
+#include <linux/types.h>
+#include <linux/if_ether.h>
+
+/* ioctl defines */
+#define MPASSTHRU_BINDDEV _IOW('M', 213, int)
+#define MPASSTHRU_UNBINDDEV _IO('M', 214)
+
+#ifdef __KERNEL__
+#if defined(CONFIG_MEDIATE_PASSTHRU) || defined(CONFIG_MEDIATE_PASSTHRU_MODULE)
+struct socket *mp_get_socket(struct file *);
+#else
+#include <linux/err.h>
+#include <linux/errno.h>
+struct file;
+struct socket;
+static inline struct socket *mp_get_socket(struct file *f)
+{
+ return ERR_PTR(-EINVAL);
+}
+#endif /* CONFIG_MEDIATE_PASSTHRU */
+#endif /* __KERNEL__ */
+#endif /* __MPASSTHRU_H */
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 12/16] Add mp(mediate passthru) device.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-13-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
The patch add mp(mediate passthru) device, which now
based on vhost-net backend driver and provides proto_ops
to send/receive guest buffers data from/to guest vitio-net
driver.
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
Signed-off-by: Zhao Yu <yzhao81new@gmail.com>
Reviewed-by: Jeff Dike <jdike@linux.intel.com>
---
drivers/vhost/mpassthru.c | 1394 +++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 1394 insertions(+), 0 deletions(-)
create mode 100644 drivers/vhost/mpassthru.c
diff --git a/drivers/vhost/mpassthru.c b/drivers/vhost/mpassthru.c
new file mode 100644
index 0000000..012ebda
--- /dev/null
+++ b/drivers/vhost/mpassthru.c
@@ -0,0 +1,1394 @@
+/*
+ * MPASSTHRU - Mediate passthrough device.
+ * Copyright (C) 2009 ZhaoYu, XinXiaohui, Dike, Jeffery G
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#define DRV_NAME "mpassthru"
+#define DRV_DESCRIPTION "Mediate passthru device driver"
+#define DRV_COPYRIGHT "(C) 2009 ZhaoYu, XinXiaohui, Dike, Jeffery G"
+
+#include <linux/compat.h>
+#include <linux/module.h>
+#include <linux/errno.h>
+#include <linux/kernel.h>
+#include <linux/major.h>
+#include <linux/slab.h>
+#include <linux/smp_lock.h>
+#include <linux/poll.h>
+#include <linux/fcntl.h>
+#include <linux/init.h>
+#include <linux/aio.h>
+
+#include <linux/skbuff.h>
+#include <linux/netdevice.h>
+#include <linux/etherdevice.h>
+#include <linux/miscdevice.h>
+#include <linux/ethtool.h>
+#include <linux/rtnetlink.h>
+#include <linux/if.h>
+#include <linux/if_arp.h>
+#include <linux/if_ether.h>
+#include <linux/crc32.h>
+#include <linux/nsproxy.h>
+#include <linux/uaccess.h>
+#include <linux/virtio_net.h>
+#include <linux/mpassthru.h>
+#include <net/net_namespace.h>
+#include <net/netns/generic.h>
+#include <net/rtnetlink.h>
+#include <net/sock.h>
+
+#include <asm/system.h>
+
+/* Uncomment to enable debugging */
+/* #define MPASSTHRU_DEBUG 1 */
+
+#ifdef MPASSTHRU_DEBUG
+static int debug;
+
+#define DBG if (mp->debug) printk
+#define DBG1 if (debug == 2) printk
+#else
+#define DBG(a...)
+#define DBG1(a...)
+#endif
+
+#define COPY_THRESHOLD (L1_CACHE_BYTES * 4)
+#define COPY_HDR_LEN (L1_CACHE_BYTES < 64 ? 64 : L1_CACHE_BYTES)
+
+struct frag {
+ u16 offset;
+ u16 size;
+};
+
+#define HASH_BUCKETS (8192*2)
+
+struct page_info {
+ struct list_head list;
+ struct page_info *next;
+ struct page_info *prev;
+ int header;
+ /* indicate the actual length of bytes
+ * send/recv in the external buffers
+ */
+ int total;
+ int offset;
+ struct page *pages[MAX_SKB_FRAGS+1];
+ struct skb_frag_struct frag[MAX_SKB_FRAGS+1];
+ struct sk_buff *skb;
+ struct page_ctor *ctor;
+
+ /* The pointer relayed to skb, to indicate
+ * it's a external allocated skb or kernel
+ */
+ struct skb_ext_page ext_page;
+ struct skb_shared_info ushinfo;
+
+#define INFO_READ 0
+#define INFO_WRITE 1
+ unsigned flags;
+ unsigned pnum;
+
+ /* It's meaningful for receive, means
+ * the max length allowed
+ */
+ size_t len;
+
+ /* The fields after that is for backend
+ * driver, now for vhost-net.
+ */
+
+ struct kiocb *iocb;
+ unsigned int desc_pos;
+ struct iovec hdr[MAX_SKB_FRAGS + 2];
+ struct iovec iov[MAX_SKB_FRAGS + 2];
+};
+
+static struct kmem_cache *ext_page_info_cache;
+
+struct page_ctor {
+ struct list_head readq;
+ int wq_len;
+ int rq_len;
+ spinlock_t read_lock;
+ /* record the locked pages */
+ int lock_pages;
+ struct rlimit o_rlim;
+ struct net_device *dev;
+ struct mpassthru_port port;
+ struct page_info **hash_table;
+};
+
+struct mp_struct {
+ struct mp_file *mfile;
+ struct net_device *dev;
+ struct page_ctor *ctor;
+ struct socket socket;
+
+#ifdef MPASSTHRU_DEBUG
+ int debug;
+#endif
+};
+
+struct mp_file {
+ atomic_t count;
+ struct mp_struct *mp;
+ struct net *net;
+};
+
+struct mp_sock {
+ struct sock sk;
+ struct mp_struct *mp;
+};
+
+static int mp_dev_change_flags(struct net_device *dev, unsigned flags)
+{
+ int ret = 0;
+
+ rtnl_lock();
+ ret = dev_change_flags(dev, flags);
+ rtnl_unlock();
+
+ if (ret < 0)
+ printk(KERN_ERR "failed to change dev state of %s", dev->name);
+
+ return ret;
+}
+
+/* The main function to allocate external buffers */
+static struct skb_ext_page *page_ctor(struct mpassthru_port *port,
+ struct sk_buff *skb, int npages)
+{
+ int i;
+ unsigned long flags;
+ struct page_ctor *ctor;
+ struct page_info *info = NULL;
+
+ ctor = container_of(port, struct page_ctor, port);
+
+ spin_lock_irqsave(&ctor->read_lock, flags);
+ if (!list_empty(&ctor->readq)) {
+ info = list_first_entry(&ctor->readq, struct page_info, list);
+ list_del(&info->list);
+ ctor->rq_len--;
+ }
+ spin_unlock_irqrestore(&ctor->read_lock, flags);
+ if (!info)
+ return NULL;
+
+ for (i = 0; i < info->pnum; i++)
+ get_page(info->pages[i]);
+ info->skb = skb;
+ return &info->ext_page;
+}
+
+static struct page_info *mp_hash_lookup(struct page_ctor *ctor,
+ struct page *page);
+
+static struct skb_ext_page *mp_lookup(struct net_device *dev,
+ struct page *page)
+{
+ struct mp_struct *mp =
+ container_of(dev->mp_port->sock->sk, struct mp_sock, sk)->mp;
+ struct page_ctor *ctor = mp->ctor;
+ struct page_info *info;
+
+ info = mp_hash_lookup(ctor, page);
+ if (!info)
+ return NULL;
+ return &info->ext_page;
+}
+
+static int page_ctor_attach(struct mp_struct *mp)
+{
+ int rc;
+ struct page_ctor *ctor;
+ struct net_device *dev = mp->dev;
+
+ /* locked by mp_mutex */
+ if (mp->ctor)
+ return -EBUSY;
+
+ ctor = kzalloc(sizeof(*ctor), GFP_KERNEL);
+ if (!ctor)
+ return -ENOMEM;
+ rc = netdev_mp_port_prep(dev, &ctor->port);
+ if (rc)
+ goto fail;
+
+ INIT_LIST_HEAD(&ctor->readq);
+ spin_lock_init(&ctor->read_lock);
+ ctor->hash_table = kzalloc(sizeof(struct page_info *) * HASH_BUCKETS,
+ GFP_KERNEL);
+ if (!ctor->hash_table)
+ goto fail_hash;
+
+ ctor->rq_len = 0;
+ ctor->wq_len = 0;
+
+ dev_hold(dev);
+ ctor->dev = dev;
+ ctor->port.ctor = page_ctor;
+ ctor->port.sock = &mp->socket;
+ ctor->port.hash = mp_lookup;
+ ctor->lock_pages = 0;
+
+ /* locked by mp_mutex */
+ dev->mp_port = &ctor->port;
+ mp->ctor = ctor;
+
+ return 0;
+
+fail_hash:
+ kfree(ctor->hash_table);
+
+fail:
+ kfree(ctor);
+ dev_put(dev);
+
+ return rc;
+}
+
+struct page_info *info_dequeue(struct page_ctor *ctor)
+{
+ unsigned long flags;
+ struct page_info *info = NULL;
+ spin_lock_irqsave(&ctor->read_lock, flags);
+ if (!list_empty(&ctor->readq)) {
+ info = list_first_entry(&ctor->readq,
+ struct page_info, list);
+ list_del(&info->list);
+ ctor->rq_len--;
+ }
+ spin_unlock_irqrestore(&ctor->read_lock, flags);
+ return info;
+}
+
+static int set_memlock_rlimit(struct page_ctor *ctor, int resource,
+ unsigned long cur, unsigned long max)
+{
+ struct rlimit new_rlim, *old_rlim;
+ int retval;
+
+ if (resource != RLIMIT_MEMLOCK)
+ return -EINVAL;
+ new_rlim.rlim_cur = cur;
+ new_rlim.rlim_max = max;
+
+ old_rlim = current->signal->rlim + resource;
+
+ /* remember the old rlimit value when backend enabled */
+ ctor->o_rlim.rlim_cur = old_rlim->rlim_cur;
+ ctor->o_rlim.rlim_max = old_rlim->rlim_max;
+
+ if ((new_rlim.rlim_max > old_rlim->rlim_max) &&
+ !capable(CAP_SYS_RESOURCE))
+ return -EPERM;
+
+ retval = security_task_setrlimit(resource, &new_rlim);
+ if (retval)
+ return retval;
+
+ task_lock(current->group_leader);
+ *old_rlim = new_rlim;
+ task_unlock(current->group_leader);
+ return 0;
+}
+
+static void relinquish_resource(struct page_ctor *ctor)
+{
+ if (!(ctor->dev->flags & IFF_UP) &&
+ !(ctor->wq_len + ctor->rq_len))
+ printk(KERN_INFO "relinquish_resource\n");
+}
+
+static void mp_ki_dtor(struct kiocb *iocb)
+{
+ struct page_info *info = (struct page_info *)(iocb->private);
+ int i;
+
+ if (info->flags == INFO_READ) {
+ for (i = 0; i < info->pnum; i++) {
+ if (info->pages[i]) {
+ set_page_dirty_lock(info->pages[i]);
+ put_page(info->pages[i]);
+ }
+ }
+ if (info->skb) {
+ info->skb->destructor = NULL;
+ kfree_skb(info->skb);
+ }
+ info->ctor->rq_len--;
+ } else
+ info->ctor->wq_len--;
+ /* Decrement the number of locked pages */
+ info->ctor->lock_pages -= info->pnum;
+ kmem_cache_free(ext_page_info_cache, info);
+ relinquish_resource(info->ctor);
+
+ return;
+}
+
+static struct kiocb *create_iocb(struct page_info *info, int size)
+{
+ struct kiocb *iocb = NULL;
+
+ iocb = info->iocb;
+ if (!iocb)
+ return iocb;
+ iocb->ki_flags = 0;
+ iocb->ki_users = 1;
+ iocb->ki_key = 0;
+ iocb->ki_ctx = NULL;
+ iocb->ki_cancel = NULL;
+ iocb->ki_retry = NULL;
+ iocb->ki_eventfd = NULL;
+ iocb->ki_pos = info->desc_pos;
+ iocb->ki_nbytes = size;
+ iocb->ki_dtor(iocb);
+ iocb->private = (void *)info;
+ iocb->ki_dtor = mp_ki_dtor;
+
+ return iocb;
+}
+
+static int page_ctor_detach(struct mp_struct *mp)
+{
+ struct page_ctor *ctor;
+ struct page_info *info;
+ int i;
+
+ /* locked by mp_mutex */
+ ctor = mp->ctor;
+ if (!ctor)
+ return -ENODEV;
+
+ while ((info = info_dequeue(ctor))) {
+ for (i = 0; i < info->pnum; i++)
+ if (info->pages[i])
+ put_page(info->pages[i]);
+ create_iocb(info, 0);
+ ctor->rq_len--;
+ kmem_cache_free(ext_page_info_cache, info);
+ }
+
+ relinquish_resource(ctor);
+
+ set_memlock_rlimit(ctor, RLIMIT_MEMLOCK,
+ ctor->o_rlim.rlim_cur,
+ ctor->o_rlim.rlim_max);
+
+ /* locked by mp_mutex */
+ ctor->dev->mp_port = NULL;
+ dev_put(ctor->dev);
+
+ mp->ctor = NULL;
+ kfree(ctor->hash_table);
+ kfree(ctor);
+ return 0;
+}
+
+static void __mp_detach(struct mp_struct *mp)
+{
+ mp->mfile = NULL;
+
+ mp_dev_change_flags(mp->dev, mp->dev->flags & ~IFF_UP);
+ page_ctor_detach(mp);
+ mp_dev_change_flags(mp->dev, mp->dev->flags | IFF_UP);
+
+ /* Drop the extra count on the net device */
+ dev_put(mp->dev);
+}
+
+static DEFINE_MUTEX(mp_mutex);
+
+static void mp_detach(struct mp_struct *mp)
+{
+ mutex_lock(&mp_mutex);
+ __mp_detach(mp);
+ mutex_unlock(&mp_mutex);
+}
+
+static struct mp_struct *mp_get(struct mp_file *mfile)
+{
+ struct mp_struct *mp = NULL;
+ if (atomic_inc_not_zero(&mfile->count))
+ mp = mfile->mp;
+
+ return mp;
+}
+
+static void mp_put(struct mp_file *mfile)
+{
+ if (atomic_dec_and_test(&mfile->count))
+ mp_detach(mfile->mp);
+}
+
+static void iocb_tag(struct kiocb *iocb)
+{
+ iocb->ki_flags = 1;
+}
+
+/* The callback to destruct the external buffers or skb */
+static void page_dtor(struct skb_ext_page *ext_page)
+{
+ struct page_info *info;
+ struct page_ctor *ctor;
+ struct sock *sk;
+ struct sk_buff *skb;
+
+ if (!ext_page)
+ return;
+ info = container_of(ext_page, struct page_info, ext_page);
+ if (!info)
+ return;
+ ctor = info->ctor;
+ skb = info->skb;
+
+ if (info->flags == INFO_READ) {
+ create_iocb(info, 0);
+ return;
+ }
+
+ /* For transmit, we should wait for the DMA finish by hardware.
+ * Queue the notifier to wake up the backend driver
+ */
+
+ iocb_tag(info->iocb);
+ sk = ctor->port.sock->sk;
+ sk->sk_write_space(sk);
+
+ return;
+}
+
+/* For small exteranl buffers transmit, we don't need to call
+ * get_user_pages().
+ */
+static struct page_info *alloc_small_page_info(struct page_ctor *ctor,
+ struct kiocb *iocb, int total)
+{
+ struct page_info *info =
+ kmem_cache_zalloc(ext_page_info_cache, GFP_KERNEL);
+
+ if (!info)
+ return NULL;
+ info->total = total;
+ info->ext_page.dtor = page_dtor;
+ info->ctor = ctor;
+ info->flags = INFO_WRITE;
+ info->iocb = iocb;
+ return info;
+}
+
+typedef u32 key_mp_t;
+static inline key_mp_t mp_hash(struct page *page, int buckets)
+{
+ key_mp_t k;
+
+ k = ((((unsigned long)page << 32UL) >> 32UL) / 0x38) % buckets ;
+ return k;
+}
+
+static void mp_hash_insert(struct page_ctor *ctor,
+ struct page *page, struct page_info *page_info)
+{
+ struct page_info *tmp;
+ key_mp_t key = mp_hash(page, HASH_BUCKETS);
+ if (!ctor->hash_table[key]) {
+ ctor->hash_table[key] = page_info;
+ return;
+ }
+
+ tmp = ctor->hash_table[key];
+ while (tmp->next)
+ tmp = tmp->next;
+
+ tmp->next = page_info;
+ page_info->prev = tmp;
+ return;
+}
+
+static struct page_info *mp_hash_lookup(struct page_ctor *ctor,
+ struct page *page)
+{
+ key_mp_t key = mp_hash(page, HASH_BUCKETS);
+ struct page_info *tmp = NULL;
+ int i;
+
+ tmp = ctor->hash_table[key];
+ while (tmp) {
+ for (i = 0; i < tmp->pnum; i++) {
+ if (tmp->pages[i] == page) {
+ if (!tmp->prev) {
+ ctor->hash_table[key] = tmp->next;
+ if (tmp->next)
+ tmp->next->prev = NULL;
+ } else {
+ tmp->prev->next = tmp->next;
+ if (tmp->next)
+ tmp->next->prev = tmp->prev;
+ }
+ return tmp;
+ }
+ }
+ tmp = tmp->next;
+ }
+ return tmp;
+}
+
+/* The main function to transform the guest user space address
+ * to host kernel address via get_user_pages(). Thus the hardware
+ * can do DMA directly to the external buffer address.
+ */
+static struct page_info *alloc_page_info(struct page_ctor *ctor,
+ struct kiocb *iocb, struct iovec *iov,
+ int count, struct frag *frags,
+ int npages, int total)
+{
+ int rc;
+ int i, j, n = 0;
+ int len;
+ unsigned long base, lock_limit;
+ struct page_info *info = NULL;
+
+ lock_limit = current->signal->rlim[RLIMIT_MEMLOCK].rlim_cur;
+ lock_limit >>= PAGE_SHIFT;
+
+ if (ctor->lock_pages + count > lock_limit && npages) {
+ printk(KERN_INFO "exceed the locked memory rlimit.");
+ return NULL;
+ }
+
+ info = kmem_cache_zalloc(ext_page_info_cache, GFP_KERNEL);
+
+ if (!info)
+ return NULL;
+
+ for (i = j = 0; i < count; i++) {
+ base = (unsigned long)iov[i].iov_base;
+ len = iov[i].iov_len;
+
+ if (!len)
+ continue;
+ n = ((base & ~PAGE_MASK) + len + ~PAGE_MASK) >> PAGE_SHIFT;
+
+ rc = get_user_pages_fast(base, n, npages ? 1 : 0,
+ &info->pages[j]);
+ if (rc != n)
+ goto failed;
+
+ while (n--) {
+ frags[j].offset = base & ~PAGE_MASK;
+ frags[j].size = min_t(int, len,
+ PAGE_SIZE - frags[j].offset);
+ len -= frags[j].size;
+ base += frags[j].size;
+ j++;
+ }
+ }
+
+#ifdef CONFIG_HIGHMEM
+ if (npages && !(dev->features & NETIF_F_HIGHDMA)) {
+ for (i = 0; i < j; i++) {
+ if (PageHighMem(info->pages[i]))
+ goto failed;
+ }
+ }
+#endif
+
+ info->total = total;
+ info->ext_page.dtor = page_dtor;
+ info->ext_page.page = info->pages[0];
+ info->ctor = ctor;
+ info->pnum = j;
+ info->iocb = iocb;
+ if (!npages)
+ info->flags = INFO_WRITE;
+ if (info->flags == INFO_READ) {
+ if (frags[0].offset == 0 && iocb->ki_iovec[0].iov_len) {
+ frags[0].offset = iocb->ki_iovec[0].iov_len;
+ ctor->port.vnet_hlen = iocb->ki_iovec[0].iov_len;
+ }
+ for (i = 0; i < j; i++)
+ mp_hash_insert(ctor, info->pages[i], info);
+ }
+ /* increment the number of locked pages */
+ ctor->lock_pages += j;
+ return info;
+
+failed:
+ for (i = 0; i < j; i++)
+ put_page(info->pages[i]);
+
+ kmem_cache_free(ext_page_info_cache, info);
+
+ return NULL;
+}
+
+static void mp_sock_destruct(struct sock *sk)
+{
+ struct mp_struct *mp = container_of(sk, struct mp_sock, sk)->mp;
+ kfree(mp);
+}
+
+static void mp_sock_state_change(struct sock *sk)
+{
+ if (sk_has_sleeper(sk))
+ wake_up_interruptible_sync_poll(sk->sk_sleep, POLLIN);
+}
+
+static void mp_sock_write_space(struct sock *sk)
+{
+ if (sk_has_sleeper(sk))
+ wake_up_interruptible_sync_poll(sk->sk_sleep, POLLOUT);
+}
+
+static void mp_sock_data_ready(struct sock *sk, int coming)
+{
+ struct mp_struct *mp = container_of(sk, struct mp_sock, sk)->mp;
+ struct page_ctor *ctor = NULL;
+ struct sk_buff *skb = NULL;
+ struct page_info *info = NULL;
+ int len;
+
+ struct virtio_net_hdr_mrg_rxbuf hdr = {
+ .hdr.flags = 0,
+ .hdr.gso_type = VIRTIO_NET_HDR_GSO_NONE
+ };
+
+ ctor = mp->ctor;
+ if (!ctor)
+ return;
+
+ while ((skb = skb_dequeue(&sk->sk_receive_queue)) != NULL) {
+ struct page *page;
+ int off;
+ int size = 0, i = 0;
+ struct skb_shared_info *shinfo = skb_shinfo(skb);
+
+ if (skb->ip_summed == CHECKSUM_COMPLETE)
+ DBG(KERN_INFO "csum %d\n", skb->ip_summed);
+
+ if (shinfo->nr_frags)
+ hdr.num_buffers = shinfo->nr_frags;
+ else
+ goto clean;
+
+ info = mp_hash_lookup(ctor, shinfo->frags[0].page);
+
+ skb_push(skb, ETH_HLEN);
+
+ if (skb_is_gso(skb)) {
+ hdr.hdr.hdr_len = skb_headlen(skb);
+ hdr.hdr.gso_size = shinfo->gso_size;
+ if (shinfo->gso_type & SKB_GSO_TCPV4)
+ hdr.hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV4;
+ else if (shinfo->gso_type & SKB_GSO_TCPV6)
+ hdr.hdr.gso_type = VIRTIO_NET_HDR_GSO_TCPV6;
+ else if (shinfo->gso_type & SKB_GSO_UDP)
+ hdr.hdr.gso_type = VIRTIO_NET_HDR_GSO_UDP;
+ else
+ BUG();
+ if (shinfo->gso_type & SKB_GSO_TCP_ECN)
+ hdr.hdr.gso_type |= VIRTIO_NET_HDR_GSO_ECN;
+
+ } else
+ hdr.hdr.gso_type = VIRTIO_NET_HDR_GSO_NONE;
+
+ if (skb->ip_summed == CHECKSUM_PARTIAL) {
+ hdr.hdr.flags = VIRTIO_NET_HDR_F_NEEDS_CSUM;
+ hdr.hdr.csum_start =
+ skb->csum_start - skb_headroom(skb);
+ hdr.hdr.csum_offset = skb->csum_offset;
+ }
+
+ info->skb = NULL;
+ off = info->hdr[0].iov_len;
+ len = memcpy_toiovec(info->iov, (unsigned char *)&hdr, off);
+ if (len) {
+ DBG(KERN_INFO
+ "Unable to write vnet_hdr at addr %p: %d\n",
+ info->iov_base, len);
+ goto clean;
+ }
+
+ memcpy_toiovec(info->iov, skb->data, skb_headlen(skb));
+
+ info->iocb->ki_left = hdr.num_buffers;
+ size = shinfo->frags[0].size +
+ shinfo->frags[0].page_offset - off;
+
+ create_iocb(info, off + size);
+
+ if (shinfo->nr_frags == 1)
+ info->skb = skb;
+ for (i = 1; i < shinfo->nr_frags; i++) {
+ page = shinfo->frags[i].page;
+ info = mp_hash_lookup(ctor, shinfo->frags[i].page);
+ info->skb = NULL;
+ create_iocb(info, shinfo->frags[i].size);
+ if (i == shinfo->nr_frags - 1)
+ info->skb = skb;
+ }
+
+ shinfo->nr_frags = 0;
+ shinfo->destructor_arg = NULL;
+ continue;
+clean:
+ kfree_skb(skb);
+ for (i = 0; info->pages[i]; i++)
+ put_page(info->pages[i]);
+ kmem_cache_free(ext_page_info_cache, info);
+ }
+ return;
+}
+
+static inline struct sk_buff *mp_alloc_skb(struct sock *sk, size_t prepad,
+ size_t len, size_t linear,
+ int noblock, int *err)
+{
+ struct sk_buff *skb;
+
+ /* Under a page? Don't bother with paged skb. */
+ if (prepad + len < PAGE_SIZE || !linear)
+ linear = len;
+
+ skb = sock_alloc_send_pskb(sk, prepad + linear, len - linear, noblock,
+ err);
+ if (!skb)
+ return NULL;
+
+ skb_reserve(skb, prepad);
+ skb_put(skb, linear);
+ skb->data_len = len - linear;
+ skb->len += len - linear;
+
+ return skb;
+}
+
+static int mp_skb_from_vnet_hdr(struct sk_buff *skb,
+ struct virtio_net_hdr *vnet_hdr)
+{
+ unsigned short gso_type = 0;
+ if (vnet_hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) {
+ switch (vnet_hdr->gso_type & ~VIRTIO_NET_HDR_GSO_ECN) {
+ case VIRTIO_NET_HDR_GSO_TCPV4:
+ gso_type = SKB_GSO_TCPV4;
+ break;
+ case VIRTIO_NET_HDR_GSO_TCPV6:
+ gso_type = SKB_GSO_TCPV6;
+ break;
+ case VIRTIO_NET_HDR_GSO_UDP:
+ gso_type = SKB_GSO_UDP;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ if (vnet_hdr->gso_type & VIRTIO_NET_HDR_GSO_ECN)
+ gso_type |= SKB_GSO_TCP_ECN;
+
+ if (vnet_hdr->gso_size == 0)
+ return -EINVAL;
+ }
+
+ if (vnet_hdr->flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) {
+ if (!skb_partial_csum_set(skb, vnet_hdr->csum_start,
+ vnet_hdr->csum_offset))
+ return -EINVAL;
+ }
+
+ if (vnet_hdr->gso_type != VIRTIO_NET_HDR_GSO_NONE) {
+ skb_shinfo(skb)->gso_size = vnet_hdr->gso_size;
+ skb_shinfo(skb)->gso_type = gso_type;
+
+ /* Header must be checked, and gso_segs computed. */
+ skb_shinfo(skb)->gso_type |= SKB_GSO_DODGY;
+ skb_shinfo(skb)->gso_segs = 0;
+ }
+ return 0;
+}
+
+static int mp_sendmsg(struct kiocb *iocb, struct socket *sock,
+ struct msghdr *m, size_t total_len)
+{
+ struct mp_struct *mp = container_of(sock->sk, struct mp_sock, sk)->mp;
+ struct virtio_net_hdr vnet_hdr = {0};
+ int hdr_len = 0;
+ struct page_ctor *ctor;
+ struct iovec *iov = m->msg_iov;
+ struct page_info *info = NULL;
+ struct frag frags[MAX_SKB_FRAGS];
+ struct sk_buff *skb;
+ int count = m->msg_iovlen;
+ int total = 0, header, n, i, len, rc;
+ unsigned long base;
+
+ ctor = mp->ctor;
+ if (!ctor)
+ return -ENODEV;
+
+ total = iov_length(iov, count);
+
+ if (total < ETH_HLEN)
+ return -EINVAL;
+
+ if (total <= COPY_THRESHOLD)
+ goto copy;
+
+ n = 0;
+ for (i = 0; i < count; i++) {
+ base = (unsigned long)iov[i].iov_base;
+ len = iov[i].iov_len;
+ if (!len)
+ continue;
+ n += ((base & ~PAGE_MASK) + len + ~PAGE_MASK) >> PAGE_SHIFT;
+ if (n > MAX_SKB_FRAGS)
+ return -EINVAL;
+ }
+
+copy:
+ hdr_len = sizeof(vnet_hdr);
+ if ((total - iocb->ki_iovec[0].iov_len) < 0)
+ return -EINVAL;
+
+ rc = memcpy_fromiovecend((void *)&vnet_hdr, iocb->ki_iovec, 0, hdr_len);
+ if (rc < 0)
+ return -EINVAL;
+
+ if ((vnet_hdr.flags & VIRTIO_NET_HDR_F_NEEDS_CSUM) &&
+ vnet_hdr.csum_start + vnet_hdr.csum_offset + 2 >
+ vnet_hdr.hdr_len)
+ vnet_hdr.hdr_len = vnet_hdr.csum_start +
+ vnet_hdr.csum_offset + 2;
+
+ if (vnet_hdr.hdr_len > total)
+ return -EINVAL;
+
+ header = total > COPY_THRESHOLD ? COPY_HDR_LEN : total;
+
+ skb = mp_alloc_skb(sock->sk, NET_IP_ALIGN, header,
+ iocb->ki_iovec[0].iov_len, 1, &rc);
+
+ if (!skb)
+ goto drop;
+
+ skb_set_network_header(skb, ETH_HLEN);
+ memcpy_fromiovec(skb->data, iov, header);
+
+ skb_reset_mac_header(skb);
+ skb->protocol = eth_hdr(skb)->h_proto;
+
+ rc = mp_skb_from_vnet_hdr(skb, &vnet_hdr);
+ if (rc)
+ goto drop;
+
+ if (header == total) {
+ rc = total;
+ info = alloc_small_page_info(ctor, iocb, total);
+ } else {
+ info = alloc_page_info(ctor, iocb, iov, count, frags, 0, total);
+ if (info)
+ for (i = 0; info->pages[i]; i++) {
+ skb_add_rx_frag(skb, i, info->pages[i],
+ frags[i].offset, frags[i].size);
+ info->pages[i] = NULL;
+ }
+ }
+ if (info != NULL) {
+ info->desc_pos = iocb->ki_pos;
+ info->total = total;
+ info->skb = skb;
+ skb_shinfo(skb)->destructor_arg = &info->ext_page;
+ skb->dev = mp->dev;
+ ctor->wq_len++;
+ create_iocb(info, info->total);
+ dev_queue_xmit(skb);
+ if (!ctor->rq_len)
+ sock->sk->sk_state_change(sock->sk);
+ return 0;
+ }
+drop:
+ kfree_skb(skb);
+ if (info) {
+ for (i = 0; info->pages[i]; i++)
+ put_page(info->pages[i]);
+ kmem_cache_free(ext_page_info_cache, info);
+ }
+ mp->dev->stats.tx_dropped++;
+ return -ENOMEM;
+}
+
+static int mp_recvmsg(struct kiocb *iocb, struct socket *sock,
+ struct msghdr *m, size_t total_len,
+ int flags)
+{
+ struct mp_struct *mp = container_of(sock->sk, struct mp_sock, sk)->mp;
+ struct page_ctor *ctor;
+ struct iovec *iov = m->msg_iov;
+ int count = m->msg_iovlen;
+ int npages, payload;
+ struct page_info *info;
+ struct frag frags[MAX_SKB_FRAGS];
+ unsigned long base;
+ int i, len;
+ unsigned long flag;
+
+ if (!(flags & MSG_DONTWAIT))
+ return -EINVAL;
+
+ ctor = mp->ctor;
+ if (!ctor)
+ return -EINVAL;
+
+ /* Error detections in case invalid external buffer */
+ if (count > 2 && iov[1].iov_len < ctor->port.hdr_len &&
+ mp->dev->features & NETIF_F_SG) {
+ return -EINVAL;
+ }
+
+ npages = ctor->port.npages;
+ payload = ctor->port.data_len;
+
+ /* If KVM guest virtio-net FE driver use SG feature */
+ if (count > 2) {
+ for (i = 2; i < count; i++) {
+ base = (unsigned long)iov[i].iov_base & ~PAGE_MASK;
+ len = iov[i].iov_len;
+ if (npages == 1)
+ len = min_t(int, len, PAGE_SIZE - base);
+ else if (base)
+ break;
+ payload -= len;
+ if (payload <= 0)
+ goto proceed;
+ if (npages == 1 || (len & ~PAGE_MASK))
+ break;
+ }
+ }
+
+ if ((((unsigned long)iov[1].iov_base & ~PAGE_MASK)
+ - NET_SKB_PAD - NET_IP_ALIGN) >= 0)
+ goto proceed;
+
+ return -EINVAL;
+
+proceed:
+ /* skip the virtnet head */
+ if (count > 1) {
+ iov++;
+ count--;
+ }
+
+ if (!ctor->lock_pages || !ctor->rq_len) {
+ set_memlock_rlimit(ctor, RLIMIT_MEMLOCK,
+ iocb->ki_user_data * 4096 * 2,
+ iocb->ki_user_data * 4096 * 2);
+ }
+
+ /* Translate address to kernel */
+ info = alloc_page_info(ctor, iocb, iov, count, frags, npages, 0);
+ if (!info)
+ return -ENOMEM;
+ info->len = total_len;
+ info->hdr[0].iov_base = iocb->ki_iovec[0].iov_base;
+ info->hdr[0].iov_len = iocb->ki_iovec[0].iov_len;
+ iocb->ki_iovec[0].iov_len = 0;
+ iocb->ki_left = 0;
+ info->offset = frags[0].offset;
+ info->desc_pos = iocb->ki_pos;
+
+ if (count > 1) {
+ iov--;
+ count++;
+ }
+
+ memcpy(info->iov, iov, sizeof(struct iovec) * count);
+
+ spin_lock_irqsave(&ctor->read_lock, flag);
+ list_add_tail(&info->list, &ctor->readq);
+ spin_unlock_irqrestore(&ctor->read_lock, flag);
+
+ ctor->rq_len++;
+
+ return 0;
+}
+
+/* Ops structure to mimic raw sockets with mp device */
+static const struct proto_ops mp_socket_ops = {
+ .sendmsg = mp_sendmsg,
+ .recvmsg = mp_recvmsg,
+};
+
+static struct proto mp_proto = {
+ .name = "mp",
+ .owner = THIS_MODULE,
+ .obj_size = sizeof(struct mp_sock),
+};
+
+static int mp_chr_open(struct inode *inode, struct file * file)
+{
+ struct mp_file *mfile;
+ cycle_kernel_lock();
+ DBG1(KERN_INFO "mp: mp_chr_open\n");
+
+ mfile = kzalloc(sizeof(*mfile), GFP_KERNEL);
+ if (!mfile)
+ return -ENOMEM;
+ atomic_set(&mfile->count, 0);
+ mfile->mp = NULL;
+ mfile->net = get_net(current->nsproxy->net_ns);
+ file->private_data = mfile;
+ return 0;
+}
+
+static int mp_attach(struct mp_struct *mp, struct file *file)
+{
+ struct mp_file *mfile = file->private_data;
+ int err;
+
+ netif_tx_lock_bh(mp->dev);
+
+ err = -EINVAL;
+
+ if (mfile->mp)
+ goto out;
+
+ err = -EBUSY;
+ if (mp->mfile)
+ goto out;
+
+ err = 0;
+ mfile->mp = mp;
+ mp->mfile = mfile;
+ mp->socket.file = file;
+ dev_hold(mp->dev);
+ sock_hold(mp->socket.sk);
+ atomic_inc(&mfile->count);
+
+out:
+ netif_tx_unlock_bh(mp->dev);
+ return err;
+}
+
+static int do_unbind(struct mp_file *mfile)
+{
+ struct mp_struct *mp = mp_get(mfile);
+
+ if (!mp)
+ return -EINVAL;
+
+ mp_detach(mp);
+ sock_put(mp->socket.sk);
+ mp_put(mfile);
+ return 0;
+}
+
+static long mp_chr_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ struct mp_file *mfile = file->private_data;
+ struct mp_struct *mp;
+ struct net_device *dev;
+ void __user* argp = (void __user *)arg;
+ struct ifreq ifr;
+ struct sock *sk;
+ int ret;
+
+ ret = -EINVAL;
+
+ switch (cmd) {
+ case MPASSTHRU_BINDDEV:
+ ret = -EFAULT;
+ if (copy_from_user(&ifr, argp, sizeof ifr))
+ break;
+
+ ifr.ifr_name[IFNAMSIZ-1] = '\0';
+
+ ret = -ENODEV;
+ dev = dev_get_by_name(mfile->net, ifr.ifr_name);
+ if (!dev)
+ break;
+
+ mutex_lock(&mp_mutex);
+
+ ret = -EBUSY;
+
+ /* the device can be only bind once */
+ if (dev_is_mpassthru(dev))
+ goto err_dev_put;
+
+ mp = mfile->mp;
+ if (mp)
+ goto err_dev_put;
+
+ mp = kzalloc(sizeof(*mp), GFP_KERNEL);
+ if (!mp) {
+ ret = -ENOMEM;
+ goto err_dev_put;
+ }
+ mp->dev = dev;
+ ret = -ENOMEM;
+
+ sk = sk_alloc(mfile->net, AF_UNSPEC, GFP_KERNEL, &mp_proto);
+ if (!sk)
+ goto err_free_mp;
+
+ init_waitqueue_head(&mp->socket.wait);
+ mp->socket.ops = &mp_socket_ops;
+ sock_init_data(&mp->socket, sk);
+ sk->sk_sndbuf = INT_MAX;
+ container_of(sk, struct mp_sock, sk)->mp = mp;
+
+ sk->sk_destruct = mp_sock_destruct;
+ sk->sk_data_ready = mp_sock_data_ready;
+ sk->sk_write_space = mp_sock_write_space;
+ sk->sk_state_change = mp_sock_state_change;
+ ret = mp_attach(mp, file);
+ if (ret < 0)
+ goto err_free_sk;
+
+ ret = page_ctor_attach(mp);
+ if (ret < 0)
+ goto err_free_sk;
+ mp_dev_change_flags(mp->dev, mp->dev->flags & (~IFF_UP));
+ mp_dev_change_flags(mp->dev, mp->dev->flags | IFF_UP);
+ sk->sk_state_change(sk);
+out:
+ mutex_unlock(&mp_mutex);
+ break;
+err_free_sk:
+ sk_free(sk);
+err_free_mp:
+ kfree(mp);
+err_dev_put:
+ dev_put(dev);
+ goto out;
+
+ case MPASSTHRU_UNBINDDEV:
+ ret = do_unbind(mfile);
+ break;
+
+ default:
+ break;
+ }
+ return ret;
+}
+
+static unsigned int mp_chr_poll(struct file *file, poll_table * wait)
+{
+ struct mp_file *mfile = file->private_data;
+ struct mp_struct *mp = mp_get(mfile);
+ struct sock *sk;
+ unsigned int mask = 0;
+
+ if (!mp)
+ return POLLERR;
+
+ sk = mp->socket.sk;
+
+ poll_wait(file, &mp->socket.wait, wait);
+
+ if (!skb_queue_empty(&sk->sk_receive_queue))
+ mask |= POLLIN | POLLRDNORM;
+
+ if (sock_writeable(sk) ||
+ (!test_and_set_bit(SOCK_ASYNC_NOSPACE, &sk->sk_socket->flags) &&
+ sock_writeable(sk)))
+ mask |= POLLOUT | POLLWRNORM;
+
+ if (mp->dev->reg_state != NETREG_REGISTERED)
+ mask = POLLERR;
+
+ mp_put(mfile);
+ return mask;
+}
+
+static ssize_t mp_chr_aio_write(struct kiocb *iocb, const struct iovec *iov,
+ unsigned long count, loff_t pos)
+{
+ struct file *file = iocb->ki_filp;
+ struct mp_struct *mp = mp_get(file->private_data);
+ struct sock *sk = mp->socket.sk;
+ struct sk_buff *skb;
+ int len, err;
+ ssize_t result = 0;
+
+ if (!mp)
+ return -EBADFD;
+
+ /* currently, async is not supported.
+ * but we may support real async aio from user application,
+ * maybe qemu virtio-net backend.
+ */
+ if (!is_sync_kiocb(iocb))
+ return -EFAULT;
+
+ len = iov_length(iov, count);
+
+ if (unlikely(len) < ETH_HLEN)
+ return -EINVAL;
+
+ skb = sock_alloc_send_skb(sk, len + NET_IP_ALIGN,
+ file->f_flags & O_NONBLOCK, &err);
+
+ if (!skb)
+ return -EFAULT;
+
+ skb_reserve(skb, NET_IP_ALIGN);
+ skb_put(skb, len);
+
+ if (skb_copy_datagram_from_iovec(skb, 0, iov, 0, len)) {
+ kfree_skb(skb);
+ return -EAGAIN;
+ }
+
+ skb->protocol = eth_type_trans(skb, mp->dev);
+ skb->dev = mp->dev;
+
+ dev_queue_xmit(skb);
+
+ mp_put(file->private_data);
+ return result;
+}
+
+static int mp_chr_close(struct inode *inode, struct file *file)
+{
+ struct mp_file *mfile = file->private_data;
+
+ /*
+ * Ignore return value since an error only means there was nothing to
+ * do
+ */
+ do_unbind(mfile);
+
+ put_net(mfile->net);
+ kfree(mfile);
+
+ return 0;
+}
+
+#ifdef CONFIG_COMPAT
+static long mp_chr_compat_ioctl(struct file *f, unsigned int ioctl,
+ unsigned long arg)
+{
+ return mp_chr_ioctl(f, ioctl, (unsigned long)compat_ptr(arg));
+}
+#endif
+
+static const struct file_operations mp_fops = {
+ .owner = THIS_MODULE,
+ .llseek = no_llseek,
+ .write = do_sync_write,
+ .aio_write = mp_chr_aio_write,
+ .poll = mp_chr_poll,
+ .unlocked_ioctl = mp_chr_ioctl,
+#ifdef CONFIG_COMPAT
+ .compat_ioctl = mp_chr_compat_ioctl,
+#endif
+ .open = mp_chr_open,
+ .release = mp_chr_close,
+};
+
+static struct miscdevice mp_miscdev = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "mp",
+ .nodename = "net/mp",
+ .fops = &mp_fops,
+};
+
+static int mp_device_event(struct notifier_block *unused,
+ unsigned long event, void *ptr)
+{
+ struct net_device *dev = ptr;
+ struct mpassthru_port *port;
+ struct mp_struct *mp = NULL;
+ struct socket *sock = NULL;
+ struct sock *sk;
+
+ port = dev->mp_port;
+ if (port == NULL)
+ return NOTIFY_DONE;
+
+ switch (event) {
+ case NETDEV_UNREGISTER:
+ sock = dev->mp_port->sock;
+ mp = container_of(sock->sk, struct mp_sock, sk)->mp;
+ do_unbind(mp->mfile);
+ break;
+ case NETDEV_CHANGE:
+ sk = dev->mp_port->sock->sk;
+ sk->sk_state_change(sk);
+ break;
+ }
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block mp_notifier_block __read_mostly = {
+ .notifier_call = mp_device_event,
+};
+
+static int mp_init(void)
+{
+ int err = 0;
+
+ ext_page_info_cache = kmem_cache_create("skb_page_info",
+ sizeof(struct page_info),
+ 0, SLAB_HWCACHE_ALIGN, NULL);
+ if (!ext_page_info_cache)
+ return -ENOMEM;
+
+ err = misc_register(&mp_miscdev);
+ if (err) {
+ printk(KERN_ERR "mp: Can't register misc device\n");
+ kmem_cache_destroy(ext_page_info_cache);
+ } else {
+ printk(KERN_INFO "Registering mp misc device - minor = %d\n",
+ mp_miscdev.minor);
+ register_netdevice_notifier(&mp_notifier_block);
+ }
+ return err;
+}
+
+void mp_exit(void)
+{
+ unregister_netdevice_notifier(&mp_notifier_block);
+ misc_deregister(&mp_miscdev);
+ kmem_cache_destroy(ext_page_info_cache);
+}
+
+/* Get an underlying socket object from mp file. Returns error unless file is
+ * attached to a device. The returned object works like a packet socket, it
+ * can be used for sock_sendmsg/sock_recvmsg. The caller is responsible for
+ * holding a reference to the file for as long as the socket is in use. */
+struct socket *mp_get_socket(struct file *file)
+{
+ struct mp_file *mfile = file->private_data;
+ struct mp_struct *mp;
+
+ if (file->f_op != &mp_fops)
+ return ERR_PTR(-EINVAL);
+ mp = mp_get(mfile);
+ if (!mp)
+ return ERR_PTR(-EBADFD);
+ mp_put(mfile);
+ return &mp->socket;
+}
+EXPORT_SYMBOL_GPL(mp_get_socket);
+
+module_init(mp_init);
+module_exit(mp_exit);
+MODULE_AUTHOR(DRV_COPYRIGHT);
+MODULE_DESCRIPTION(DRV_DESCRIPTION);
+MODULE_LICENSE("GPL v2");
--
1.5.4.4
^ permalink raw reply related
* [RFC PATCH v8 14/16] Provides multiple submits and asynchronous notifications.
From: xiaohui.xin @ 2010-07-29 11:14 UTC (permalink / raw)
To: netdev, kvm, linux-kernel, mst, mingo, davem, herbert, jdike; +Cc: Xin Xiaohui
In-Reply-To: <1280402088-5849-14-git-send-email-xiaohui.xin@intel.com>
From: Xin Xiaohui <xiaohui.xin@intel.com>
The vhost-net backend now only supports synchronous send/recv
operations. The patch provides multiple submits and asynchronous
notifications. This is needed for zero-copy case.
Signed-off-by: Xin Xiaohui <xiaohui.xin@intel.com>
---
drivers/vhost/net.c | 348 +++++++++++++++++++++++++++++++++++++++++++++----
drivers/vhost/vhost.c | 79 +++++++++++
drivers/vhost/vhost.h | 15 ++
3 files changed, 414 insertions(+), 28 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index b38abc6..c4bc815 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -24,6 +24,8 @@
#include <linux/if_arp.h>
#include <linux/if_tun.h>
#include <linux/if_macvlan.h>
+#include <linux/mpassthru.h>
+#include <linux/aio.h>
#include <net/sock.h>
@@ -39,6 +41,8 @@ enum {
VHOST_NET_VQ_MAX = 2,
};
+static struct kmem_cache *notify_cache;
+
enum vhost_net_poll_state {
VHOST_NET_POLL_DISABLED = 0,
VHOST_NET_POLL_STARTED = 1,
@@ -49,6 +53,7 @@ struct vhost_net {
struct vhost_dev dev;
struct vhost_virtqueue vqs[VHOST_NET_VQ_MAX];
struct vhost_poll poll[VHOST_NET_VQ_MAX];
+ struct kmem_cache *cache;
/* Tells us whether we are polling a socket for TX.
* We only do this when socket buffer fills up.
* Protected by tx vq lock. */
@@ -93,11 +98,190 @@ static void tx_poll_start(struct vhost_net *net, struct socket *sock)
net->tx_poll_state = VHOST_NET_POLL_STARTED;
}
+struct kiocb *notify_dequeue(struct vhost_virtqueue *vq)
+{
+ struct kiocb *iocb = NULL;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vq->notify_lock, flags);
+ if (!list_empty(&vq->notifier)) {
+ iocb = list_first_entry(&vq->notifier,
+ struct kiocb, ki_list);
+ list_del(&iocb->ki_list);
+ }
+ spin_unlock_irqrestore(&vq->notify_lock, flags);
+ return iocb;
+}
+
+static void handle_iocb(struct kiocb *iocb)
+{
+ struct vhost_virtqueue *vq = iocb->private;
+ unsigned long flags;
+
+ spin_lock_irqsave(&vq->notify_lock, flags);
+ list_add_tail(&iocb->ki_list, &vq->notifier);
+ spin_unlock_irqrestore(&vq->notify_lock, flags);
+}
+
+static int is_async_vq(struct vhost_virtqueue *vq)
+{
+ return (vq->link_state == VHOST_VQ_LINK_ASYNC);
+}
+
+static void handle_async_rx_events_notify(struct vhost_net *net,
+ struct vhost_virtqueue *vq,
+ struct socket *sock)
+{
+ struct kiocb *iocb = NULL;
+ struct vhost_log *vq_log = NULL;
+ int rx_total_len = 0;
+ unsigned int head, log, in, out;
+ int size;
+ int count;
+
+ struct virtio_net_hdr_mrg_rxbuf hdr = {
+ .hdr.flags = 0,
+ .hdr.gso_type = VIRTIO_NET_HDR_GSO_NONE
+ };
+
+ if (!is_async_vq(vq))
+ return;
+
+ if (sock->sk->sk_data_ready)
+ sock->sk->sk_data_ready(sock->sk, 0);
+
+ vq_log = unlikely(vhost_has_feature(&net->dev, VHOST_F_LOG_ALL)) ?
+ vq->log : NULL;
+
+ while ((iocb = notify_dequeue(vq)) != NULL) {
+ if (!iocb->ki_left) {
+ vhost_add_used_and_signal(&net->dev, vq,
+ iocb->ki_pos, iocb->ki_nbytes);
+ size = iocb->ki_nbytes;
+ head = iocb->ki_pos;
+ rx_total_len += iocb->ki_nbytes;
+
+ if (iocb->ki_dtor)
+ iocb->ki_dtor(iocb);
+ kmem_cache_free(net->cache, iocb);
+
+ /* when log is enabled, recomputing the log is needed,
+ * since these buffers are in async queue, may not get
+ * the log info before.
+ */
+ if (unlikely(vq_log)) {
+ if (!log)
+ __vhost_get_desc(&net->dev, vq, vq->iov,
+ ARRAY_SIZE(vq->iov),
+ &out, &in, vq_log,
+ &log, head);
+ vhost_log_write(vq, vq_log, log, size);
+ }
+ if (unlikely(rx_total_len >= VHOST_NET_WEIGHT)) {
+ vhost_poll_queue(&vq->poll);
+ break;
+ }
+ } else {
+ int i = 0;
+ int count = iocb->ki_left;
+ int hc = count;
+ while (count--) {
+ if (iocb) {
+ vq->heads[i].id = iocb->ki_pos;
+ vq->heads[i].len = iocb->ki_nbytes;
+ size = iocb->ki_nbytes;
+ head = iocb->ki_pos;
+ rx_total_len += iocb->ki_nbytes;
+
+ if (iocb->ki_dtor)
+ iocb->ki_dtor(iocb);
+ kmem_cache_free(net->cache, iocb);
+
+ if (unlikely(vq_log)) {
+ if (!log)
+ __vhost_get_desc(
+ &net->dev, vq, vq->iov,
+ ARRAY_SIZE(vq->iov),
+ &out, &in, vq_log,
+ &log, head);
+ vhost_log_write(
+ vq, vq_log, log, size);
+ }
+ } else
+ break;
+
+ i++;
+ iocb == NULL;
+ if (count)
+ iocb = notify_dequeue(vq);
+ }
+ vhost_add_used_and_signal_n(
+ &net->dev, vq, vq->heads, hc);
+ }
+ }
+}
+
+static void handle_async_tx_events_notify(struct vhost_net *net,
+ struct vhost_virtqueue *vq)
+{
+ struct kiocb *iocb = NULL;
+ struct list_head *entry, *tmp;
+ unsigned long flags;
+ int tx_total_len = 0;
+
+ if (!is_async_vq(vq))
+ return;
+
+ spin_lock_irqsave(&vq->notify_lock, flags);
+ list_for_each_safe(entry, tmp, &vq->notifier) {
+ iocb = list_entry(entry,
+ struct kiocb, ki_list);
+ if (!iocb->ki_flags)
+ continue;
+ list_del(&iocb->ki_list);
+ vhost_add_used_and_signal(&net->dev, vq,
+ iocb->ki_pos, 0);
+ tx_total_len += iocb->ki_nbytes;
+
+ if (iocb->ki_dtor)
+ iocb->ki_dtor(iocb);
+
+ kmem_cache_free(net->cache, iocb);
+ if (unlikely(tx_total_len >= VHOST_NET_WEIGHT)) {
+ vhost_poll_queue(&vq->poll);
+ break;
+ }
+ }
+ spin_unlock_irqrestore(&vq->notify_lock, flags);
+}
+
+static struct kiocb *create_iocb(struct vhost_net *net,
+ struct vhost_virtqueue *vq,
+ unsigned head)
+{
+ struct kiocb *iocb = NULL;
+
+ if (!is_async_vq(vq))
+ return NULL;
+
+ iocb = kmem_cache_zalloc(net->cache, GFP_KERNEL);
+ if (!iocb)
+ return NULL;
+ iocb->private = vq;
+ iocb->ki_pos = head;
+ iocb->ki_dtor = handle_iocb;
+ if (vq == &net->dev.vqs[VHOST_NET_VQ_RX])
+ iocb->ki_user_data = vq->num;
+ iocb->ki_iovec = vq->hdr;
+ return iocb;
+}
+
/* Expects to be always run from workqueue - which acts as
* read-size critical section for our kind of RCU. */
static void handle_tx(struct vhost_net *net)
{
struct vhost_virtqueue *vq = &net->dev.vqs[VHOST_NET_VQ_TX];
+ struct kiocb *iocb = NULL;
unsigned head, out, in, s;
struct msghdr msg = {
.msg_name = NULL,
@@ -130,6 +314,8 @@ static void handle_tx(struct vhost_net *net)
tx_poll_stop(net);
vhost_hlen = vq->vhost_hlen;
+ handle_async_tx_events_notify(net, vq);
+
for (;;) {
head = vhost_get_desc(&net->dev, vq, vq->iov,
ARRAY_SIZE(vq->iov),
@@ -138,10 +324,13 @@ static void handle_tx(struct vhost_net *net)
/* Nothing new? Wait for eventfd to tell us they refilled. */
if (head == vq->num) {
wmem = atomic_read(&sock->sk->sk_wmem_alloc);
- if (wmem >= sock->sk->sk_sndbuf * 3 / 4) {
- tx_poll_start(net, sock);
- set_bit(SOCK_ASYNC_NOSPACE, &sock->flags);
- break;
+ if (!is_async_vq(vq)) {
+ if (wmem >= sock->sk->sk_sndbuf * 3 / 4) {
+ tx_poll_start(net, sock);
+ set_bit(SOCK_ASYNC_NOSPACE,
+ &sock->flags);
+ break;
+ }
}
if (unlikely(vhost_enable_notify(vq))) {
vhost_disable_notify(vq);
@@ -158,6 +347,13 @@ static void handle_tx(struct vhost_net *net)
s = move_iovec_hdr(vq->iov, vq->hdr, vhost_hlen, out);
msg.msg_iovlen = out;
len = iov_length(vq->iov, out);
+
+ if (is_async_vq(vq)) {
+ iocb = create_iocb(net, vq, head);
+ if (!iocb)
+ break;
+ }
+
/* Sanity check */
if (!len) {
vq_err(vq, "Unexpected header len for TX: "
@@ -166,12 +362,18 @@ static void handle_tx(struct vhost_net *net)
break;
}
/* TODO: Check specific error and bomb out unless ENOBUFS? */
- err = sock->ops->sendmsg(NULL, sock, &msg, len);
+ err = sock->ops->sendmsg(iocb, sock, &msg, len);
if (unlikely(err < 0)) {
+ if (is_async_vq(vq))
+ kmem_cache_free(net->cache, iocb);
vhost_discard_desc(vq, 1);
tx_poll_start(net, sock);
break;
}
+
+ if (is_async_vq(vq))
+ continue;
+
if (err != len)
pr_err("Truncated TX packet: "
" len %d != %zd\n", err, len);
@@ -183,6 +385,8 @@ static void handle_tx(struct vhost_net *net)
}
}
+ handle_async_tx_events_notify(net, vq);
+
mutex_unlock(&vq->mutex);
unuse_mm(net->dev.mm);
}
@@ -205,7 +409,8 @@ static int vhost_head_len(struct vhost_virtqueue *vq, struct sock *sk)
static void handle_rx(struct vhost_net *net)
{
struct vhost_virtqueue *vq = &net->dev.vqs[VHOST_NET_VQ_RX];
- unsigned in, log, s;
+ struct kiocb *iocb = NULL;
+ unsigned in, out, log, s;
struct vhost_log *vq_log;
struct msghdr msg = {
.msg_name = NULL,
@@ -225,25 +430,42 @@ static void handle_rx(struct vhost_net *net)
int err, headcount, datalen;
size_t vhost_hlen;
struct socket *sock = rcu_dereference(vq->private_data);
- if (!sock || skb_queue_empty(&sock->sk->sk_receive_queue))
+ if (!sock || (skb_queue_empty(&sock->sk->sk_receive_queue) &&
+ !is_async_vq(vq)))
return;
-
use_mm(net->dev.mm);
mutex_lock(&vq->mutex);
vhost_disable_notify(vq);
vhost_hlen = vq->vhost_hlen;
+ /* In async cases, when write log is enabled, in case the submitted
+ * buffers did not get log info before the log enabling, so we'd
+ * better recompute the log info when needed. We do this in
+ * handle_async_rx_events_notify().
+ */
+
vq_log = unlikely(vhost_has_feature(&net->dev, VHOST_F_LOG_ALL)) ?
vq->log : NULL;
- while ((datalen = vhost_head_len(vq, sock->sk))) {
- headcount = vhost_get_desc_n(vq, vq->heads,
- datalen + vhost_hlen,
- &in, vq_log, &log);
+ handle_async_rx_events_notify(net, vq, sock);
+
+ while (is_async_vq(vq) ||
+ (datalen = vhost_head_len(vq, sock->sk)) != 0) {
+ if (is_async_vq(vq))
+ headcount =
+ vhost_get_desc(&net->dev, vq, vq->iov,
+ ARRAY_SIZE(vq->iov),
+ &out, &in,
+ vq->log, &log);
+ else
+ headcount = vhost_get_desc_n(vq, vq->heads,
+ datalen + vhost_hlen,
+ &in, vq_log, &log);
if (headcount < 0)
break;
/* OK, now we need to know about added descriptors. */
- if (!headcount) {
+ if ((!headcount && !is_async_vq(vq)) ||
+ (headcount == vq->num && is_async_vq(vq))) {
if (unlikely(vhost_enable_notify(vq))) {
/* They have slipped one in as we were
* doing that: check again. */
@@ -256,7 +478,12 @@ static void handle_rx(struct vhost_net *net)
}
/* We don't need to be notified again. */
/* Skip header. TODO: support TSO. */
+ if (is_async_vq(vq) && vhost_hlen == sizeof(hdr)) {
+ vq->hdr[0].iov_len = vhost_hlen;
+ goto nomove;
+ }
s = move_iovec_hdr(vq->iov, vq->hdr, vhost_hlen, in);
+nomove:
msg.msg_iovlen = in;
len = iov_length(vq->iov, in);
/* Sanity check */
@@ -266,13 +493,23 @@ static void handle_rx(struct vhost_net *net)
iov_length(vq->hdr, s), vhost_hlen);
break;
}
- err = sock->ops->recvmsg(NULL, sock, &msg,
+ if (is_async_vq(vq)) {
+ iocb = create_iocb(net, vq, headcount);
+ if (!iocb)
+ break;
+ }
+ err = sock->ops->recvmsg(iocb, sock, &msg,
len, MSG_DONTWAIT | MSG_TRUNC);
/* TODO: Check specific error and bomb out unless EAGAIN? */
if (err < 0) {
+ if (is_async_vq(vq))
+ kmem_cache_free(net->cache, iocb);
vhost_discard_desc(vq, headcount);
break;
}
+ if (is_async_vq(vq))
+ continue;
+
if (err != datalen) {
pr_err("Discarded rx packet: "
" len %d, expected %zd\n", err, datalen);
@@ -280,6 +517,9 @@ static void handle_rx(struct vhost_net *net)
continue;
}
len = err;
+ if (vhost_has_feature(&net->dev, VIRTIO_NET_F_MRG_RXBUF))
+ hdr.num_buffers = headcount;
+
err = memcpy_toiovec(vq->hdr, (unsigned char *)&hdr,
vhost_hlen);
if (err) {
@@ -287,18 +527,7 @@ static void handle_rx(struct vhost_net *net)
vq->iov->iov_base, err);
break;
}
- /* TODO: Should check and handle checksum. */
- if (vhost_has_feature(&net->dev, VIRTIO_NET_F_MRG_RXBUF)) {
- struct iovec *iov = vhost_hlen ? vq->hdr : vq->iov;
-
- if (memcpy_toiovecend(iov, (unsigned char *)&headcount,
- offsetof(typeof(hdr), num_buffers),
- sizeof(hdr.num_buffers))) {
- vq_err(vq, "Failed num_buffers write");
- vhost_discard_desc(vq, headcount);
- break;
- }
- }
+
len += vhost_hlen;
vhost_add_used_and_signal_n(&net->dev, vq, vq->heads,
headcount);
@@ -311,6 +540,8 @@ static void handle_rx(struct vhost_net *net)
}
}
+ handle_async_rx_events_notify(net, vq, sock);
+
mutex_unlock(&vq->mutex);
unuse_mm(net->dev.mm);
}
@@ -364,6 +595,7 @@ static int vhost_net_open(struct inode *inode, struct file *f)
vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, POLLOUT);
vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, POLLIN);
n->tx_poll_state = VHOST_NET_POLL_DISABLED;
+ n->cache = NULL;
f->private_data = n;
@@ -427,6 +659,21 @@ static void vhost_net_flush(struct vhost_net *n)
vhost_net_flush_vq(n, VHOST_NET_VQ_RX);
}
+static void vhost_async_cleanup(struct vhost_net *n)
+{
+ /* clean the notifier */
+ struct vhost_virtqueue *vq;
+ struct kiocb *iocb = NULL;
+ if (n->cache) {
+ vq = &n->dev.vqs[VHOST_NET_VQ_RX];
+ while ((iocb = notify_dequeue(vq)) != NULL)
+ kmem_cache_free(n->cache, iocb);
+ vq = &n->dev.vqs[VHOST_NET_VQ_TX];
+ while ((iocb = notify_dequeue(vq)) != NULL)
+ kmem_cache_free(n->cache, iocb);
+ }
+}
+
static int vhost_net_release(struct inode *inode, struct file *f)
{
struct vhost_net *n = f->private_data;
@@ -443,6 +690,7 @@ static int vhost_net_release(struct inode *inode, struct file *f)
/* We do an extra flush before freeing memory,
* since jobs can re-queue themselves. */
vhost_net_flush(n);
+ vhost_async_cleanup(n);
kfree(n);
return 0;
}
@@ -494,21 +742,58 @@ static struct socket *get_tap_socket(int fd)
return sock;
}
-static struct socket *get_socket(int fd)
+static struct socket *get_mp_socket(int fd)
+{
+ struct file *file = fget(fd);
+ struct socket *sock;
+ if (!file)
+ return ERR_PTR(-EBADF);
+ sock = mp_get_socket(file);
+ if (IS_ERR(sock))
+ fput(file);
+ return sock;
+}
+
+static struct socket *get_socket(struct vhost_virtqueue *vq, int fd,
+ enum vhost_vq_link_state *state)
{
struct socket *sock;
/* special case to disable backend */
if (fd == -1)
return NULL;
+
+ *state = VHOST_VQ_LINK_SYNC;
+
sock = get_raw_socket(fd);
if (!IS_ERR(sock))
return sock;
sock = get_tap_socket(fd);
if (!IS_ERR(sock))
return sock;
+ /* If we dont' have notify_cache, then dont do mpassthru */
+ if (!notify_cache)
+ return ERR_PTR(-ENOTSOCK);
+ sock = get_mp_socket(fd);
+ if (!IS_ERR(sock)) {
+ *state = VHOST_VQ_LINK_ASYNC;
+ return sock;
+ }
return ERR_PTR(-ENOTSOCK);
}
+static void vhost_init_link_state(struct vhost_net *n, int index)
+{
+ struct vhost_virtqueue *vq = n->vqs + index;
+
+ WARN_ON(!mutex_is_locked(&vq->mutex));
+ if (vq->link_state == VHOST_VQ_LINK_ASYNC) {
+ INIT_LIST_HEAD(&vq->notifier);
+ spin_lock_init(&vq->notify_lock);
+ if (!n->cache)
+ n->cache = notify_cache;
+ }
+}
+
static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd)
{
struct socket *sock, *oldsock;
@@ -532,12 +817,14 @@ static long vhost_net_set_backend(struct vhost_net *n, unsigned index, int fd)
r = -EFAULT;
goto err_vq;
}
- sock = get_socket(fd);
+ sock = get_socket(vq, fd, &vq->link_state);
if (IS_ERR(sock)) {
r = PTR_ERR(sock);
goto err_vq;
}
+ vhost_init_link_state(n, index);
+
/* start polling new socket */
oldsock = vq->private_data;
if (sock == oldsock)
@@ -687,6 +974,9 @@ int vhost_net_init(void)
r = misc_register(&vhost_net_misc);
if (r)
goto err_reg;
+ notify_cache = kmem_cache_create("vhost_kiocb",
+ sizeof(struct kiocb), 0,
+ SLAB_HWCACHE_ALIGN, NULL);
return 0;
err_reg:
vhost_cleanup();
@@ -700,6 +990,8 @@ void vhost_net_exit(void)
{
misc_deregister(&vhost_net_misc);
vhost_cleanup();
+ if (notify_cache)
+ kmem_cache_destroy(notify_cache);
}
module_exit(vhost_net_exit);
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index 118c8e0..66ff5c5 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -909,6 +909,85 @@ err:
return r;
}
+unsigned __vhost_get_desc(struct vhost_dev *dev, struct vhost_virtqueue *vq,
+ struct iovec iov[], unsigned int iov_size,
+ unsigned int *out_num, unsigned int *in_num,
+ struct vhost_log *log, unsigned int *log_num,
+ unsigned int head)
+{
+ struct vring_desc desc;
+ unsigned int i, found = 0;
+ u16 last_avail_idx;
+ int ret;
+
+ /* When we start there are none of either input nor output. */
+ *out_num = *in_num = 0;
+ if (unlikely(log))
+ *log_num = 0;
+
+ i = head;
+ do {
+ unsigned iov_count = *in_num + *out_num;
+ if (i >= vq->num) {
+ vq_err(vq, "Desc index is %u > %u, head = %u",
+ i, vq->num, head);
+ return vq->num;
+ }
+ if (++found > vq->num) {
+ vq_err(vq, "Loop detected: last one at %u "
+ "vq size %u head %u\n",
+ i, vq->num, head);
+ return vq->num;
+ }
+ ret = copy_from_user(&desc, vq->desc + i, sizeof desc);
+ if (ret) {
+ vq_err(vq, "Failed to get descriptor: idx %d addr %p\n",
+ i, vq->desc + i);
+ return vq->num;
+ }
+ if (desc.flags & VRING_DESC_F_INDIRECT) {
+ ret = get_indirect(dev, vq, iov, iov_size,
+ out_num, in_num,
+ log, log_num, &desc);
+ if (ret < 0) {
+ vq_err(vq, "Failure detected "
+ "in indirect descriptor at idx %d\n", i);
+ return vq->num;
+ }
+ continue;
+ }
+
+ ret = translate_desc(dev, desc.addr, desc.len, iov + iov_count,
+ iov_size - iov_count);
+ if (ret < 0) {
+ vq_err(vq, "Translation failure %d descriptor idx %d\n",
+ ret, i);
+ return vq->num;
+ }
+ if (desc.flags & VRING_DESC_F_WRITE) {
+ /* If this is an input descriptor,
+ * increment that count. */
+ *in_num += ret;
+ if (unlikely(log)) {
+ log[*log_num].addr = desc.addr;
+ log[*log_num].len = desc.len;
+ ++*log_num;
+ }
+ } else {
+ /* If it's an output descriptor, they're all supposed
+ * to come before any input descriptors. */
+ if (*in_num) {
+ vq_err(vq, "Descriptor has out after in: "
+ "idx %d\n", i);
+ return vq->num;
+ }
+ *out_num += ret;
+ }
+ } while ((i = next_desc(&desc)) != -1);
+
+ return head;
+}
+
/* This looks in the virtqueue and for the first available buffer, and converts
* it to an iovec for convenient access. Since descriptors consist of some
* number of output then some number of input descriptors, it's actually two
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index 08d740a..54c6d0b 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -43,6 +43,11 @@ struct vhost_log {
u64 len;
};
+enum vhost_vq_link_state {
+ VHOST_VQ_LINK_SYNC = 0,
+ VHOST_VQ_LINK_ASYNC = 1,
+};
+
/* The virtqueue structure describes a queue attached to a device. */
struct vhost_virtqueue {
struct vhost_dev *dev;
@@ -98,6 +103,10 @@ struct vhost_virtqueue {
/* Log write descriptors */
void __user *log_base;
struct vhost_log log[VHOST_NET_MAX_SG];
+ /* Differiate async socket for 0-copy from normal */
+ enum vhost_vq_link_state link_state;
+ struct list_head notifier;
+ spinlock_t notify_lock;
};
struct vhost_dev {
@@ -125,6 +134,11 @@ int vhost_log_access_ok(struct vhost_dev *);
int vhost_get_desc_n(struct vhost_virtqueue *, struct vring_used_elem *heads,
int datalen, unsigned int *iovcount, struct vhost_log *log,
unsigned int *log_num);
+unsigned __vhost_get_desc(struct vhost_dev *, struct vhost_virtqueue *,
+ struct iovec iov[], unsigned int iov_count,
+ unsigned int *out_num, unsigned int *in_num,
+ struct vhost_log *log, unsigned int *log_num,
+ unsigned int head);
unsigned vhost_get_desc(struct vhost_dev *, struct vhost_virtqueue *,
struct iovec iov[], unsigned int iov_count,
unsigned int *out_num, unsigned int *in_num,
@@ -165,6 +179,7 @@ enum {
static inline int vhost_has_feature(struct vhost_dev *dev, int bit)
{
unsigned acked_features = rcu_dereference(dev->acked_features);
+ acked_features |= (1 << VIRTIO_NET_F_MRG_RXBUF);
return acked_features & (1 << bit);
}
--
1.5.4.4
^ permalink raw reply related
* Re: can: expected receive behavior broken
From: Oliver Hartkopp @ 2010-07-29 11:16 UTC (permalink / raw)
To: Matthias Fuchs
Cc: Socketcan-core-0fE9KPoRgkgATYTw5x5z8w, Linux Netdev List,
Wolfgang Grandegger
In-Reply-To: <4C514BAE.4070201-fJ+pQTUTwRTk1uMJSBkQmQ@public.gmane.org>
On 29.07.2010 11:36, Oliver Hartkopp wrote:
> Invoking tst-rcv-own-msgs produces this output, which is far away from the
> correct (wanted) output seen in the commit message below.
>
> sockopt default
> s : 0
> t : 0
> timeout
> sockopt - -
> timeout
> sockopt - R
> timeout
> sockopt L -
> s : 3
> t : 3
> timeout
> sockopt L R
> s : 4
> t : 4
> timeout
> done.
>
> I'll check that with the latest linux-2.6 (after rebooting :-)
Grr - it is also broken in 2.6.34-rc7-05126-g278554b ...
I'll go and hunt the bad commit in 2.6.34 and will fix it.
Tnx & regards,
Oliver
> It needs a vcan0 virtual CAN network interface and should produce an output
> like this, when invoked:
>
> sockopt default
> t : 0
> timeout
> sockopt - -
> timeout
> sockopt - R
> timeout
> sockopt L -
> t : 3
> timeout
> sockopt L R
> s : 4
> t : 4
> timeout
> done.
>
^ permalink raw reply
* [PATCH] tcp: cookie transactions setsockopt memory leak
From: Dmitry Popov @ 2010-07-29 11:59 UTC (permalink / raw)
To: David S. Miller, Alexey Kuznetsov, James Morris, Patrick McHardy
Cc: Pekka Savola (ipv6), Hideaki YOSHIFUJI, Eric Dumazet,
Ilpo Järvinen, Andrew Morton, Steven J. Magnani, netdev,
linux-kernel, William Allen Simpson
From: Dmitry Popov <dp@highloadlab.com>
There is a bug in do_tcp_setsockopt(net/ipv4/tcp.c),
TCP_COOKIE_TRANSACTIONS case.
In some cases (when tp->cookie_values == NULL) new tcp_cookie_values
structure can be allocated (at cvp), but not bound to
tp->cookie_values. So a memory leak occurs.
Signed-off-by: Dmitry Popov <dp@highloadlab.com>
---
tp->cookie_values can be NULL if socket was initialized with
sysctl_tcp_cookie_size == 0 (tcp_v4_init_sock, net/ipv4/tcp_ipv4.c)
Buggy releases: 2.6.33+ (since commit e56fb50f2b7958b931c8a2fc0966061b3f3c8f3a)
net/ipv4/tcp.c | 7 +++++--
1 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/net/ipv4/tcp.c b/net/ipv4/tcp.c
index 83d0213..9c490a1 100644
--- a/net/ipv4/tcp.c
+++ b/net/ipv4/tcp.c
@@ -2179,6 +2179,8 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
GFP_KERNEL);
if (cvp == NULL)
return -ENOMEM;
+
+ kref_init(&cvp->kref);
}
lock_sock(sk);
tp->rx_opt.cookie_in_always =
@@ -2193,12 +2195,11 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
*/
kref_put(&tp->cookie_values->kref,
tcp_cookie_values_release);
- kref_init(&cvp->kref);
- tp->cookie_values = cvp;
} else {
cvp = tp->cookie_values;
}
}
+
if (cvp != NULL) {
cvp->cookie_desired = ctd.tcpct_cookie_desired;
@@ -2212,6 +2213,8 @@ static int do_tcp_setsockopt(struct sock *sk, int level,
cvp->s_data_desired = ctd.tcpct_s_data_desired;
cvp->s_data_constant = 0; /* false */
}
+
+ tp->cookie_values = cvp;
}
release_sock(sk);
return err;
^ permalink raw reply related
* [PATCH] vhost: locking/rcu cleanup
From: Michael S. Tsirkin @ 2010-07-29 12:23 UTC (permalink / raw)
To: Michael S. Tsirkin, David S. Miller, Sridhar Samudrala, Tejun Heo,
Jeff Dike
I saw WARN_ON(!list_empty(&dev->work_list)) trigger
so our custom flush is not as airtight as need be.
This patch switches to a simple atomic counter + srcu instead of
the custom locked queue + flush implementation.
This will slow down the setup ioctls, which should not matter -
it's slow path anyway. We use the expedited flush to at least
make sure it has a sane time bound.
Works fine for me. I got reports that with many guests,
work lock is highly contended, and this patch should in theory
fix this as well - but I haven't tested this yet.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
drivers/vhost/net.c | 55 +++++--------------
drivers/vhost/vhost.c | 140 ++++++++++++++++++++++---------------------------
drivers/vhost/vhost.h | 47 +++++++++-------
3 files changed, 103 insertions(+), 139 deletions(-)
diff --git a/drivers/vhost/net.c b/drivers/vhost/net.c
index f13e56b..ee69c51 100644
--- a/drivers/vhost/net.c
+++ b/drivers/vhost/net.c
@@ -111,8 +111,9 @@ static void tx_poll_start(struct vhost_net *net, struct socket *sock)
/* Expects to be always run from workqueue - which acts as
* read-size critical section for our kind of RCU. */
-static void handle_tx(struct vhost_net *net)
+static void handle_tx(struct vhost_dev *dev)
{
+ struct vhost_net *net = container_of(dev, struct vhost_net, dev);
struct vhost_virtqueue *vq = &net->dev.vqs[VHOST_NET_VQ_TX];
unsigned out, in, s;
int head;
@@ -127,7 +128,7 @@ static void handle_tx(struct vhost_net *net)
size_t len, total_len = 0;
int err, wmem;
size_t hdr_size;
- struct socket *sock = rcu_dereference(vq->private_data);
+ struct socket *sock = vhost_vq_data(vq, &net->dev);
if (!sock)
return;
@@ -305,7 +306,7 @@ static void handle_rx_big(struct vhost_net *net)
size_t len, total_len = 0;
int err;
size_t hdr_size;
- struct socket *sock = rcu_dereference(vq->private_data);
+ struct socket *sock = vhost_vq_data(vq, &net->dev);
if (!sock || skb_queue_empty(&sock->sk->sk_receive_queue))
return;
@@ -416,7 +417,7 @@ static void handle_rx_mergeable(struct vhost_net *net)
int err, headcount;
size_t vhost_hlen, sock_hlen;
size_t vhost_len, sock_len;
- struct socket *sock = rcu_dereference(vq->private_data);
+ struct socket *sock = vhost_vq_data(vq, &net->dev);
if (!sock || skb_queue_empty(&sock->sk->sk_receive_queue))
return;
@@ -500,46 +501,15 @@ static void handle_rx_mergeable(struct vhost_net *net)
unuse_mm(net->dev.mm);
}
-static void handle_rx(struct vhost_net *net)
+static void handle_rx(struct vhost_dev *dev)
{
+ struct vhost_net *net = container_of(dev, struct vhost_net, dev);
if (vhost_has_feature(&net->dev, VIRTIO_NET_F_MRG_RXBUF))
handle_rx_mergeable(net);
else
handle_rx_big(net);
}
-static void handle_tx_kick(struct vhost_work *work)
-{
- struct vhost_virtqueue *vq = container_of(work, struct vhost_virtqueue,
- poll.work);
- struct vhost_net *net = container_of(vq->dev, struct vhost_net, dev);
-
- handle_tx(net);
-}
-
-static void handle_rx_kick(struct vhost_work *work)
-{
- struct vhost_virtqueue *vq = container_of(work, struct vhost_virtqueue,
- poll.work);
- struct vhost_net *net = container_of(vq->dev, struct vhost_net, dev);
-
- handle_rx(net);
-}
-
-static void handle_tx_net(struct vhost_work *work)
-{
- struct vhost_net *net = container_of(work, struct vhost_net,
- poll[VHOST_NET_VQ_TX].work);
- handle_tx(net);
-}
-
-static void handle_rx_net(struct vhost_work *work)
-{
- struct vhost_net *net = container_of(work, struct vhost_net,
- poll[VHOST_NET_VQ_RX].work);
- handle_rx(net);
-}
-
static int vhost_net_open(struct inode *inode, struct file *f)
{
struct vhost_net *n = kmalloc(sizeof *n, GFP_KERNEL);
@@ -550,16 +520,18 @@ static int vhost_net_open(struct inode *inode, struct file *f)
return -ENOMEM;
dev = &n->dev;
- n->vqs[VHOST_NET_VQ_TX].handle_kick = handle_tx_kick;
- n->vqs[VHOST_NET_VQ_RX].handle_kick = handle_rx_kick;
+ vhost_work_set_fn(&n->vqs[VHOST_NET_VQ_TX].work, handle_tx);
+ vhost_work_set_fn(&n->vqs[VHOST_NET_VQ_RX].work, handle_rx);
r = vhost_dev_init(dev, n->vqs, VHOST_NET_VQ_MAX);
if (r < 0) {
kfree(n);
return r;
}
- vhost_poll_init(n->poll + VHOST_NET_VQ_TX, handle_tx_net, POLLOUT, dev);
- vhost_poll_init(n->poll + VHOST_NET_VQ_RX, handle_rx_net, POLLIN, dev);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_TX,
+ &n->vqs[VHOST_NET_VQ_TX].work, POLLOUT, dev);
+ vhost_poll_init(n->poll + VHOST_NET_VQ_RX,
+ &n->vqs[VHOST_NET_VQ_RX].work, POLLIN, dev);
n->tx_poll_state = VHOST_NET_POLL_DISABLED;
f->private_data = n;
@@ -640,6 +612,7 @@ static int vhost_net_release(struct inode *inode, struct file *f)
/* We do an extra flush before freeing memory,
* since jobs can re-queue themselves. */
vhost_net_flush(n);
+ vhost_dev_free(&n->dev);
kfree(n);
return 0;
}
diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
index e05557d..daa95c8 100644
--- a/drivers/vhost/vhost.c
+++ b/drivers/vhost/vhost.c
@@ -60,22 +60,27 @@ static int vhost_poll_wakeup(wait_queue_t *wait, unsigned mode, int sync,
return 0;
}
+/* Must be called for each vq before vhost_dev_init. */
+void vhost_work_set_fn(struct vhost_work *work, vhost_work_fn_t fn)
+{
+ work->fn = fn;
+}
+
+static void vhost_work_init(struct vhost_work *work)
+{
+ atomic_set(&work->queue_seq, 0);
+ work->done_seq = 0;
+}
+
/* Init poll structure */
-void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
+void vhost_poll_init(struct vhost_poll *poll, struct vhost_work *work,
unsigned long mask, struct vhost_dev *dev)
{
- struct vhost_work *work = &poll->work;
-
+ poll->work = work;
init_waitqueue_func_entry(&poll->wait, vhost_poll_wakeup);
init_poll_funcptr(&poll->table, vhost_poll_func);
poll->mask = mask;
poll->dev = dev;
-
- INIT_LIST_HEAD(&work->node);
- work->fn = fn;
- init_waitqueue_head(&work->done);
- work->flushing = 0;
- work->queue_seq = work->done_seq = 0;
}
/* Start polling a file. We add ourselves to file's wait queue. The caller must
@@ -99,40 +104,16 @@ void vhost_poll_stop(struct vhost_poll *poll)
* locks that are also used by the callback. */
void vhost_poll_flush(struct vhost_poll *poll)
{
- struct vhost_work *work = &poll->work;
- unsigned seq;
- int left;
- int flushing;
-
- spin_lock_irq(&poll->dev->work_lock);
- seq = work->queue_seq;
- work->flushing++;
- spin_unlock_irq(&poll->dev->work_lock);
- wait_event(work->done, ({
- spin_lock_irq(&poll->dev->work_lock);
- left = seq - work->done_seq <= 0;
- spin_unlock_irq(&poll->dev->work_lock);
- left;
- }));
- spin_lock_irq(&poll->dev->work_lock);
- flushing = --work->flushing;
- spin_unlock_irq(&poll->dev->work_lock);
- BUG_ON(flushing < 0);
+ synchronize_srcu_expedited(&poll->dev->worker_srcu);
}
void vhost_poll_queue(struct vhost_poll *poll)
{
struct vhost_dev *dev = poll->dev;
- struct vhost_work *work = &poll->work;
- unsigned long flags;
-
- spin_lock_irqsave(&dev->work_lock, flags);
- if (list_empty(&work->node)) {
- list_add_tail(&work->node, &dev->work_list);
- work->queue_seq++;
- wake_up_process(dev->worker);
- }
- spin_unlock_irqrestore(&dev->work_lock, flags);
+ struct vhost_work *work = poll->work;
+
+ atomic_inc(&work->queue_seq);
+ wake_up_process(dev->worker);
}
static void vhost_vq_reset(struct vhost_dev *dev,
@@ -164,41 +145,39 @@ static void vhost_vq_reset(struct vhost_dev *dev,
static int vhost_worker(void *data)
{
struct vhost_dev *dev = data;
- struct vhost_work *work = NULL;
- unsigned uninitialized_var(seq);
+ struct vhost_work *uninitialized_var(work);
+ unsigned n, i, vq = 0;
+ int seq;
- for (;;) {
- /* mb paired w/ kthread_stop */
- set_current_state(TASK_INTERRUPTIBLE);
+ n = dev->nvqs;
+repeat:
+ set_current_state(TASK_INTERRUPTIBLE); /* mb paired w/ kthread_stop */
- spin_lock_irq(&dev->work_lock);
- if (work) {
- work->done_seq = seq;
- if (work->flushing)
- wake_up_all(&work->done);
- }
+ if (kthread_should_stop()) {
+ __set_current_state(TASK_RUNNING);
+ return 0;
+ }
- if (kthread_should_stop()) {
- spin_unlock_irq(&dev->work_lock);
- __set_current_state(TASK_RUNNING);
- return 0;
+ for (i = 0; i < n; ++i) {
+ work = &dev->vqs[(vq + i) % n].work;
+ seq = atomic_read(&work->queue_seq);
+ if (seq != work->done_seq) {
+ work->done_seq = seq;
+ break;
}
- if (!list_empty(&dev->work_list)) {
- work = list_first_entry(&dev->work_list,
- struct vhost_work, node);
- list_del_init(&work->node);
- seq = work->queue_seq;
- } else
- work = NULL;
- spin_unlock_irq(&dev->work_lock);
+ work = NULL;
+ }
- if (work) {
- __set_current_state(TASK_RUNNING);
- work->fn(work);
- } else
- schedule();
+ if (work) {
+ int idx;
+ __set_current_state(TASK_RUNNING);
+ idx = srcu_read_lock(&dev->worker_srcu);
+ work->fn(dev);
+ srcu_read_unlock(&dev->worker_srcu, idx);
+ } else
+ schedule();
- }
+ goto repeat;
}
long vhost_dev_init(struct vhost_dev *dev,
@@ -213,20 +192,22 @@ long vhost_dev_init(struct vhost_dev *dev,
dev->log_file = NULL;
dev->memory = NULL;
dev->mm = NULL;
- spin_lock_init(&dev->work_lock);
- INIT_LIST_HEAD(&dev->work_list);
dev->worker = NULL;
for (i = 0; i < dev->nvqs; ++i) {
dev->vqs[i].dev = dev;
mutex_init(&dev->vqs[i].mutex);
vhost_vq_reset(dev, dev->vqs + i);
- if (dev->vqs[i].handle_kick)
+ if (dev->vqs[i].work.fn)
vhost_poll_init(&dev->vqs[i].poll,
- dev->vqs[i].handle_kick, POLLIN, dev);
+ &dev->vqs[i].work, POLLIN, dev);
}
+ return init_srcu_struct(&dev->worker_srcu);
+}
- return 0;
+void vhost_dev_free(struct vhost_dev *dev)
+{
+ cleanup_srcu_struct(&dev->worker_srcu);
}
/* Caller should have device mutex */
@@ -240,7 +221,7 @@ long vhost_dev_check_owner(struct vhost_dev *dev)
static long vhost_dev_set_owner(struct vhost_dev *dev)
{
struct task_struct *worker;
- int err;
+ int i, err;
/* Is there an owner already? */
if (dev->mm) {
err = -EBUSY;
@@ -258,6 +239,10 @@ static long vhost_dev_set_owner(struct vhost_dev *dev)
err = cgroup_attach_task_current_cg(worker);
if (err)
goto err_cgroup;
+
+ for (i = 0; i < dev->nvqs; ++i) {
+ vhost_work_init(&dev->vqs[i].work);
+ }
wake_up_process(worker); /* avoid contributing to loadavg */
return 0;
@@ -293,7 +278,7 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
{
int i;
for (i = 0; i < dev->nvqs; ++i) {
- if (dev->vqs[i].kick && dev->vqs[i].handle_kick) {
+ if (dev->vqs[i].kick && dev->vqs[i].work.fn) {
vhost_poll_stop(&dev->vqs[i].poll);
vhost_poll_flush(&dev->vqs[i].poll);
}
@@ -322,7 +307,6 @@ void vhost_dev_cleanup(struct vhost_dev *dev)
mmput(dev->mm);
dev->mm = NULL;
- WARN_ON(!list_empty(&dev->work_list));
kthread_stop(dev->worker);
}
@@ -644,7 +628,7 @@ static long vhost_set_vring(struct vhost_dev *d, int ioctl, void __user *argp)
r = -ENOIOCTLCMD;
}
- if (pollstop && vq->handle_kick)
+ if (pollstop && vq->work.fn)
vhost_poll_stop(&vq->poll);
if (ctx)
@@ -652,12 +636,12 @@ static long vhost_set_vring(struct vhost_dev *d, int ioctl, void __user *argp)
if (filep)
fput(filep);
- if (pollstart && vq->handle_kick)
+ if (pollstart && vq->work.fn)
vhost_poll_start(&vq->poll, vq->kick);
mutex_unlock(&vq->mutex);
- if (pollstop && vq->handle_kick)
+ if (pollstop && vq->work.fn)
vhost_poll_flush(&vq->poll);
return r;
}
diff --git a/drivers/vhost/vhost.h b/drivers/vhost/vhost.h
index afd7729..9c990ea 100644
--- a/drivers/vhost/vhost.h
+++ b/drivers/vhost/vhost.h
@@ -11,9 +11,10 @@
#include <linux/uio.h>
#include <linux/virtio_config.h>
#include <linux/virtio_ring.h>
+#include <linux/srcu.h>
#include <asm/atomic.h>
-struct vhost_device;
+struct vhost_dev;
enum {
/* Enough place for all fragments, head, and virtio net header. */
@@ -21,29 +22,33 @@ enum {
};
struct vhost_work;
-typedef void (*vhost_work_fn_t)(struct vhost_work *work);
+typedef void (*vhost_work_fn_t)(struct vhost_dev *dev);
struct vhost_work {
- struct list_head node;
+ /* Callback function to execute. */
vhost_work_fn_t fn;
- wait_queue_head_t done;
- int flushing;
- unsigned queue_seq;
- unsigned done_seq;
+ /* Incremented to request callback execution.
+ * Atomic to allow multiple writers. */
+ atomic_t queue_seq;
+ /* Used by worker to track execution requests.
+ * Used from a single thread so no locking. */
+ int done_seq;
};
+void vhost_work_set_fn(struct vhost_work *work, vhost_work_fn_t fn);
+
/* Poll a file (eventfd or socket) */
/* Note: there's nothing vhost specific about this structure. */
struct vhost_poll {
poll_table table;
wait_queue_head_t *wqh;
wait_queue_t wait;
- struct vhost_work work;
+ struct vhost_work *work;
unsigned long mask;
struct vhost_dev *dev;
};
-void vhost_poll_init(struct vhost_poll *poll, vhost_work_fn_t fn,
+void vhost_poll_init(struct vhost_poll *poll, struct vhost_work* work,
unsigned long mask, struct vhost_dev *dev);
void vhost_poll_start(struct vhost_poll *poll, struct file *file);
void vhost_poll_stop(struct vhost_poll *poll);
@@ -72,11 +77,12 @@ struct vhost_virtqueue {
struct eventfd_ctx *error_ctx;
struct eventfd_ctx *log_ctx;
+ /* The work to execute when the Guest kicks us,
+ * on Host activity, or timeout. */
+ struct vhost_work work;
+ /* Poll Guest for kicks */
struct vhost_poll poll;
- /* The routine to call when the Guest pings us, or timeout. */
- vhost_work_fn_t handle_kick;
-
/* Last available index we saw. */
u16 last_avail_idx;
@@ -99,12 +105,7 @@ struct vhost_virtqueue {
size_t vhost_hlen;
size_t sock_hlen;
struct vring_used_elem heads[VHOST_NET_MAX_SG];
- /* We use a kind of RCU to access private pointer.
- * All readers access it from worker, which makes it possible to
- * flush the vhost_work instead of synchronize_rcu. Therefore readers do
- * not need to call rcu_read_lock/rcu_read_unlock: the beginning of
- * vhost_work execution acts instead of rcu_read_lock() and the end of
- * vhost_work execution acts instead of rcu_read_lock().
+ /* Readers use worker_srcu in device to access private pointer.
* Writers use virtqueue mutex. */
void *private_data;
/* Log write descriptors */
@@ -112,6 +113,12 @@ struct vhost_virtqueue {
struct vhost_log log[VHOST_NET_MAX_SG];
};
+static inline void *vhost_vq_data(struct vhost_virtqueue *vq,
+ struct vhost_dev *dev)
+{
+ return srcu_dereference(vq->private_data, &dev->worker_srcu);
+}
+
struct vhost_dev {
/* Readers use RCU to access memory table pointer
* log base pointer and features.
@@ -124,12 +131,12 @@ struct vhost_dev {
int nvqs;
struct file *log_file;
struct eventfd_ctx *log_ctx;
- spinlock_t work_lock;
- struct list_head work_list;
struct task_struct *worker;
+ struct srcu_struct worker_srcu;
};
long vhost_dev_init(struct vhost_dev *, struct vhost_virtqueue *vqs, int nvqs);
+void vhost_dev_free(struct vhost_dev *);
long vhost_dev_check_owner(struct vhost_dev *);
long vhost_dev_reset_owner(struct vhost_dev *);
void vhost_dev_cleanup(struct vhost_dev *);
--
1.7.2.rc0.14.g41c1c
^ permalink raw reply related
* [PATCH 3/3] TI DaVinci EMAC: Fix incorrect reference to EMAC_CTRL registers.
From: Sriramakrishnan @ 2010-07-29 12:34 UTC (permalink / raw)
To: netdev, davinci-linux-open-source; +Cc: nsekhar, anantgole, Sriramakrishnan
In-Reply-To: <1280406840-27393-2-git-send-email-srk@ti.com>
The EMAC modules control registers vary as per the version of the
EMAC module. EMAC_CTRL_EWCTL,EMAC_CTRL_EWINTTCNT are available
only on EMAC_VERSION_1. The emac_dump_regs() function accesses
these indiscriminately. This patch fixes the issue.
Signed-off-by: Sriramakrishnan <srk@ti.com>
---
drivers/net/davinci_emac.c | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
index 4ee9af6..12196ad 100644
--- a/drivers/net/davinci_emac.c
+++ b/drivers/net/davinci_emac.c
@@ -565,9 +565,11 @@ static void emac_dump_regs(struct emac_priv *priv)
/* Print important registers in EMAC */
dev_info(emac_dev, "EMAC Basic registers\n");
- dev_info(emac_dev, "EMAC: EWCTL: %08X, EWINTTCNT: %08X\n",
- emac_ctrl_read(EMAC_CTRL_EWCTL),
- emac_ctrl_read(EMAC_CTRL_EWINTTCNT));
+ if (priv->version == EMAC_VERSION_1) {
+ dev_info(emac_dev, "EMAC: EWCTL: %08X, EWINTTCNT: %08X\n",
+ emac_ctrl_read(EMAC_CTRL_EWCTL),
+ emac_ctrl_read(EMAC_CTRL_EWINTTCNT));
+ }
dev_info(emac_dev, "EMAC: TXID: %08X %s, RXID: %08X %s\n",
emac_read(EMAC_TXIDVER),
((emac_read(EMAC_TXCONTROL)) ? "enabled" : "disabled"),
--
1.6.2.4
^ permalink raw reply related
* [PATCH 2/3] TI DaVinci EMAC: Fix asymmetric handling of packets in NAPI Poll function.
From: Sriramakrishnan @ 2010-07-29 12:33 UTC (permalink / raw)
To: netdev, davinci-linux-open-source; +Cc: nsekhar, anantgole, Sriramakrishnan
In-Reply-To: <1280406840-27393-1-git-send-email-srk@ti.com>
The current implementation of NAPI poll function in the driver does not service
Rx packets, error condition even if a single Tx packet gets serviced in
the napi poll call. This behavior severely affects performance for specific use
cases. This patch modifies the poll function implementation to service tx/rx
packets in an identical manner.
Signed-off-by: Sriramakrishnan <srk@ti.com>
---
drivers/net/davinci_emac.c | 21 ++++++++-------------
1 files changed, 8 insertions(+), 13 deletions(-)
diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
index bc1b270..4ee9af6 100644
--- a/drivers/net/davinci_emac.c
+++ b/drivers/net/davinci_emac.c
@@ -2266,7 +2266,7 @@ static int emac_poll(struct napi_struct *napi, int budget)
struct net_device *ndev = priv->ndev;
struct device *emac_dev = &ndev->dev;
u32 status = 0;
- u32 num_pkts = 0;
+ u32 num_tx_pkts = 0, num_rx_pkts = 0;
/* Check interrupt vectors and call packet processing */
status = emac_read(EMAC_MACINVECTOR);
@@ -2277,27 +2277,19 @@ static int emac_poll(struct napi_struct *napi, int budget)
mask = EMAC_DM646X_MAC_IN_VECTOR_TX_INT_VEC;
if (status & mask) {
- num_pkts = emac_tx_bdproc(priv, EMAC_DEF_TX_CH,
+ num_tx_pkts = emac_tx_bdproc(priv, EMAC_DEF_TX_CH,
EMAC_DEF_TX_MAX_SERVICE);
} /* TX processing */
- if (num_pkts)
- return budget;
-
mask = EMAC_DM644X_MAC_IN_VECTOR_RX_INT_VEC;
if (priv->version == EMAC_VERSION_2)
mask = EMAC_DM646X_MAC_IN_VECTOR_RX_INT_VEC;
if (status & mask) {
- num_pkts = emac_rx_bdproc(priv, EMAC_DEF_RX_CH, budget);
+ num_rx_pkts = emac_rx_bdproc(priv, EMAC_DEF_RX_CH, budget);
} /* RX processing */
- if (num_pkts < budget) {
- napi_complete(napi);
- emac_int_enable(priv);
- }
-
mask = EMAC_DM644X_MAC_IN_VECTOR_HOST_INT;
if (priv->version == EMAC_VERSION_2)
mask = EMAC_DM646X_MAC_IN_VECTOR_HOST_INT;
@@ -2328,9 +2320,12 @@ static int emac_poll(struct napi_struct *napi, int budget)
dev_err(emac_dev, "RX Host error %s on ch=%d\n",
&emac_rxhost_errcodes[cause][0], ch);
}
- } /* Host error processing */
+ } else if (num_rx_pkts < budget) {
+ napi_complete(napi);
+ emac_int_enable(priv);
+ }
- return num_pkts;
+ return num_rx_pkts;
}
#ifdef CONFIG_NET_POLL_CONTROLLER
--
1.6.2.4
^ permalink raw reply related
* [PATCH 1/3] TI DaVinci EMAC : Implement interrupt pacing functionality.
From: Sriramakrishnan @ 2010-07-29 12:33 UTC (permalink / raw)
To: netdev, davinci-linux-open-source; +Cc: nsekhar, anantgole, Sriramakrishnan
DaVinci EMAC module includes an interrupt pacing block that can
be programmed to throttle the rate at which interrupts are
generated. This patch implements interrupt pacing logic that can
be controlled through the ethtool interface(only rx_coalesce_usecs
param is honored)
Signed-off-by: Sriramakrishnan <srk@ti.com>
---
drivers/net/davinci_emac.c | 133 +++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 131 insertions(+), 2 deletions(-)
diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
index 08e82b1..bc1b270 100644
--- a/drivers/net/davinci_emac.c
+++ b/drivers/net/davinci_emac.c
@@ -298,6 +298,11 @@ static const char emac_version_string[] = "TI DaVinci EMAC Linux v6.1";
#define EMAC_CTRL_EWCTL (0x4)
#define EMAC_CTRL_EWINTTCNT (0x8)
+/* EMAC DM644x control module masks */
+#define EMAC_DM644X_EWINTCNT_MASK 0x1FFFF
+#define EMAC_DM644X_INTMIN_INTVL 0x1
+#define EMAC_DM644X_INTMAX_INTVL (EMAC_DM644X_EWINTCNT_MASK)
+
/* EMAC MDIO related */
/* Mask & Control defines */
#define MDIO_CONTROL_CLKDIV (0xFF)
@@ -318,8 +323,20 @@ static const char emac_version_string[] = "TI DaVinci EMAC Linux v6.1";
#define MDIO_CONTROL (0x04)
/* EMAC DM646X control module registers */
-#define EMAC_DM646X_CMRXINTEN (0x14)
-#define EMAC_DM646X_CMTXINTEN (0x18)
+#define EMAC_DM646X_CMINTCTRL 0x0C
+#define EMAC_DM646X_CMRXINTEN 0x14
+#define EMAC_DM646X_CMTXINTEN 0x18
+#define EMAC_DM646X_CMRXINTMAX 0x70
+#define EMAC_DM646X_CMTXINTMAX 0x74
+
+/* EMAC DM646X control module masks */
+#define EMAC_DM646X_INTPACEEN (0x3 << 16)
+#define EMAC_DM646X_INTPRESCALE_MASK (0x7FF << 0)
+#define EMAC_DM646X_CMINTMAX_CNT 63
+#define EMAC_DM646X_CMINTMIN_CNT 2
+#define EMAC_DM646X_CMINTMAX_INTVL (1000 / EMAC_DM646X_CMINTMIN_CNT)
+#define EMAC_DM646X_CMINTMIN_INTVL ((1000 / EMAC_DM646X_CMINTMAX_CNT) + 1)
+
/* EMAC EOI codes for C0 */
#define EMAC_DM646X_MAC_EOI_C0_RXEN (0x01)
@@ -468,6 +485,8 @@ struct emac_priv {
u32 duplex; /* Link duplex: 0=Half, 1=Full */
u32 rx_buf_size;
u32 isr_count;
+ u32 coal_intvl;
+ u32 bus_freq_mhz;
u8 rmii_en;
u8 version;
struct net_device_stats net_dev_stats;
@@ -692,6 +711,103 @@ static int emac_set_settings(struct net_device *ndev, struct ethtool_cmd *ecmd)
}
/**
+ * emac_get_coalesce : Get interrupt coalesce settings for this device
+ * @ndev : The DaVinci EMAC network adapter
+ * @coal : ethtool coalesce settings structure
+ *
+ * Fetch the current interrupt coalesce settings
+ *
+ */
+static int emac_get_coalesce(struct net_device *ndev,
+ struct ethtool_coalesce *coal)
+{
+ struct emac_priv *priv = netdev_priv(ndev);
+
+ coal->rx_coalesce_usecs = priv->coal_intvl;
+ return 0;
+
+}
+
+/**
+ * emac_set_coalesce : Set interrupt coalesce settings for this device
+ * @ndev : The DaVinci EMAC network adapter
+ * @coal : ethtool coalesce settings structure
+ *
+ * Set interrupt coalesce parameters
+ *
+ */
+static int emac_set_coalesce(struct net_device *ndev,
+ struct ethtool_coalesce *coal)
+{
+ struct emac_priv *priv = netdev_priv(ndev);
+ u32 int_ctrl, num_interrupts = 0;
+ u32 prescale = 0, addnl_dvdr = 1, coal_intvl = 0;
+
+ if (!coal->rx_coalesce_usecs)
+ return -EINVAL;
+
+ coal_intvl = coal->rx_coalesce_usecs;
+
+ switch (priv->version) {
+ case EMAC_VERSION_2:
+ int_ctrl = emac_ctrl_read(EMAC_DM646X_CMINTCTRL);
+ prescale = priv->bus_freq_mhz * 4;
+
+ if (coal_intvl < EMAC_DM646X_CMINTMIN_INTVL)
+ coal_intvl = EMAC_DM646X_CMINTMIN_INTVL;
+
+ if (coal_intvl > EMAC_DM646X_CMINTMAX_INTVL) {
+ /*
+ * Interrupt pacer works with 4us Pulse, we can
+ * throttle further by dilating the 4us pulse.
+ */
+ addnl_dvdr = EMAC_DM646X_INTPRESCALE_MASK / prescale;
+
+ if (addnl_dvdr > 1) {
+ prescale *= addnl_dvdr;
+ if (coal_intvl > (EMAC_DM646X_CMINTMAX_INTVL
+ * addnl_dvdr))
+ coal_intvl = (EMAC_DM646X_CMINTMAX_INTVL
+ * addnl_dvdr);
+ } else {
+ addnl_dvdr = 1;
+ coal_intvl = EMAC_DM646X_CMINTMAX_INTVL;
+ }
+ }
+
+ num_interrupts = (1000 * addnl_dvdr) / coal_intvl;
+
+ int_ctrl |= EMAC_DM646X_INTPACEEN;
+ int_ctrl &= (~EMAC_DM646X_INTPRESCALE_MASK);
+ int_ctrl |= (prescale & EMAC_DM646X_INTPRESCALE_MASK);
+ emac_ctrl_write(EMAC_DM646X_CMINTCTRL, int_ctrl);
+
+ emac_ctrl_write(EMAC_DM646X_CMRXINTMAX, num_interrupts);
+ emac_ctrl_write(EMAC_DM646X_CMTXINTMAX, num_interrupts);
+
+ break;
+ default:
+ int_ctrl = emac_ctrl_read(EMAC_CTRL_EWINTTCNT);
+ int_ctrl &= (~EMAC_DM644X_EWINTCNT_MASK);
+ prescale = coal_intvl * priv->bus_freq_mhz;
+ if (prescale > EMAC_DM644X_EWINTCNT_MASK) {
+ prescale = EMAC_DM644X_EWINTCNT_MASK;
+ coal_intvl = prescale / priv->bus_freq_mhz;
+ }
+ emac_ctrl_write(EMAC_CTRL_EWINTTCNT, (int_ctrl | prescale));
+
+ break;
+ }
+
+ printk(KERN_INFO"Set coalesce to %d usecs.\n", coal_intvl);
+ priv->coal_intvl = coal_intvl;
+
+ return 0;
+
+}
+
+
+/**
* ethtool_ops: DaVinci EMAC Ethtool structure
*
* Ethtool support for EMAC adapter
@@ -702,6 +818,8 @@ static const struct ethtool_ops ethtool_ops = {
.get_settings = emac_get_settings,
.set_settings = emac_set_settings,
.get_link = ethtool_op_get_link,
+ .get_coalesce = emac_get_coalesce,
+ .set_coalesce = emac_set_coalesce,
};
/**
@@ -2437,6 +2555,14 @@ static int emac_dev_open(struct net_device *ndev)
/* Start/Enable EMAC hardware */
emac_hw_enable(priv);
+ /* Enable Interrupt pacing if configured */
+ if (priv->coal_intvl != 0) {
+ struct ethtool_coalesce coal;
+
+ coal.rx_coalesce_usecs = (priv->coal_intvl << 4);
+ emac_set_coalesce(ndev, &coal);
+ }
+
/* find the first phy */
priv->phydev = NULL;
if (priv->phy_mask) {
@@ -2677,6 +2803,9 @@ static int __devinit davinci_emac_probe(struct platform_device *pdev)
priv->int_enable = pdata->interrupt_enable;
priv->int_disable = pdata->interrupt_disable;
+ priv->coal_intvl = 0;
+ priv->bus_freq_mhz = (u32)(emac_bus_frequency / 1000000);
+
emac_dev = &ndev->dev;
/* Get EMAC platform data */
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
--
1.6.2.4
^ permalink raw reply related
* [PATCH] sky2: Code style fixes
From: Mike McCormack @ 2010-07-29 13:34 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev
Fix selected style problems reported by checkpatch.
Signed-off-by: Mike McCormack <mikem@ring3k.org>
---
drivers/net/sky2.c | 24 ++++++++++++------------
1 files changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/net/sky2.c b/drivers/net/sky2.c
index 8b85c02..8723e3d 100644
--- a/drivers/net/sky2.c
+++ b/drivers/net/sky2.c
@@ -79,7 +79,7 @@
#define SKY2_EEPROM_MAGIC 0x9955aabb
-#define RING_NEXT(x,s) (((x)+1) & ((s)-1))
+#define RING_NEXT(x, s) (((x)+1) & ((s)-1))
static const u32 default_msg =
NETIF_MSG_DRV | NETIF_MSG_PROBE | NETIF_MSG_LINK
@@ -172,7 +172,7 @@ static int gm_phy_write(struct sky2_hw *hw, unsigned port, u16 reg, u16 val)
udelay(10);
}
- dev_warn(&hw->pdev->dev,"%s: phy write timeout\n", hw->dev[port]->name);
+ dev_warn(&hw->pdev->dev, "%s: phy write timeout\n", hw->dev[port]->name);
return -ETIMEDOUT;
io_error:
@@ -1078,7 +1078,7 @@ static inline struct sky2_rx_le *sky2_next_rx(struct sky2_port *sky2)
return le;
}
-static unsigned sky2_get_rx_threshold(struct sky2_port* sky2)
+static unsigned sky2_get_rx_threshold(struct sky2_port *sky2)
{
unsigned size;
@@ -1089,7 +1089,7 @@ static unsigned sky2_get_rx_threshold(struct sky2_port* sky2)
return (size - 8) / sizeof(u32);
}
-static unsigned sky2_get_rx_data_size(struct sky2_port* sky2)
+static unsigned sky2_get_rx_data_size(struct sky2_port *sky2)
{
struct rx_ring_info *re;
unsigned size;
@@ -1113,7 +1113,7 @@ static unsigned sky2_get_rx_data_size(struct sky2_port* sky2)
}
/* Build description to hardware for one receive segment */
-static void sky2_rx_add(struct sky2_port *sky2, u8 op,
+static void sky2_rx_add(struct sky2_port *sky2, u8 op,
dma_addr_t map, unsigned len)
{
struct sky2_rx_le *le;
@@ -3019,7 +3019,7 @@ static int __devinit sky2_init(struct sky2_hw *hw)
hw->chip_id = sky2_read8(hw, B2_CHIP_ID);
hw->chip_rev = (sky2_read8(hw, B2_MAC_CFG) & CFG_CHIP_R_MSK) >> 4;
- switch(hw->chip_id) {
+ switch (hw->chip_id) {
case CHIP_ID_YUKON_XL:
hw->flags = SKY2_HW_GIGABIT | SKY2_HW_NEWER_PHY;
if (hw->chip_rev < CHIP_REV_YU_XL_A2)
@@ -3690,7 +3690,7 @@ static int sky2_set_mac_address(struct net_device *dev, void *p)
return 0;
}
-static void inline sky2_add_filter(u8 filter[8], const u8 *addr)
+static inline void sky2_add_filter(u8 filter[8], const u8 *addr)
{
u32 bit;
@@ -3916,7 +3916,7 @@ static int sky2_set_coalesce(struct net_device *dev,
return -EINVAL;
if (ecmd->rx_max_coalesced_frames > RX_MAX_PENDING)
return -EINVAL;
- if (ecmd->rx_max_coalesced_frames_irq >RX_MAX_PENDING)
+ if (ecmd->rx_max_coalesced_frames_irq > RX_MAX_PENDING)
return -EINVAL;
if (ecmd->tx_coalesce_usecs == 0)
@@ -4381,7 +4381,7 @@ static int sky2_debug_show(struct seq_file *seq, void *v)
seq_printf(seq, "%u:", idx);
sop = 0;
- switch(le->opcode & ~HW_OWNER) {
+ switch (le->opcode & ~HW_OWNER) {
case OP_ADDR64:
seq_printf(seq, " %#x:", a);
break;
@@ -4450,7 +4450,7 @@ static int sky2_device_event(struct notifier_block *unused,
if (dev->netdev_ops->ndo_open != sky2_up || !sky2_debug)
return NOTIFY_DONE;
- switch(event) {
+ switch (event) {
case NETDEV_CHANGENAME:
if (sky2->debugfs) {
sky2->debugfs = debugfs_rename(sky2_debug, sky2->debugfs,
@@ -4645,7 +4645,7 @@ static int __devinit sky2_test_msi(struct sky2_hw *hw)
struct pci_dev *pdev = hw->pdev;
int err;
- init_waitqueue_head (&hw->msi_wait);
+ init_waitqueue_head(&hw->msi_wait);
sky2_write32(hw, B0_IMSK, Y2_IS_IRQ_SW);
@@ -4762,7 +4762,7 @@ static int __devinit sky2_probe(struct pci_dev *pdev,
* this driver uses software swapping.
*/
reg &= ~PCI_REV_DESC;
- err = pci_write_config_dword(pdev,PCI_DEV_REG2, reg);
+ err = pci_write_config_dword(pdev, PCI_DEV_REG2, reg);
if (err) {
dev_err(&pdev->dev, "PCI write config failed\n");
goto err_out_free_regions;
--
1.5.6.5
^ permalink raw reply related
* [PATCH] drivers: Fixes a typo from "dev" to "ndev" that caused compilation erros.
From: Henrique Camargo @ 2010-07-29 15:51 UTC (permalink / raw)
To: David S. Miller, Chaithrika U S, Sriramakrishnan, Kevin Hilman,
Jiri Pirko
Cc: segooon
From: Henrique Camargo
Fixes a typo from "dev" to "ndev" that caused compilation errors.
Signed-off-by: Henrique Camargo <henrique.camargo@ensitec.com.br>
---
drivers/net/davinci_emac.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
index 25e14d2..b89b7bf 100644
--- a/drivers/net/davinci_emac.c
+++ b/drivers/net/davinci_emac.c
@@ -1182,8 +1182,8 @@ static int emac_net_tx_complete(struct emac_priv *priv,
struct net_device *ndev = priv->ndev;
u32 cnt;
- if (unlikely(num_tokens && netif_queue_stopped(dev)))
- netif_start_queue(dev);
+ if (unlikely(num_tokens && netif_queue_stopped(ndev)))
+ netif_start_queue(ndev);
for (cnt = 0; cnt < num_tokens; cnt++) {
struct sk_buff *skb = (struct sk_buff *)net_data_tokens[cnt];
if (skb == NULL)
--
1.7.0.4
^ permalink raw reply related
* Re: [PATCH] drivers: Fixes a typo from "dev" to "ndev" that caused compilation erros.
From: Vasiliy Kulikov @ 2010-07-29 16:03 UTC (permalink / raw)
To: Henrique Camargo
Cc: David S. Miller, Chaithrika U S, Sriramakrishnan, Kevin Hilman,
Jiri Pirko, netdev, linux-kernel
In-Reply-To: <AANLkTiko540hMAUhaJr5mNKzFK2-vBOTZmEOxZWZv7ci@mail.gmail.com>
On Thu, Jul 29, 2010 at 12:51 -0300, Henrique Camargo wrote:
> From: Henrique Camargo
>
> Fixes a typo from "dev" to "ndev" that caused compilation errors.
>
> Signed-off-by: Henrique Camargo <henrique.camargo@ensitec.com.br>
Thanks,
Reviewed-by: Kulikov Vasiliy <segooon@gmail.com>
Also I've checked my other patches of removing private stats, the rest
are ok.
> ---
> drivers/net/davinci_emac.c | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/net/davinci_emac.c b/drivers/net/davinci_emac.c
> index 25e14d2..b89b7bf 100644
> --- a/drivers/net/davinci_emac.c
> +++ b/drivers/net/davinci_emac.c
> @@ -1182,8 +1182,8 @@ static int emac_net_tx_complete(struct emac_priv *priv,
> struct net_device *ndev = priv->ndev;
> u32 cnt;
>
> - if (unlikely(num_tokens && netif_queue_stopped(dev)))
> - netif_start_queue(dev);
> + if (unlikely(num_tokens && netif_queue_stopped(ndev)))
> + netif_start_queue(ndev);
> for (cnt = 0; cnt < num_tokens; cnt++) {
> struct sk_buff *skb = (struct sk_buff *)net_data_tokens[cnt];
> if (skb == NULL)
> --
> 1.7.0.4
^ permalink raw reply
* [PATCH net-next 1/4] be2net: fix to correctly know if driver needs to run for a VF or a PF
From: Ajit Khaparde @ 2010-07-29 16:15 UTC (permalink / raw)
To: David Miller; +Cc: netdev
Move be_check_sriov_fn_type to appropriate place to correctly determine
if the be2net driver needs to work as a VF driver or a PF driver.
Signed-off-by: Ajit Khaparde <ajitk@serverengines.com>
---
drivers/net/benet/be_main.c | 5 +++--
1 files changed, 3 insertions(+), 2 deletions(-)
diff --git a/drivers/net/benet/be_main.c b/drivers/net/benet/be_main.c
index d5b097d..e72b482 100644
--- a/drivers/net/benet/be_main.c
+++ b/drivers/net/benet/be_main.c
@@ -1717,10 +1717,11 @@ static void be_msix_enable(struct be_adapter *adapter)
static void be_sriov_enable(struct be_adapter *adapter)
{
-#ifdef CONFIG_PCI_IOV
- int status;
be_check_sriov_fn_type(adapter);
+#ifdef CONFIG_PCI_IOV
if (be_physfn(adapter) && num_vfs) {
+ int status;
+
status = pci_enable_sriov(adapter->pdev, num_vfs);
adapter->sriov_enabled = status ? false : true;
}
--
1.7.0.4
^ permalink raw reply related
* [PATCH net-next 2/4] be2net: add code to dump registers for debug
From: Ajit Khaparde @ 2010-07-29 16:16 UTC (permalink / raw)
To: David Miller; +Cc: netdev
when the BE device becomes unresponsive, dump the registers to help debugging
Signed-off-by: Somnath K <somnathk@serverengines.com>
Signed-off-by: Ajit Khaparde <ajitk@serverengines.com>
---
drivers/net/benet/be.h | 1 +
drivers/net/benet/be_cmds.c | 1 +
drivers/net/benet/be_cmds.h | 1 +
drivers/net/benet/be_hw.h | 10 ++++
drivers/net/benet/be_main.c | 127 +++++++++++++++++++++++++++++++++++++++++++
5 files changed, 140 insertions(+), 0 deletions(-)
diff --git a/drivers/net/benet/be.h b/drivers/net/benet/be.h
index e06369c..5e6f581 100644
--- a/drivers/net/benet/be.h
+++ b/drivers/net/benet/be.h
@@ -288,6 +288,7 @@ struct be_adapter {
u32 function_mode;
u32 rx_fc; /* Rx flow control */
u32 tx_fc; /* Tx flow control */
+ bool ue_detected;
int link_speed;
u8 port_type;
u8 transceiver;
diff --git a/drivers/net/benet/be_cmds.c b/drivers/net/benet/be_cmds.c
index 6eaf8a3..7fd860d 100644
--- a/drivers/net/benet/be_cmds.c
+++ b/drivers/net/benet/be_cmds.c
@@ -206,6 +206,7 @@ static int be_mbox_db_ready_wait(struct be_adapter *adapter, void __iomem *db)
if (msecs > 4000) {
dev_err(&adapter->pdev->dev, "mbox poll timed out\n");
+ be_dump_ue(adapter);
return -1;
}
diff --git a/drivers/net/benet/be_cmds.h b/drivers/net/benet/be_cmds.h
index 036531c..bdc10a2 100644
--- a/drivers/net/benet/be_cmds.h
+++ b/drivers/net/benet/be_cmds.h
@@ -992,4 +992,5 @@ extern int be_cmd_set_loopback(struct be_adapter *adapter, u8 port_num,
extern int be_cmd_get_phy_info(struct be_adapter *adapter,
struct be_dma_mem *cmd);
extern int be_cmd_set_qos(struct be_adapter *adapter, u32 bps, u32 domain);
+extern void be_dump_ue(struct be_adapter *adapter);
diff --git a/drivers/net/benet/be_hw.h b/drivers/net/benet/be_hw.h
index 0683967..6c8f9bb 100644
--- a/drivers/net/benet/be_hw.h
+++ b/drivers/net/benet/be_hw.h
@@ -56,6 +56,16 @@
#define PCICFG_PM_CONTROL_OFFSET 0x44
#define PCICFG_PM_CONTROL_MASK 0x108 /* bits 3 & 8 */
+/********* Online Control Registers *******/
+#define PCICFG_ONLINE0 0xB0
+#define PCICFG_ONLINE1 0xB4
+
+/********* UE Status and Mask Registers ***/
+#define PCICFG_UE_STATUS_LOW 0xA0
+#define PCICFG_UE_STATUS_HIGH 0xA4
+#define PCICFG_UE_STATUS_LOW_MASK 0xA8
+#define PCICFG_UE_STATUS_HI_MASK 0xAC
+
/********* ISR0 Register offset **********/
#define CEV_ISR0_OFFSET 0xC18
#define CEV_ISR_SIZE 4
diff --git a/drivers/net/benet/be_main.c b/drivers/net/benet/be_main.c
index e72b482..e4a8ae3 100644
--- a/drivers/net/benet/be_main.c
+++ b/drivers/net/benet/be_main.c
@@ -40,6 +40,76 @@ static DEFINE_PCI_DEVICE_TABLE(be_dev_ids) = {
{ 0 }
};
MODULE_DEVICE_TABLE(pci, be_dev_ids);
+/* UE Status Low CSR */
+static char *ue_status_low_desc[] = {
+ "CEV",
+ "CTX",
+ "DBUF",
+ "ERX",
+ "Host",
+ "MPU",
+ "NDMA",
+ "PTC ",
+ "RDMA ",
+ "RXF ",
+ "RXIPS ",
+ "RXULP0 ",
+ "RXULP1 ",
+ "RXULP2 ",
+ "TIM ",
+ "TPOST ",
+ "TPRE ",
+ "TXIPS ",
+ "TXULP0 ",
+ "TXULP1 ",
+ "UC ",
+ "WDMA ",
+ "TXULP2 ",
+ "HOST1 ",
+ "P0_OB_LINK ",
+ "P1_OB_LINK ",
+ "HOST_GPIO ",
+ "MBOX ",
+ "AXGMAC0",
+ "AXGMAC1",
+ "JTAG",
+ "MPU_INTPEND"
+};
+/* UE Status High CSR */
+static char *ue_status_hi_desc[] = {
+ "LPCMEMHOST",
+ "MGMT_MAC",
+ "PCS0ONLINE",
+ "MPU_IRAM",
+ "PCS1ONLINE",
+ "PCTL0",
+ "PCTL1",
+ "PMEM",
+ "RR",
+ "TXPB",
+ "RXPP",
+ "XAUI",
+ "TXP",
+ "ARM",
+ "IPC",
+ "HOST2",
+ "HOST3",
+ "HOST4",
+ "HOST5",
+ "HOST6",
+ "HOST7",
+ "HOST8",
+ "HOST9",
+ "NETC"
+ "Unknown",
+ "Unknown",
+ "Unknown",
+ "Unknown",
+ "Unknown",
+ "Unknown",
+ "Unknown",
+ "Unknown"
+};
static void be_queue_free(struct be_adapter *adapter, struct be_queue_info *q)
{
@@ -1673,6 +1743,59 @@ static int be_poll_tx_mcc(struct napi_struct *napi, int budget)
return 1;
}
+static inline bool be_detect_ue(struct be_adapter *adapter)
+{
+ u32 online0 = 0, online1 = 0;
+
+ pci_read_config_dword(adapter->pdev, PCICFG_ONLINE0, &online0);
+
+ pci_read_config_dword(adapter->pdev, PCICFG_ONLINE1, &online1);
+
+ if (!online0 || !online1) {
+ adapter->ue_detected = true;
+ dev_err(&adapter->pdev->dev,
+ "UE Detected!! online0=%d online1=%d\n",
+ online0, online1);
+ return true;
+ }
+
+ return false;
+}
+
+void be_dump_ue(struct be_adapter *adapter)
+{
+ u32 ue_status_lo, ue_status_hi, ue_status_lo_mask, ue_status_hi_mask;
+ u32 i;
+
+ pci_read_config_dword(adapter->pdev,
+ PCICFG_UE_STATUS_LOW, &ue_status_lo);
+ pci_read_config_dword(adapter->pdev,
+ PCICFG_UE_STATUS_HIGH, &ue_status_hi);
+ pci_read_config_dword(adapter->pdev,
+ PCICFG_UE_STATUS_LOW_MASK, &ue_status_lo_mask);
+ pci_read_config_dword(adapter->pdev,
+ PCICFG_UE_STATUS_HI_MASK, &ue_status_hi_mask);
+
+ ue_status_lo = (ue_status_lo & (~ue_status_lo_mask));
+ ue_status_hi = (ue_status_hi & (~ue_status_hi_mask));
+
+ if (ue_status_lo) {
+ for (i = 0; ue_status_lo; ue_status_lo >>= 1, i++) {
+ if (ue_status_lo & 1)
+ dev_err(&adapter->pdev->dev,
+ "UE: %s bit set\n", ue_status_low_desc[i]);
+ }
+ }
+ if (ue_status_hi) {
+ for (i = 0; ue_status_hi; ue_status_hi >>= 1, i++) {
+ if (ue_status_hi & 1)
+ dev_err(&adapter->pdev->dev,
+ "UE: %s bit set\n", ue_status_hi_desc[i]);
+ }
+ }
+
+}
+
static void be_worker(struct work_struct *work)
{
struct be_adapter *adapter =
@@ -1690,6 +1813,10 @@ static void be_worker(struct work_struct *work)
adapter->rx_post_starved = false;
be_post_rx_frags(adapter);
}
+ if (!adapter->ue_detected) {
+ if (be_detect_ue(adapter))
+ be_dump_ue(adapter);
+ }
schedule_delayed_work(&adapter->work, msecs_to_jiffies(1000));
}
--
1.7.0.4
^ permalink raw reply related
* [PATCH net-next 3/4] be2net: change to show correct physical link status
From: Ajit Khaparde @ 2010-07-29 16:17 UTC (permalink / raw)
To: David Miller; +Cc: netdev
link status is wrongly displayed under certain circumstances.
This change fixes it.
Signed-off-by: Somnath K <somnathk@serverengines.com>
Signed-off-by: Ajit Khaparde <ajitk@serverengines.com>
---
drivers/net/benet/be_ethtool.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/benet/be_ethtool.c b/drivers/net/benet/be_ethtool.c
index c0ade24..cd16243 100644
--- a/drivers/net/benet/be_ethtool.c
+++ b/drivers/net/benet/be_ethtool.c
@@ -322,10 +322,11 @@ static int be_get_settings(struct net_device *netdev, struct ethtool_cmd *ecmd)
int status;
u16 intf_type;
- if (adapter->link_speed < 0) {
+ if ((adapter->link_speed < 0) || (!(netdev->flags & IFF_UP))) {
status = be_cmd_link_status_query(adapter, &link_up,
&mac_speed, &link_speed);
+ be_link_status_update(adapter, link_up);
/* link_speed is in units of 10 Mbps */
if (link_speed) {
ecmd->speed = link_speed*10;
--
1.7.0.4
^ permalink raw reply related
* [PATCH net-next 4/4] be2net: fix to avoid sending get_stats request if one is already being processed.
From: Ajit Khaparde @ 2010-07-29 16:18 UTC (permalink / raw)
To: David Miller; +Cc: netdev
GET_STATS request uses the same memory region as the response.
If a new request for get stats is fired before the response for
the previous get_stats request is received, the response will
corrupt the new request, causing the f/w to misbehave.
Signed-off-by: Somnath K <somnathk@serverengines.com>
Signed-off-by: Ajit Khaparde <ajitk@serverengines.com>
---
drivers/net/benet/be.h | 1 +
drivers/net/benet/be_cmds.c | 2 ++
drivers/net/benet/be_main.c | 3 ++-
3 files changed, 5 insertions(+), 1 deletions(-)
diff --git a/drivers/net/benet/be.h b/drivers/net/benet/be.h
index 5e6f581..99197bd 100644
--- a/drivers/net/benet/be.h
+++ b/drivers/net/benet/be.h
@@ -289,6 +289,7 @@ struct be_adapter {
u32 rx_fc; /* Rx flow control */
u32 tx_fc; /* Tx flow control */
bool ue_detected;
+ bool stats_ioctl_sent;
int link_speed;
u8 port_type;
u8 transceiver;
diff --git a/drivers/net/benet/be_cmds.c b/drivers/net/benet/be_cmds.c
index 7fd860d..3d30549 100644
--- a/drivers/net/benet/be_cmds.c
+++ b/drivers/net/benet/be_cmds.c
@@ -75,6 +75,7 @@ static int be_mcc_compl_process(struct be_adapter *adapter,
be_dws_le_to_cpu(&resp->hw_stats,
sizeof(resp->hw_stats));
netdev_stats_update(adapter);
+ adapter->stats_ioctl_sent = false;
}
} else if ((compl_status != MCC_STATUS_NOT_SUPPORTED) &&
(compl->tag0 != OPCODE_COMMON_NTWK_MAC_QUERY)) {
@@ -951,6 +952,7 @@ int be_cmd_get_stats(struct be_adapter *adapter, struct be_dma_mem *nonemb_cmd)
sge->len = cpu_to_le32(nonemb_cmd->size);
be_mcc_notify(adapter);
+ adapter->stats_ioctl_sent = true;
err:
spin_unlock_bh(&adapter->mcc_lock);
diff --git a/drivers/net/benet/be_main.c b/drivers/net/benet/be_main.c
index e4a8ae3..74e146f 100644
--- a/drivers/net/benet/be_main.c
+++ b/drivers/net/benet/be_main.c
@@ -1801,7 +1801,8 @@ static void be_worker(struct work_struct *work)
struct be_adapter *adapter =
container_of(work, struct be_adapter, work.work);
- be_cmd_get_stats(adapter, &adapter->stats.cmd);
+ if (!adapter->stats_ioctl_sent)
+ be_cmd_get_stats(adapter, &adapter->stats.cmd);
/* Set EQ delay */
be_rx_eqd_update(adapter);
--
1.7.0.4
^ permalink raw reply related
* Re: [Uclinux-dist-devel] [PATCH net-next] drivers/net/bfin_mac.c: Use pr_fmt, netdev_<level>
From: Mike Frysinger @ 2010-07-29 16:24 UTC (permalink / raw)
To: Joe Perches; +Cc: Michael Hennerich, uclinux-dist-devel, LKML, netdev
In-Reply-To: <AANLkTimHYS8BRBbomxwyqwayu8eyp+k6TurtsniNTMfs@mail.gmail.com>
On Wed, Jul 28, 2010 at 11:36, Mike Frysinger wrote:
> On Wed, Jul 28, 2010 at 06:00, Joe Perches wrote:
>> On Wed, 2010-07-28 at 03:50 -0400, Mike Frysinger wrote:
>>> On Tue, Jul 27, 2010 at 15:22, Joe Perches wrote:
>>> > $ ./scripts/checkpatch.pl -f drivers/net/bfin_mac.c | grep "^total:"
>>> > total: 2 errors, 25 warnings, 1723 lines checked
>>> > $ ./scripts/checkpatch.pl -f drivers/net/bfin_mac.c | grep "^total:"
>>> > total: 0 errors, 0 warnings, 1743 lines checked
>>> i dislike the mixing of whitespace and useful changes
>>> if they were split, and they worked, then i wouldnt have a problem with them
>>
>> Is that a nak or a dislike?
>
> if you arent going to bother fixing things, then i guess it's a NAK ;)
to be clear, i dont have a problem with the direction you're going
with things, just the steps to get there. i dont like throwing up
"NAK" at postings because that often sounds like a "f-you!" in my
head. i'd rather suggest changes and then ACK the updated patch(es).
-mike
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox