* [PATCH] net: Remove netdevice ec_ptr, no longer used.
From: David Miller @ 2012-05-18 5:40 UTC (permalink / raw)
To: netdev
ECONET is gone, thus this can be deleted as well.
Signed-off-by: David S. Miller <davem@davemloft.net>
---
include/linux/netdevice.h | 1 -
1 file changed, 1 deletion(-)
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index b0f6f22..9c710bd 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1155,7 +1155,6 @@ struct net_device {
struct in_device __rcu *ip_ptr; /* IPv4 specific data */
struct dn_dev __rcu *dn_ptr; /* DECnet specific data */
struct inet6_dev __rcu *ip6_ptr; /* IPv6 specific data */
- void *ec_ptr; /* Econet specific data */
void *ax25_ptr; /* AX.25 specific data */
struct wireless_dev *ieee80211_ptr; /* IEEE 802.11 specific data,
assign before registering */
--
1.7.10.1
^ permalink raw reply related
* Re: [PATCH net-next] ip_frag: struct inet_frags match() method returns a bool
From: David Miller @ 2012-05-18 5:40 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <1337313433.3403.73.camel@edumazet-glaptop>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 18 May 2012 05:57:13 +0200
> From: Eric Dumazet <edumazet@google.com>
>
> - match() method returns a boolean
> - return (A && B && C && D) -> return A && B && C && D
> - fix indentation
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Applied, thanks Eric.
^ permalink raw reply
* Re: [RFC] econet: remove ancient bug ridden protocol
From: David Miller @ 2012-05-18 5:41 UTC (permalink / raw)
To: remi; +Cc: shemminger, netdev
In-Reply-To: <201205180831.47820.remi@remlab.net>
From: "Rémi Denis-Courmont" <remi@remlab.net>
Date: Fri, 18 May 2012 08:31:47 +0300
> Shouldn't this patch remove ec_ptr from struct net_device?
I took care of it with a follow-on patch.
^ permalink raw reply
* Re: [PATCH] STA2X11 CAN: CAN driver for the STA2X11 board
From: Wolfgang Grandegger @ 2012-05-18 6:00 UTC (permalink / raw)
To: Federico Vaga
Cc: Marc Kleine-Budde, linux-can, netdev, linux-kernel,
Giancarlo Asnaghi, Alan Cox
In-Reply-To: <1337288364-17572-1-git-send-email-federico.vaga@gmail.com>
On 05/17/2012 10:59 PM, Federico Vaga wrote:
> Signed-off-by: Federico Vaga <federico.vaga@gmail.com>
> Acked-by: Giancarlo Asnaghi <giancarlo.asnaghi@st.com>
> Cc: Alan Cox <alan@linux.intel.com>
> ---
> drivers/net/can/Kconfig | 11 +
> drivers/net/can/Makefile | 1 +
> drivers/net/can/sta2x11_can.c | 1085 +++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 1097 insertions(+), 0 deletions(-)
> create mode 100644 drivers/net/can/sta2x11_can.c
Thanks for your contribution. At a first glance, this driver looks
similar to the pch_can and the c_can driver. It seems that a C_CAN based
controller is used on that board as well. If that's true, it should be
handled by the C_CAN driver. To get ride of the obsolete pch_can driver,
I sent some time ago the patch "[RFC/PATCH] c_can: add driver for the
PCH CAN controller":
http://marc.info/?t=132991563600003&r=1&w=4
I could serve as base of a generic c_can_pci driver.
Wolfgang.
^ permalink raw reply
* [PATCH net-next] ipv6: bool conversions phase1
From: Eric Dumazet @ 2012-05-18 6:14 UTC (permalink / raw)
To: David Miller; +Cc: netdev
From: Eric Dumazet <edumazet@google.com>
ipv6_opt_accepted() returns a bool, and can use const pointers
ipv6_addr_equal(), ipv6_addr_any(), ipv6_addr_loopback(),
ipv6_addr_orchid() return a bool.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
include/net/ipv6.h | 30 +++++++++++++++---------------
net/ipv6/af_inet6.c | 10 +++++-----
net/ipv6/ip6_tunnel.c | 2 +-
3 files changed, 21 insertions(+), 21 deletions(-)
diff --git a/include/net/ipv6.h b/include/net/ipv6.h
index 5f65e7c..1402139 100644
--- a/include/net/ipv6.h
+++ b/include/net/ipv6.h
@@ -263,7 +263,7 @@ extern struct ipv6_txoptions * ipv6_renew_options(struct sock *sk, struct ipv6_t
struct ipv6_txoptions *ipv6_fixup_options(struct ipv6_txoptions *opt_space,
struct ipv6_txoptions *opt);
-extern int ipv6_opt_accepted(struct sock *sk, struct sk_buff *skb);
+extern bool ipv6_opt_accepted(const struct sock *sk, const struct sk_buff *skb);
int ip6_frag_nqueues(struct net *net);
int ip6_frag_mem(struct net *net);
@@ -332,8 +332,8 @@ static inline void ipv6_addr_set(struct in6_addr *addr,
addr->s6_addr32[3] = w4;
}
-static inline int ipv6_addr_equal(const struct in6_addr *a1,
- const struct in6_addr *a2)
+static inline bool ipv6_addr_equal(const struct in6_addr *a1,
+ const struct in6_addr *a2)
{
return ((a1->s6_addr32[0] ^ a2->s6_addr32[0]) |
(a1->s6_addr32[1] ^ a2->s6_addr32[1]) |
@@ -341,27 +341,27 @@ static inline int ipv6_addr_equal(const struct in6_addr *a1,
(a1->s6_addr32[3] ^ a2->s6_addr32[3])) == 0;
}
-static inline int __ipv6_prefix_equal(const __be32 *a1, const __be32 *a2,
- unsigned int prefixlen)
+static inline bool __ipv6_prefix_equal(const __be32 *a1, const __be32 *a2,
+ unsigned int prefixlen)
{
unsigned int pdw, pbi;
/* check complete u32 in prefix */
pdw = prefixlen >> 5;
if (pdw && memcmp(a1, a2, pdw << 2))
- return 0;
+ return false;
/* check incomplete u32 in prefix */
pbi = prefixlen & 0x1f;
if (pbi && ((a1[pdw] ^ a2[pdw]) & htonl((0xffffffff) << (32 - pbi))))
- return 0;
+ return false;
- return 1;
+ return true;
}
-static inline int ipv6_prefix_equal(const struct in6_addr *a1,
- const struct in6_addr *a2,
- unsigned int prefixlen)
+static inline bool ipv6_prefix_equal(const struct in6_addr *a1,
+ const struct in6_addr *a2,
+ unsigned int prefixlen)
{
return __ipv6_prefix_equal(a1->s6_addr32, a2->s6_addr32,
prefixlen);
@@ -389,19 +389,19 @@ struct ip6_create_arg {
void ip6_frag_init(struct inet_frag_queue *q, void *a);
bool ip6_frag_match(struct inet_frag_queue *q, void *a);
-static inline int ipv6_addr_any(const struct in6_addr *a)
+static inline bool ipv6_addr_any(const struct in6_addr *a)
{
return (a->s6_addr32[0] | a->s6_addr32[1] |
a->s6_addr32[2] | a->s6_addr32[3]) == 0;
}
-static inline int ipv6_addr_loopback(const struct in6_addr *a)
+static inline bool ipv6_addr_loopback(const struct in6_addr *a)
{
return (a->s6_addr32[0] | a->s6_addr32[1] |
a->s6_addr32[2] | (a->s6_addr32[3] ^ htonl(1))) == 0;
}
-static inline int ipv6_addr_v4mapped(const struct in6_addr *a)
+static inline bool ipv6_addr_v4mapped(const struct in6_addr *a)
{
return (a->s6_addr32[0] | a->s6_addr32[1] |
(a->s6_addr32[2] ^ htonl(0x0000ffff))) == 0;
@@ -411,7 +411,7 @@ static inline int ipv6_addr_v4mapped(const struct in6_addr *a)
* Check for a RFC 4843 ORCHID address
* (Overlay Routable Cryptographic Hash Identifiers)
*/
-static inline int ipv6_addr_orchid(const struct in6_addr *a)
+static inline bool ipv6_addr_orchid(const struct in6_addr *a)
{
return (a->s6_addr32[0] & htonl(0xfffffff0)) == htonl(0x20010010);
}
diff --git a/net/ipv6/af_inet6.c b/net/ipv6/af_inet6.c
index 138d498..e22e6d8 100644
--- a/net/ipv6/af_inet6.c
+++ b/net/ipv6/af_inet6.c
@@ -678,10 +678,10 @@ int inet6_sk_rebuild_header(struct sock *sk)
}
EXPORT_SYMBOL_GPL(inet6_sk_rebuild_header);
-int ipv6_opt_accepted(struct sock *sk, struct sk_buff *skb)
+bool ipv6_opt_accepted(const struct sock *sk, const struct sk_buff *skb)
{
- struct ipv6_pinfo *np = inet6_sk(sk);
- struct inet6_skb_parm *opt = IP6CB(skb);
+ const struct ipv6_pinfo *np = inet6_sk(sk);
+ const struct inet6_skb_parm *opt = IP6CB(skb);
if (np->rxopt.all) {
if ((opt->hop && (np->rxopt.bits.hopopts ||
@@ -693,9 +693,9 @@ int ipv6_opt_accepted(struct sock *sk, struct sk_buff *skb)
np->rxopt.bits.osrcrt)) ||
((opt->dst1 || opt->dst0) &&
(np->rxopt.bits.dstopts || np->rxopt.bits.odstopts)))
- return 1;
+ return true;
}
- return 0;
+ return false;
}
EXPORT_SYMBOL_GPL(ipv6_opt_accepted);
diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c
index e65c560..c9015fa 100644
--- a/net/ipv6/ip6_tunnel.c
+++ b/net/ipv6/ip6_tunnel.c
@@ -818,7 +818,7 @@ static void init_tel_txopt(struct ipv6_tel_txoption *opt, __u8 encap_limit)
* 0 else
**/
-static inline int
+static inline bool
ip6_tnl_addr_conflict(const struct ip6_tnl *t, const struct ipv6hdr *hdr)
{
return ipv6_addr_equal(&t->parms.raddr, &hdr->saddr);
^ permalink raw reply related
* Re: [PATCH net-next] ipv6: bool conversions phase1
From: David Miller @ 2012-05-18 6:24 UTC (permalink / raw)
To: eric.dumazet; +Cc: netdev
In-Reply-To: <1337321651.3403.92.camel@edumazet-glaptop>
From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 18 May 2012 08:14:11 +0200
> From: Eric Dumazet <edumazet@google.com>
>
> ipv6_opt_accepted() returns a bool, and can use const pointers
>
> ipv6_addr_equal(), ipv6_addr_any(), ipv6_addr_loopback(),
> ipv6_addr_orchid() return a bool.
>
> Signed-off-by: Eric Dumazet <edumazet@google.com>
Applied, thanks Eric.
^ permalink raw reply
* [v2 PATCH] netfilter: xt_HMARK: fix endian bugs and warnings
From: Hans Schillstrom @ 2012-05-18 8:35 UTC (permalink / raw)
To: pablo, kaber, jengelh, netfilter-devel, netdev, dan.carpenter
Cc: hans.schillstrom, Hans Schillstrom
A mix of u32 and __be32 causes endian warning.
The hash value produced is now the same for big and little endian machines.
i.e. a mix of Big and Little endian in a cluster is now possible.
Reported-by: Dan Carpenter <dan.carpenter@oracle.com>
Signed-off-by: Hans Schillstrom <hans@schillstrom.com>
---
include/linux/netfilter/xt_HMARK.h | 5 +++
net/netfilter/xt_HMARK.c | 69 ++++++++++++++++++++----------------
2 files changed, 43 insertions(+), 31 deletions(-)
diff --git a/include/linux/netfilter/xt_HMARK.h b/include/linux/netfilter/xt_HMARK.h
index abb1650..826fc58 100644
--- a/include/linux/netfilter/xt_HMARK.h
+++ b/include/linux/netfilter/xt_HMARK.h
@@ -27,7 +27,12 @@ union hmark_ports {
__u16 src;
__u16 dst;
} p16;
+ struct {
+ __be16 src;
+ __be16 dst;
+ } b16;
__u32 v32;
+ __be32 b32;
};
struct xt_hmark_info {
diff --git a/net/netfilter/xt_HMARK.c b/net/netfilter/xt_HMARK.c
index 0a96a43..5119666 100644
--- a/net/netfilter/xt_HMARK.c
+++ b/net/netfilter/xt_HMARK.c
@@ -32,13 +32,13 @@ MODULE_ALIAS("ipt_HMARK");
MODULE_ALIAS("ip6t_HMARK");
struct hmark_tuple {
- u32 src;
- u32 dst;
+ __be32 src;
+ __be32 dst;
union hmark_ports uports;
uint8_t proto;
};
-static inline u32 hmark_addr6_mask(const __u32 *addr32, const __u32 *mask)
+static inline __be32 hmark_addr6_mask(const __be32 *addr32, const __be32 *mask)
{
return (addr32[0] & mask[0]) ^
(addr32[1] & mask[1]) ^
@@ -46,8 +46,8 @@ static inline u32 hmark_addr6_mask(const __u32 *addr32, const __u32 *mask)
(addr32[3] & mask[3]);
}
-static inline u32
-hmark_addr_mask(int l3num, const __u32 *addr32, const __u32 *mask)
+static inline __be32
+hmark_addr_mask(int l3num, const __be32 *addr32, const __be32 *mask)
{
switch (l3num) {
case AF_INET:
@@ -58,6 +58,22 @@ hmark_addr_mask(int l3num, const __u32 *addr32, const __u32 *mask)
return 0;
}
+static inline void hmark_swap_ports(union hmark_ports *uports,
+ const struct xt_hmark_info *info)
+{
+ union hmark_ports hp;
+ u16 src,dst;
+
+ hp.b32 = (uports->b32 & info->port_mask.b32) | info->port_set.b32;
+ src = ntohs(hp.b16.src);
+ dst = ntohs(hp.b16.dst);
+
+ if (dst > src)
+ uports->v32 = (dst << 16) | src;
+ else
+ uports->v32 = (src << 16) | dst;
+}
+
static int
hmark_ct_set_htuple(const struct sk_buff *skb, struct hmark_tuple *t,
const struct xt_hmark_info *info)
@@ -74,22 +90,19 @@ hmark_ct_set_htuple(const struct sk_buff *skb, struct hmark_tuple *t,
otuple = &ct->tuplehash[IP_CT_DIR_ORIGINAL].tuple;
rtuple = &ct->tuplehash[IP_CT_DIR_REPLY].tuple;
- t->src = hmark_addr_mask(otuple->src.l3num, otuple->src.u3.all,
- info->src_mask.all);
- t->dst = hmark_addr_mask(otuple->src.l3num, rtuple->src.u3.all,
- info->dst_mask.all);
+ t->src = hmark_addr_mask(otuple->src.l3num, otuple->src.u3.ip6,
+ info->src_mask.ip6);
+ t->dst = hmark_addr_mask(otuple->src.l3num, rtuple->src.u3.ip6,
+ info->dst_mask.ip6);
if (info->flags & XT_HMARK_FLAG(XT_HMARK_METHOD_L3))
return 0;
t->proto = nf_ct_protonum(ct);
if (t->proto != IPPROTO_ICMP) {
- t->uports.p16.src = otuple->src.u.all;
- t->uports.p16.dst = rtuple->src.u.all;
- t->uports.v32 = (t->uports.v32 & info->port_mask.v32) |
- info->port_set.v32;
- if (t->uports.p16.dst < t->uports.p16.src)
- swap(t->uports.p16.dst, t->uports.p16.src);
+ t->uports.b16.src = otuple->src.u.all;
+ t->uports.b16.dst = rtuple->src.u.all;
+ hmark_swap_ports(&t->uports, info);
}
return 0;
@@ -102,11 +115,13 @@ static inline u32
hmark_hash(struct hmark_tuple *t, const struct xt_hmark_info *info)
{
u32 hash;
+ u32 src = ntohl(t->src);
+ u32 dst = ntohl(t->dst);
- if (t->dst < t->src)
- swap(t->src, t->dst);
+ if (dst < src)
+ swap(src, dst);
- hash = jhash_3words(t->src, t->dst, t->uports.v32, info->hashrnd);
+ hash = jhash_3words(src, dst, t->uports.v32, info->hashrnd);
hash = hash ^ (t->proto & info->proto_mask);
return (((u64)hash * info->hmodulus) >> 32) + info->hoffset;
@@ -125,12 +140,7 @@ hmark_set_tuple_ports(const struct sk_buff *skb, unsigned int nhoff,
nhoff += protoff;
if (skb_copy_bits(skb, nhoff, &t->uports, sizeof(t->uports)) < 0)
return;
-
- t->uports.v32 = (t->uports.v32 & info->port_mask.v32) |
- info->port_set.v32;
-
- if (t->uports.p16.dst < t->uports.p16.src)
- swap(t->uports.p16.dst, t->uports.p16.src);
+ hmark_swap_ports(&t->uports, info);
}
#if IS_ENABLED(CONFIG_IP6_NF_IPTABLES)
@@ -178,8 +188,8 @@ hmark_pkt_set_htuple_ipv6(const struct sk_buff *skb, struct hmark_tuple *t,
return -1;
}
noicmp:
- t->src = hmark_addr6_mask(ip6->saddr.s6_addr32, info->src_mask.all);
- t->dst = hmark_addr6_mask(ip6->daddr.s6_addr32, info->dst_mask.all);
+ t->src = hmark_addr6_mask(ip6->saddr.s6_addr32, info->src_mask.ip6);
+ t->dst = hmark_addr6_mask(ip6->daddr.s6_addr32, info->dst_mask.ip6);
if (info->flags & XT_HMARK_FLAG(XT_HMARK_METHOD_L3))
return 0;
@@ -255,11 +265,8 @@ hmark_pkt_set_htuple_ipv4(const struct sk_buff *skb, struct hmark_tuple *t,
}
}
- t->src = (__force u32) ip->saddr;
- t->dst = (__force u32) ip->daddr;
-
- t->src &= info->src_mask.ip;
- t->dst &= info->dst_mask.ip;
+ t->src = ip->saddr & info->src_mask.ip;
+ t->dst = ip->daddr & info->dst_mask.ip;
if (info->flags & XT_HMARK_FLAG(XT_HMARK_METHOD_L3))
return 0;
--
1.7.2.3
^ permalink raw reply related
* [PATCH net-next v5] be2net: Fix to allow get/set of debug levels in the firmware.
From: Somnath Kotur @ 2012-05-18 8:59 UTC (permalink / raw)
To: netdev, bhutchings; +Cc: Somnath Kotur, Suresh Reddy
Patch re-spin.
Incorporated review comments by Ben Hutchings.
Signed-off-by: Suresh Reddy <suresh.reddy@emulex.com>
Signed-off-by: Somnath Kotur <somnath.kotur@emulex.com>
---
drivers/net/ethernet/emulex/benet/be.h | 3 +
drivers/net/ethernet/emulex/benet/be_cmds.c | 56 +++++++++++++++++
drivers/net/ethernet/emulex/benet/be_cmds.h | 57 +++++++++++++++++
drivers/net/ethernet/emulex/benet/be_ethtool.c | 77 ++++++++++++++++++++++++
drivers/net/ethernet/emulex/benet/be_main.c | 37 +++++++++++
5 files changed, 230 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/emulex/benet/be.h b/drivers/net/ethernet/emulex/benet/be.h
index ecf1a81..ff4eb8f 100644
--- a/drivers/net/ethernet/emulex/benet/be.h
+++ b/drivers/net/ethernet/emulex/benet/be.h
@@ -421,6 +421,7 @@ struct be_adapter {
bool wol;
u32 max_pmac_cnt; /* Max secondary UC MACs programmable */
u32 uc_macs; /* Count of secondary UC MAC programmed */
+ u32 msg_enable;
};
#define be_physfn(adapter) (!adapter->virtfn)
@@ -603,4 +604,6 @@ extern void be_parse_stats(struct be_adapter *adapter);
extern int be_load_fw(struct be_adapter *adapter, u8 *func);
extern bool be_is_wol_supported(struct be_adapter *adapter);
extern bool be_pause_supported(struct be_adapter *adapter);
+extern u32 be_get_fw_log_level(struct be_adapter *adapter);
+
#endif /* BE_H */
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.c b/drivers/net/ethernet/emulex/benet/be_cmds.c
index 43167e8..b24623c 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.c
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.c
@@ -2589,4 +2589,60 @@ err:
mutex_unlock(&adapter->mbox_lock);
pci_free_consistent(adapter->pdev, cmd.size, cmd.va, cmd.dma);
return status;
+
+}
+int be_cmd_get_ext_fat_capabilites(struct be_adapter *adapter,
+ struct be_dma_mem *cmd)
+{
+ struct be_mcc_wrb *wrb;
+ struct be_cmd_req_get_ext_fat_caps *req;
+ int status;
+
+ if (mutex_lock_interruptible(&adapter->mbox_lock))
+ return -1;
+
+ wrb = wrb_from_mbox(adapter);
+ if (!wrb) {
+ status = -EBUSY;
+ goto err;
+ }
+
+ req = cmd->va;
+ be_wrb_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_COMMON,
+ OPCODE_COMMON_GET_EXT_FAT_CAPABILITES,
+ cmd->size, wrb, cmd);
+ req->parameter_type = cpu_to_le32(1);
+
+ status = be_mbox_notify_wait(adapter);
+err:
+ mutex_unlock(&adapter->mbox_lock);
+ return status;
+}
+
+int be_cmd_set_ext_fat_capabilites(struct be_adapter *adapter,
+ struct be_dma_mem *cmd,
+ struct be_fat_conf_params *configs)
+{
+ struct be_mcc_wrb *wrb;
+ struct be_cmd_req_set_ext_fat_caps *req;
+ int status;
+
+ spin_lock_bh(&adapter->mcc_lock);
+
+ wrb = wrb_from_mccq(adapter);
+ if (!wrb) {
+ status = -EBUSY;
+ goto err;
+ }
+
+ req = cmd->va;
+ memcpy(&req->set_params, configs, sizeof(struct be_fat_conf_params));
+ be_wrb_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_COMMON,
+ OPCODE_COMMON_SET_EXT_FAT_CAPABILITES,
+ cmd->size, wrb, cmd);
+
+ status = be_mcc_notify_wait(adapter);
+err:
+ spin_unlock_bh(&adapter->mcc_lock);
+ return status;
}
diff --git a/drivers/net/ethernet/emulex/benet/be_cmds.h b/drivers/net/ethernet/emulex/benet/be_cmds.h
index 944f031..0b1029b 100644
--- a/drivers/net/ethernet/emulex/benet/be_cmds.h
+++ b/drivers/net/ethernet/emulex/benet/be_cmds.h
@@ -189,6 +189,8 @@ struct be_mcc_mailbox {
#define OPCODE_COMMON_GET_PHY_DETAILS 102
#define OPCODE_COMMON_SET_DRIVER_FUNCTION_CAP 103
#define OPCODE_COMMON_GET_CNTL_ADDITIONAL_ATTRIBUTES 121
+#define OPCODE_COMMON_GET_EXT_FAT_CAPABILITES 125
+#define OPCODE_COMMON_SET_EXT_FAT_CAPABILITES 126
#define OPCODE_COMMON_GET_MAC_LIST 147
#define OPCODE_COMMON_SET_MAC_LIST 148
#define OPCODE_COMMON_GET_HSW_CONFIG 152
@@ -1602,6 +1604,56 @@ static inline void *be_erx_stats_from_cmd(struct be_adapter *adapter)
}
}
+
+/************** get fat capabilites *******************/
+#define MAX_MODULES 27
+#define MAX_MODES 4
+#define MODE_UART 0
+#define FW_LOG_LEVEL_DEFAULT 48
+#define FW_LOG_LEVEL_FATAL 64
+
+struct ext_fat_mode {
+ u8 mode;
+ u8 rsvd0;
+ u16 port_mask;
+ u32 dbg_lvl;
+ u64 fun_mask;
+} __packed;
+
+struct ext_fat_modules {
+ u8 modules_str[32];
+ u32 modules_id;
+ u32 num_modes;
+ struct ext_fat_mode trace_lvl[MAX_MODES];
+} __packed;
+
+struct be_fat_conf_params {
+ u32 max_log_entries;
+ u32 log_entry_size;
+ u8 log_type;
+ u8 max_log_funs;
+ u8 max_log_ports;
+ u8 rsvd0;
+ u32 supp_modes;
+ u32 num_modules;
+ struct ext_fat_modules module[MAX_MODULES];
+} __packed;
+
+struct be_cmd_req_get_ext_fat_caps {
+ struct be_cmd_req_hdr hdr;
+ u32 parameter_type;
+};
+
+struct be_cmd_resp_get_ext_fat_caps {
+ struct be_cmd_resp_hdr hdr;
+ struct be_fat_conf_params get_params;
+};
+
+struct be_cmd_req_set_ext_fat_caps {
+ struct be_cmd_req_hdr hdr;
+ struct be_fat_conf_params set_params;
+};
+
extern int be_pci_fnum_get(struct be_adapter *adapter);
extern int be_cmd_POST(struct be_adapter *adapter);
extern int be_cmd_mac_addr_query(struct be_adapter *adapter, u8 *mac_addr,
@@ -1707,4 +1759,9 @@ extern int be_cmd_set_hsw_config(struct be_adapter *adapter, u16 pvid,
extern int be_cmd_get_hsw_config(struct be_adapter *adapter, u16 *pvid,
u32 domain, u16 intf_id);
extern int be_cmd_get_acpi_wol_cap(struct be_adapter *adapter);
+extern int be_cmd_get_ext_fat_capabilites(struct be_adapter *adapter,
+ struct be_dma_mem *cmd);
+extern int be_cmd_set_ext_fat_capabilites(struct be_adapter *adapter,
+ struct be_dma_mem *cmd,
+ struct be_fat_conf_params *cfgs);
diff --git a/drivers/net/ethernet/emulex/benet/be_ethtool.c b/drivers/net/ethernet/emulex/benet/be_ethtool.c
index 747f68f..63e51d4 100644
--- a/drivers/net/ethernet/emulex/benet/be_ethtool.c
+++ b/drivers/net/ethernet/emulex/benet/be_ethtool.c
@@ -878,6 +878,81 @@ be_read_eeprom(struct net_device *netdev, struct ethtool_eeprom *eeprom,
return status;
}
+static u32 be_get_msg_level(struct net_device *netdev)
+{
+ struct be_adapter *adapter = netdev_priv(netdev);
+
+ if (lancer_chip(adapter)) {
+ dev_err(&adapter->pdev->dev, "Operation not supported\n");
+ return -EOPNOTSUPP;
+ }
+
+ return adapter->msg_enable;
+}
+
+static void be_set_fw_log_level(struct be_adapter *adapter, u32 level)
+{
+ struct be_dma_mem extfat_cmd;
+ struct be_fat_conf_params *cfgs;
+ int status;
+ int i, j;
+
+ memset(&extfat_cmd, 0, sizeof(struct be_dma_mem));
+ extfat_cmd.size = sizeof(struct be_cmd_resp_get_ext_fat_caps);
+ extfat_cmd.va = pci_alloc_consistent(adapter->pdev, extfat_cmd.size,
+ &extfat_cmd.dma);
+ if (!extfat_cmd.va) {
+ dev_err(&adapter->pdev->dev, "%s: Memory allocation failure\n",
+ __func__);
+ goto err;
+ }
+ status = be_cmd_get_ext_fat_capabilites(adapter, &extfat_cmd);
+ if (!status) {
+ cfgs = (struct be_fat_conf_params *)(extfat_cmd.va +
+ sizeof(struct be_cmd_resp_hdr));
+ for (i = 0; i < cfgs->num_modules; i++) {
+ for (j = 0; j < cfgs->module[i].num_modes; j++) {
+ if (cfgs->module[i].trace_lvl[j].mode ==
+ MODE_UART)
+ cfgs->module[i].trace_lvl[j].dbg_lvl =
+ cpu_to_le32(level);
+ }
+ }
+ status = be_cmd_set_ext_fat_capabilites(adapter, &extfat_cmd,
+ cfgs);
+ if (status)
+ dev_err(&adapter->pdev->dev,
+ "Message level set failed\n");
+ } else {
+ dev_err(&adapter->pdev->dev, "Message level get failed\n");
+ }
+
+ pci_free_consistent(adapter->pdev, extfat_cmd.size, extfat_cmd.va,
+ extfat_cmd.dma);
+err:
+ return;
+}
+
+static void be_set_msg_level(struct net_device *netdev, u32 level)
+{
+ struct be_adapter *adapter = netdev_priv(netdev);
+
+ if (lancer_chip(adapter)) {
+ dev_err(&adapter->pdev->dev, "Operation not supported\n");
+ return;
+ }
+
+ if (adapter->msg_enable == level)
+ return;
+
+ if ((level & NETIF_MSG_HW) != (adapter->msg_enable & NETIF_MSG_HW))
+ be_set_fw_log_level(adapter, level & NETIF_MSG_HW ?
+ FW_LOG_LEVEL_DEFAULT : FW_LOG_LEVEL_FATAL);
+ adapter->msg_enable = level;
+
+ return;
+}
+
const struct ethtool_ops be_ethtool_ops = {
.get_settings = be_get_settings,
.get_drvinfo = be_get_drvinfo,
@@ -893,6 +968,8 @@ const struct ethtool_ops be_ethtool_ops = {
.set_pauseparam = be_set_pauseparam,
.get_strings = be_get_stat_strings,
.set_phys_id = be_set_phys_id,
+ .get_msglevel = be_get_msg_level,
+ .set_msglevel = be_set_msg_level,
.get_sset_count = be_get_sset_count,
.get_ethtool_stats = be_get_ethtool_stats,
.get_regs_len = be_get_reg_len,
diff --git a/drivers/net/ethernet/emulex/benet/be_main.c b/drivers/net/ethernet/emulex/benet/be_main.c
index a01f734..081c777 100644
--- a/drivers/net/ethernet/emulex/benet/be_main.c
+++ b/drivers/net/ethernet/emulex/benet/be_main.c
@@ -3403,9 +3403,43 @@ bool be_is_wol_supported(struct be_adapter *adapter)
!be_is_wol_excluded(adapter)) ? true : false;
}
+u32 be_get_fw_log_level(struct be_adapter *adapter)
+{
+ struct be_dma_mem extfat_cmd;
+ struct be_fat_conf_params *cfgs;
+ int status;
+ u32 level = 0;
+ int j;
+
+ memset(&extfat_cmd, 0, sizeof(struct be_dma_mem));
+ extfat_cmd.size = sizeof(struct be_cmd_resp_get_ext_fat_caps);
+ extfat_cmd.va = pci_alloc_consistent(adapter->pdev, extfat_cmd.size,
+ &extfat_cmd.dma);
+
+ if (!extfat_cmd.va) {
+ dev_err(&adapter->pdev->dev, "%s: Memory allocation failure\n",
+ __func__);
+ goto err;
+ }
+
+ status = be_cmd_get_ext_fat_capabilites(adapter, &extfat_cmd);
+ if (!status) {
+ cfgs = (struct be_fat_conf_params *)(extfat_cmd.va +
+ sizeof(struct be_cmd_resp_hdr));
+ for (j = 0; j < cfgs->module[0].num_modes; j++) {
+ if (cfgs->module[0].trace_lvl[j].mode == MODE_UART)
+ level = cfgs->module[0].trace_lvl[j].dbg_lvl;
+ }
+ }
+ pci_free_consistent(adapter->pdev, extfat_cmd.size, extfat_cmd.va,
+ extfat_cmd.dma);
+err:
+ return level;
+}
static int be_get_initial_config(struct be_adapter *adapter)
{
int status;
+ u32 level;
status = be_cmd_query_fw_cfg(adapter, &adapter->port_num,
&adapter->function_mode, &adapter->function_caps);
@@ -3443,6 +3477,9 @@ static int be_get_initial_config(struct be_adapter *adapter)
if (be_is_wol_supported(adapter))
adapter->wol = true;
+ level = be_get_fw_log_level(adapter);
+ adapter->msg_enable = level <= FW_LOG_LEVEL_DEFAULT ? NETIF_MSG_HW : 0;
+
return 0;
}
--
1.5.6.1
^ permalink raw reply related
* [PATCH] pktgen: fix module unload for good
From: Eric Dumazet @ 2012-05-18 9:52 UTC (permalink / raw)
To: David Miller; +Cc: netdev
From: Eric Dumazet <edumazet@google.com>
commit c57b5468406 (pktgen: fix crash at module unload) did a very poor
job with list primitives.
1) list_splice() arguments were in the wrong order
2) list_splice(list, head) has undefined behavior if head is not
initialized.
3) We should use the list_splice_init() variant to clear pktgen_threads
list.
Signed-off-by: Eric Dumazet <edumazet@google.com>
---
Sorry for the mess, I really dont understand how my machine did not
crash with c57b5468406 commit...
net/core/pktgen.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/net/core/pktgen.c b/net/core/pktgen.c
index 77a5998..b81369b 100644
--- a/net/core/pktgen.c
+++ b/net/core/pktgen.c
@@ -3755,13 +3755,13 @@ static void __exit pg_cleanup(void)
{
struct pktgen_thread *t;
struct list_head *q, *n;
- struct list_head list;
+ LIST_HEAD(list);
/* Stop all interfaces & threads */
pktgen_exiting = true;
mutex_lock(&pktgen_thread_lock);
- list_splice(&list, &pktgen_threads);
+ list_splice_init(&pktgen_threads, &list);
mutex_unlock(&pktgen_thread_lock);
list_for_each_safe(q, n, &list) {
^ permalink raw reply related
* Re: [V2 PATCH 9/9] vhost: zerocopy: poll vq in zerocopy callback
From: Jason Wang @ 2012-05-18 9:58 UTC (permalink / raw)
To: Shirley Ma
Cc: Michael S. Tsirkin, eric.dumazet, netdev, linux-kernel, ebiederm,
davem
In-Reply-To: <1337268862.10741.58.camel@oc3660625478.ibm.com>
On 05/17/2012 11:34 PM, Shirley Ma wrote:
> On Thu, 2012-05-17 at 10:50 +0800, Jason Wang wrote:
>> The problem is we may stop the tx queue when there no enough capacity
>> to
>> place packets, at this moment we depends on the tx interrupt to
>> re-enable the tx queue. So if we didn't poll the vhost during
>> callback,
>> guest may lose the tx interrupt to re-enable the tx queue which could
>> stall the whole tx queue.
> VHOST_MAX_PEND should handle the capacity.
>
> Hasn't the above situation been handled in handle_tx() code?:
> ...
> if (unlikely(num_pends> VHOST_MAX_PEND)) {
> tx_poll_start(net, sock);
> set_bit(SOCK_ASYNC_NOSPACE,&sock->flags);
> break;
> }
> ...
>
> Thanks
> Shirley
It may not help in because:
- tx polling depends on skb_orphan() which is often called by device
driver when it place the packet into the queue of the devices instead
of when the packets were sent. So it was too early for vhost to be
notified.
- it only works when the pending DMAs exceeds VHOST_MAX_PEND, it's
highly possible that guest needs to be notified when the pending packets
isn't so much.
So this piece of code may not help and could be removed and we need to
poll the virt-queue during zerocopy callback ( through it could be
further optimized but may not be easy).
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply
* [PATCH 01/10] cxgb4: Detect DB FULL events and notify RDMA ULD.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: roland-BHEL68pLQRGGvPXPguhicg, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
divy-ut6Up61K2wZBDgjK7y7TUQ, dm-ut6Up61K2wZBDgjK7y7TUQ,
kumaras-ut6Up61K2wZBDgjK7y7TUQ,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Vipul Pandya <vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 4 +
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 77 +++++++++++++++++++++++
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 7 ++
drivers/net/ethernet/chelsio/cxgb4/sge.c | 6 ++
drivers/net/ethernet/chelsio/cxgb4/t4_hw.c | 9 +++
5 files changed, 103 insertions(+), 0 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
index 0fe1885..f91b259 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
@@ -504,6 +504,8 @@ struct adapter {
void **tid_release_head;
spinlock_t tid_release_lock;
struct work_struct tid_release_task;
+ struct work_struct db_full_task;
+ struct work_struct db_drop_task;
bool tid_release_task_busy;
struct dentry *debugfs_root;
@@ -719,4 +721,6 @@ int t4_ctrl_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
int t4_ofld_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
unsigned int vf, unsigned int eqid);
int t4_handle_fw_rpl(struct adapter *adap, const __be64 *rpl);
+void t4_db_full(struct adapter *adapter);
+void t4_db_dropped(struct adapter *adapter);
#endif /* __CXGB4_H__ */
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index b126b98..c243f93 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -2366,6 +2366,16 @@ unsigned int cxgb4_port_chan(const struct net_device *dev)
}
EXPORT_SYMBOL(cxgb4_port_chan);
+unsigned int cxgb4_dbfifo_count(const struct net_device *dev, int lpfifo)
+{
+ struct adapter *adap = netdev2adap(dev);
+ u32 v;
+
+ v = t4_read_reg(adap, A_SGE_DBFIFO_STATUS);
+ return lpfifo ? G_LP_COUNT(v) : G_HP_COUNT(v);
+}
+EXPORT_SYMBOL(cxgb4_dbfifo_count);
+
/**
* cxgb4_port_viid - get the VI id of a port
* @dev: the net device for the port
@@ -2446,6 +2456,69 @@ static struct notifier_block cxgb4_netevent_nb = {
.notifier_call = netevent_cb
};
+static void notify_rdma_uld(struct adapter *adap, enum cxgb4_control cmd)
+{
+ mutex_lock(&uld_mutex);
+ if (adap->uld_handle[CXGB4_ULD_RDMA])
+ ulds[CXGB4_ULD_RDMA].control(adap->uld_handle[CXGB4_ULD_RDMA],
+ cmd);
+ mutex_unlock(&uld_mutex);
+}
+
+static void process_db_full(struct work_struct *work)
+{
+ struct adapter *adap;
+ static int delay = 1000;
+ u32 v;
+
+ adap = container_of(work, struct adapter, db_full_task);
+
+
+ /* stop LLD queues */
+
+ notify_rdma_uld(adap, CXGB4_CONTROL_DB_FULL);
+ do {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(delay));
+ v = t4_read_reg(adap, A_SGE_DBFIFO_STATUS);
+ if (G_LP_COUNT(v) == 0 && G_HP_COUNT(v) == 0)
+ break;
+ } while (1);
+ notify_rdma_uld(adap, CXGB4_CONTROL_DB_EMPTY);
+
+
+ /*
+ * The more we get db full interrupts, the more we'll delay
+ * in re-enabling db rings on queues, capped off at 200ms.
+ */
+ delay = min(delay << 1, 200000);
+
+ /* resume LLD queues */
+}
+
+static void process_db_drop(struct work_struct *work)
+{
+ struct adapter *adap;
+ adap = container_of(work, struct adapter, db_drop_task);
+
+
+ /*
+ * sync the PIDX values in HW and SW for LLD queues.
+ */
+
+ notify_rdma_uld(adap, CXGB4_CONTROL_DB_DROP);
+}
+
+void t4_db_full(struct adapter *adap)
+{
+ schedule_work(&adap->db_full_task);
+}
+
+void t4_db_dropped(struct adapter *adap)
+{
+ schedule_work(&adap->db_drop_task);
+}
+
static void uld_attach(struct adapter *adap, unsigned int uld)
{
void *handle;
@@ -2649,6 +2722,8 @@ static void cxgb_down(struct adapter *adapter)
{
t4_intr_disable(adapter);
cancel_work_sync(&adapter->tid_release_task);
+ cancel_work_sync(&adapter->db_full_task);
+ cancel_work_sync(&adapter->db_drop_task);
adapter->tid_release_task_busy = false;
adapter->tid_release_head = NULL;
@@ -3601,6 +3676,8 @@ static int __devinit init_one(struct pci_dev *pdev,
spin_lock_init(&adapter->tid_release_lock);
INIT_WORK(&adapter->tid_release_task, process_tid_release_list);
+ INIT_WORK(&adapter->db_full_task, process_db_full);
+ INIT_WORK(&adapter->db_drop_task, process_db_drop);
err = t4_prep_adapter(adapter);
if (err)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
index b1d39b8..5cc2f27 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
@@ -163,6 +163,12 @@ enum cxgb4_state {
CXGB4_STATE_DETACH
};
+enum cxgb4_control {
+ CXGB4_CONTROL_DB_FULL,
+ CXGB4_CONTROL_DB_EMPTY,
+ CXGB4_CONTROL_DB_DROP,
+};
+
struct pci_dev;
struct l2t_data;
struct net_device;
@@ -225,6 +231,7 @@ struct cxgb4_uld_info {
int cxgb4_register_uld(enum cxgb4_uld type, const struct cxgb4_uld_info *p);
int cxgb4_unregister_uld(enum cxgb4_uld type);
int cxgb4_ofld_send(struct net_device *dev, struct sk_buff *skb);
+unsigned int cxgb4_dbfifo_count(const struct net_device *dev, int lpfifo);
unsigned int cxgb4_port_chan(const struct net_device *dev);
unsigned int cxgb4_port_viid(const struct net_device *dev);
unsigned int cxgb4_port_idx(const struct net_device *dev);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c
index 2dae795..234c157 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c
@@ -2415,6 +2415,12 @@ void t4_sge_init(struct adapter *adap)
RXPKTCPLMODE |
(STAT_LEN == 128 ? EGRSTATUSPAGESIZE : 0));
+ t4_set_reg_field(adap, A_SGE_DBFIFO_STATUS,
+ V_HP_INT_THRESH(5) | V_LP_INT_THRESH(5),
+ V_HP_INT_THRESH(5) | V_LP_INT_THRESH(5));
+ t4_set_reg_field(adap, A_SGE_DOORBELL_CONTROL, F_ENABLE_DROP,
+ F_ENABLE_DROP);
+
for (i = v = 0; i < 32; i += 4)
v |= (PAGE_SHIFT - 10) << i;
t4_write_reg(adap, SGE_HOST_PAGE_SIZE, v);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
index d1ec111..13609bf 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
@@ -1013,6 +1013,8 @@ static void sge_intr_handler(struct adapter *adapter)
{ ERR_INVALID_CIDX_INC,
"SGE GTS CIDX increment too large", -1, 0 },
{ ERR_CPL_OPCODE_0, "SGE received 0-length CPL", -1, 0 },
+ { F_DBFIFO_LP_INT, NULL, -1, 0 },
+ { F_DBFIFO_HP_INT, NULL, -1, 0 },
{ ERR_DROPPED_DB, "SGE doorbell dropped", -1, 0 },
{ ERR_DATA_CPL_ON_HIGH_QID1 | ERR_DATA_CPL_ON_HIGH_QID0,
"SGE IQID > 1023 received CPL for FL", -1, 0 },
@@ -1042,6 +1044,12 @@ static void sge_intr_handler(struct adapter *adapter)
t4_write_reg(adapter, SGE_INT_CAUSE2, v >> 32);
}
+ err = t4_read_reg(adapter, A_SGE_INT_CAUSE3);
+ if (err & (F_DBFIFO_HP_INT|F_DBFIFO_LP_INT))
+ t4_db_full(adapter);
+ if (err & F_ERR_DROPPED_DB)
+ t4_db_dropped(adapter);
+
if (t4_handle_intr_status(adapter, SGE_INT_CAUSE3, sge_intr_info) ||
v != 0)
t4_fatal_err(adapter);
@@ -1513,6 +1521,7 @@ void t4_intr_enable(struct adapter *adapter)
ERR_BAD_DB_PIDX2 | ERR_BAD_DB_PIDX1 |
ERR_BAD_DB_PIDX0 | ERR_ING_CTXT_PRIO |
ERR_EGR_CTXT_PRIO | INGRESS_SIZE_ERR |
+ F_DBFIFO_HP_INT | F_DBFIFO_LP_INT |
EGRESS_SIZE_ERR);
t4_write_reg(adapter, MYPF_REG(PL_PF_INT_ENABLE), PF_INTR_MASK);
t4_set_reg_field(adapter, PL_INT_MAP0, 0, 1 << pf);
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 02/10] cxgb4: Common platform specific changes for DB Drop Recovery
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: roland-BHEL68pLQRGGvPXPguhicg, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
divy-ut6Up61K2wZBDgjK7y7TUQ, dm-ut6Up61K2wZBDgjK7y7TUQ,
kumaras-ut6Up61K2wZBDgjK7y7TUQ,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Add platform-specific callback functions for interrupts. This is
needed to do a single read-clear of the CAUSE register and then call
out to platform specific functions for DB threshold interrupts and DB
drop interrupts.
Add t4_mem_win_read_len() - mem-window reads for arbitrary lengths.
This is used to read the CIDX/PIDX values from EC contexts during DB
drop recovery.
Add t4_fwaddrspace_write() - sends addrspace write cmds to the fw.
Needed to flush the sge eq context cache.
Signed-off-by: Vipul Pandya <vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 3 +
drivers/net/ethernet/chelsio/cxgb4/t4_hw.c | 69 +++++++++++++++++++++++----
2 files changed, 61 insertions(+), 11 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
index f91b259..5f3c0a7 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
@@ -723,4 +723,7 @@ int t4_ofld_eq_free(struct adapter *adap, unsigned int mbox, unsigned int pf,
int t4_handle_fw_rpl(struct adapter *adap, const __be64 *rpl);
void t4_db_full(struct adapter *adapter);
void t4_db_dropped(struct adapter *adapter);
+int t4_mem_win_read_len(struct adapter *adap, u32 addr, __be32 *data, int len);
+int t4_fwaddrspace_write(struct adapter *adap, unsigned int mbox,
+ u32 addr, u32 val);
#endif /* __CXGB4_H__ */
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
index 13609bf..32e1dd5 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_hw.c
@@ -868,11 +868,14 @@ int t4_restart_aneg(struct adapter *adap, unsigned int mbox, unsigned int port)
return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
}
+typedef void (*int_handler_t)(struct adapter *adap);
+
struct intr_info {
unsigned int mask; /* bits to check in interrupt status */
const char *msg; /* message to print or NULL */
short stat_idx; /* stat counter to increment or -1 */
unsigned short fatal; /* whether the condition reported is fatal */
+ int_handler_t int_handler; /* platform-specific int handler */
};
/**
@@ -905,6 +908,8 @@ static int t4_handle_intr_status(struct adapter *adapter, unsigned int reg,
} else if (acts->msg && printk_ratelimit())
dev_warn(adapter->pdev_dev, "%s (0x%x)\n", acts->msg,
status & acts->mask);
+ if (acts->int_handler)
+ acts->int_handler(adapter);
mask |= acts->mask;
}
status &= mask;
@@ -1013,9 +1018,9 @@ static void sge_intr_handler(struct adapter *adapter)
{ ERR_INVALID_CIDX_INC,
"SGE GTS CIDX increment too large", -1, 0 },
{ ERR_CPL_OPCODE_0, "SGE received 0-length CPL", -1, 0 },
- { F_DBFIFO_LP_INT, NULL, -1, 0 },
- { F_DBFIFO_HP_INT, NULL, -1, 0 },
- { ERR_DROPPED_DB, "SGE doorbell dropped", -1, 0 },
+ { F_DBFIFO_LP_INT, NULL, -1, 0, t4_db_full },
+ { F_DBFIFO_HP_INT, NULL, -1, 0, t4_db_full },
+ { F_ERR_DROPPED_DB, NULL, -1, 0, t4_db_dropped },
{ ERR_DATA_CPL_ON_HIGH_QID1 | ERR_DATA_CPL_ON_HIGH_QID0,
"SGE IQID > 1023 received CPL for FL", -1, 0 },
{ ERR_BAD_DB_PIDX3, "SGE DBP 3 pidx increment too large", -1,
@@ -1036,20 +1041,14 @@ static void sge_intr_handler(struct adapter *adapter)
};
v = (u64)t4_read_reg(adapter, SGE_INT_CAUSE1) |
- ((u64)t4_read_reg(adapter, SGE_INT_CAUSE2) << 32);
+ ((u64)t4_read_reg(adapter, SGE_INT_CAUSE2) << 32);
if (v) {
dev_alert(adapter->pdev_dev, "SGE parity error (%#llx)\n",
- (unsigned long long)v);
+ (unsigned long long)v);
t4_write_reg(adapter, SGE_INT_CAUSE1, v);
t4_write_reg(adapter, SGE_INT_CAUSE2, v >> 32);
}
- err = t4_read_reg(adapter, A_SGE_INT_CAUSE3);
- if (err & (F_DBFIFO_HP_INT|F_DBFIFO_LP_INT))
- t4_db_full(adapter);
- if (err & F_ERR_DROPPED_DB)
- t4_db_dropped(adapter);
-
if (t4_handle_intr_status(adapter, SGE_INT_CAUSE3, sge_intr_info) ||
v != 0)
t4_fatal_err(adapter);
@@ -1995,6 +1994,54 @@ int t4_wol_pat_enable(struct adapter *adap, unsigned int port, unsigned int map,
(var).retval_len16 = htonl(FW_LEN16(var)); \
} while (0)
+int t4_fwaddrspace_write(struct adapter *adap, unsigned int mbox,
+ u32 addr, u32 val)
+{
+ struct fw_ldst_cmd c;
+
+ memset(&c, 0, sizeof(c));
+ c.op_to_addrspace = htonl(V_FW_CMD_OP(FW_LDST_CMD) | F_FW_CMD_REQUEST |
+ F_FW_CMD_WRITE |
+ V_FW_LDST_CMD_ADDRSPACE(FW_LDST_ADDRSPC_FIRMWARE));
+ c.cycles_to_len16 = htonl(FW_LEN16(c));
+ c.u.addrval.addr = htonl(addr);
+ c.u.addrval.val = htonl(val);
+
+ return t4_wr_mbox(adap, mbox, &c, sizeof(c), NULL);
+}
+
+/*
+ * t4_mem_win_read_len - read memory through PCIE memory window
+ * @adap: the adapter
+ * @addr: address of first byte requested aligned on 32b.
+ * @data: len bytes to hold the data read
+ * @len: amount of data to read from window. Must be <=
+ * MEMWIN0_APERATURE after adjusting for 16B alignment
+ * requirements of the the memory window.
+ *
+ * Read len bytes of data from MC starting at @addr.
+ */
+int t4_mem_win_read_len(struct adapter *adap, u32 addr, __be32 *data, int len)
+{
+ int i;
+ int off;
+
+ /*
+ * Align on a 16B boundary.
+ */
+ off = addr & 15;
+ if ((addr & 3) || (len + off) > MEMWIN0_APERTURE)
+ return -EINVAL;
+
+ t4_write_reg(adap, A_PCIE_MEM_ACCESS_OFFSET, addr & ~15);
+ t4_read_reg(adap, A_PCIE_MEM_ACCESS_OFFSET);
+
+ for (i = 0; i < len; i += 4)
+ *data++ = t4_read_reg(adap, (MEMWIN0_BASE + off + i));
+
+ return 0;
+}
+
/**
* t4_mdio_rd - read a PHY register through MDIO
* @adap: the adapter
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 03/10] cxgb4: DB Drop Recovery for RDMA and LLD queues.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: roland-BHEL68pLQRGGvPXPguhicg, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
divy-ut6Up61K2wZBDgjK7y7TUQ, dm-ut6Up61K2wZBDgjK7y7TUQ,
kumaras-ut6Up61K2wZBDgjK7y7TUQ,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
recover LLD EQs for DB drop interrupts. This includes adding a new
db_lock, a spin lock disabling BH too, used by the recovery thread and
the ring_tx_db() paths to allow db drop recovery.
cleaned up initial db avoidance code.
add read_eq_indices() - allows the LLD to use the pcie mw to efficiently
read hw eq contexts.
add cxgb4_sync_txq_pidx() - called by iw_cxgb4 to sync up the sw/hw pidx
value.
add flush_eq_cache() and cxgb4_flush_eq_cache(). This allows iw_cxgb4
to flush the sge eq context cache before beginning db drop recovery.
add module parameter, dbfoifo_int_thresh, to allow tuning the db
interrupt threshold value.
add dbfifo_int_thresh to cxgb4_lld_info so iw_cxgb4 knows the threshold.
add module parameter, dbfoifo_drain_delay, to allow tuning the amount
of time delay between DB FULL and EMPTY upcalls to iw_cxgb4.
Signed-off-by: Vipul Pandya <vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 16 ++
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 214 +++++++++++++++++++----
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 4 +
drivers/net/ethernet/chelsio/cxgb4/sge.c | 20 ++-
drivers/net/ethernet/chelsio/cxgb4/t4_regs.h | 53 ++++++
drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h | 15 ++
6 files changed, 280 insertions(+), 42 deletions(-)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
index 5f3c0a7..ec2dafe 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4.h
@@ -51,6 +51,8 @@
#define FW_VERSION_MINOR 1
#define FW_VERSION_MICRO 0
+#define CH_WARN(adap, fmt, ...) dev_warn(adap->pdev_dev, fmt, ## __VA_ARGS__)
+
enum {
MAX_NPORTS = 4, /* max # of ports */
SERNUM_LEN = 24, /* Serial # length */
@@ -64,6 +66,15 @@ enum {
MEM_MC
};
+enum {
+ MEMWIN0_APERTURE = 65536,
+ MEMWIN0_BASE = 0x30000,
+ MEMWIN1_APERTURE = 32768,
+ MEMWIN1_BASE = 0x28000,
+ MEMWIN2_APERTURE = 2048,
+ MEMWIN2_BASE = 0x1b800,
+};
+
enum dev_master {
MASTER_CANT,
MASTER_MAY,
@@ -403,6 +414,9 @@ struct sge_txq {
struct tx_sw_desc *sdesc; /* address of SW Tx descriptor ring */
struct sge_qstat *stat; /* queue status entry */
dma_addr_t phys_addr; /* physical address of the ring */
+ spinlock_t db_lock;
+ int db_disabled;
+ unsigned short db_pidx;
};
struct sge_eth_txq { /* state for an SGE Ethernet Tx queue */
@@ -475,6 +489,7 @@ struct adapter {
void __iomem *regs;
struct pci_dev *pdev;
struct device *pdev_dev;
+ unsigned int mbox;
unsigned int fn;
unsigned int flags;
@@ -607,6 +622,7 @@ irqreturn_t t4_sge_intr_msix(int irq, void *cookie);
void t4_sge_init(struct adapter *adap);
void t4_sge_start(struct adapter *adap);
void t4_sge_stop(struct adapter *adap);
+extern int dbfifo_int_thresh;
#define for_each_port(adapter, iter) \
for (iter = 0; iter < (adapter)->params.nports; ++iter)
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
index c243f93..e1f96fb 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c
@@ -149,15 +149,6 @@ static unsigned int pfvfres_pmask(struct adapter *adapter,
#endif
enum {
- MEMWIN0_APERTURE = 65536,
- MEMWIN0_BASE = 0x30000,
- MEMWIN1_APERTURE = 32768,
- MEMWIN1_BASE = 0x28000,
- MEMWIN2_APERTURE = 2048,
- MEMWIN2_BASE = 0x1b800,
-};
-
-enum {
MAX_TXQ_ENTRIES = 16384,
MAX_CTRL_TXQ_ENTRIES = 1024,
MAX_RSPQ_ENTRIES = 16384,
@@ -371,6 +362,15 @@ static int set_addr_filters(const struct net_device *dev, bool sleep)
uhash | mhash, sleep);
}
+int dbfifo_int_thresh = 10; /* 10 == 640 entry threshold */
+module_param(dbfifo_int_thresh, int, 0644);
+MODULE_PARM_DESC(dbfifo_int_thresh, "doorbell fifo interrupt threshold");
+
+int dbfifo_drain_delay = 1000; /* usecs to sleep while draining the dbfifo */
+module_param(dbfifo_drain_delay, int, 0644);
+MODULE_PARM_DESC(dbfifo_drain_delay,
+ "usecs to sleep while draining the dbfifo");
+
/*
* Set Rx properties of a port, such as promiscruity, address filters, and MTU.
* If @mtu is -1 it is left unchanged.
@@ -389,6 +389,8 @@ static int set_rxmode(struct net_device *dev, int mtu, bool sleep_ok)
return ret;
}
+static struct workqueue_struct *workq;
+
/**
* link_start - enable a port
* @dev: the port to enable
@@ -2196,7 +2198,7 @@ static void cxgb4_queue_tid_release(struct tid_info *t, unsigned int chan,
adap->tid_release_head = (void **)((uintptr_t)p | chan);
if (!adap->tid_release_task_busy) {
adap->tid_release_task_busy = true;
- schedule_work(&adap->tid_release_task);
+ queue_work(workq, &adap->tid_release_task);
}
spin_unlock_bh(&adap->tid_release_lock);
}
@@ -2423,6 +2425,59 @@ void cxgb4_iscsi_init(struct net_device *dev, unsigned int tag_mask,
}
EXPORT_SYMBOL(cxgb4_iscsi_init);
+int cxgb4_flush_eq_cache(struct net_device *dev)
+{
+ struct adapter *adap = netdev2adap(dev);
+ int ret;
+
+ ret = t4_fwaddrspace_write(adap, adap->mbox,
+ 0xe1000000 + A_SGE_CTXT_CMD, 0x20000000);
+ return ret;
+}
+EXPORT_SYMBOL(cxgb4_flush_eq_cache);
+
+static int read_eq_indices(struct adapter *adap, u16 qid, u16 *pidx, u16 *cidx)
+{
+ u32 addr = t4_read_reg(adap, A_SGE_DBQ_CTXT_BADDR) + 24 * qid + 8;
+ __be64 indices;
+ int ret;
+
+ ret = t4_mem_win_read_len(adap, addr, (__be32 *)&indices, 8);
+ if (!ret) {
+ indices = be64_to_cpu(indices);
+ *cidx = (indices >> 25) & 0xffff;
+ *pidx = (indices >> 9) & 0xffff;
+ }
+ return ret;
+}
+
+int cxgb4_sync_txq_pidx(struct net_device *dev, u16 qid, u16 pidx,
+ u16 size)
+{
+ struct adapter *adap = netdev2adap(dev);
+ u16 hw_pidx, hw_cidx;
+ int ret;
+
+ ret = read_eq_indices(adap, qid, &hw_pidx, &hw_cidx);
+ if (ret)
+ goto out;
+
+ if (pidx != hw_pidx) {
+ u16 delta;
+
+ if (pidx >= hw_pidx)
+ delta = pidx - hw_pidx;
+ else
+ delta = size - hw_pidx + pidx;
+ wmb();
+ t4_write_reg(adap, MYPF_REG(A_SGE_PF_KDOORBELL),
+ V_QID(qid) | V_PIDX(delta));
+ }
+out:
+ return ret;
+}
+EXPORT_SYMBOL(cxgb4_sync_txq_pidx);
+
static struct pci_driver cxgb4_driver;
static void check_neigh_update(struct neighbour *neigh)
@@ -2456,6 +2511,95 @@ static struct notifier_block cxgb4_netevent_nb = {
.notifier_call = netevent_cb
};
+static void drain_db_fifo(struct adapter *adap, int usecs)
+{
+ u32 v;
+
+ do {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(usecs));
+ v = t4_read_reg(adap, A_SGE_DBFIFO_STATUS);
+ if (G_LP_COUNT(v) == 0 && G_HP_COUNT(v) == 0)
+ break;
+ } while (1);
+}
+
+static void disable_txq_db(struct sge_txq *q)
+{
+ spin_lock_irq(&q->db_lock);
+ q->db_disabled = 1;
+ spin_unlock_irq(&q->db_lock);
+}
+
+static void enable_txq_db(struct sge_txq *q)
+{
+ spin_lock_irq(&q->db_lock);
+ q->db_disabled = 0;
+ spin_unlock_irq(&q->db_lock);
+}
+
+static void disable_dbs(struct adapter *adap)
+{
+ int i;
+
+ for_each_ethrxq(&adap->sge, i)
+ disable_txq_db(&adap->sge.ethtxq[i].q);
+ for_each_ofldrxq(&adap->sge, i)
+ disable_txq_db(&adap->sge.ofldtxq[i].q);
+ for_each_port(adap, i)
+ disable_txq_db(&adap->sge.ctrlq[i].q);
+}
+
+static void enable_dbs(struct adapter *adap)
+{
+ int i;
+
+ for_each_ethrxq(&adap->sge, i)
+ enable_txq_db(&adap->sge.ethtxq[i].q);
+ for_each_ofldrxq(&adap->sge, i)
+ enable_txq_db(&adap->sge.ofldtxq[i].q);
+ for_each_port(adap, i)
+ enable_txq_db(&adap->sge.ctrlq[i].q);
+}
+
+static void sync_txq_pidx(struct adapter *adap, struct sge_txq *q)
+{
+ u16 hw_pidx, hw_cidx;
+ int ret;
+
+ spin_lock_bh(&q->db_lock);
+ ret = read_eq_indices(adap, (u16)q->cntxt_id, &hw_pidx, &hw_cidx);
+ if (ret)
+ goto out;
+ if (q->db_pidx != hw_pidx) {
+ u16 delta;
+
+ if (q->db_pidx >= hw_pidx)
+ delta = q->db_pidx - hw_pidx;
+ else
+ delta = q->size - hw_pidx + q->db_pidx;
+ wmb();
+ t4_write_reg(adap, MYPF_REG(A_SGE_PF_KDOORBELL),
+ V_QID(q->cntxt_id) | V_PIDX(delta));
+ }
+out:
+ q->db_disabled = 0;
+ spin_unlock_bh(&q->db_lock);
+ if (ret)
+ CH_WARN(adap, "DB drop recovery failed.\n");
+}
+static void recover_all_queues(struct adapter *adap)
+{
+ int i;
+
+ for_each_ethrxq(&adap->sge, i)
+ sync_txq_pidx(adap, &adap->sge.ethtxq[i].q);
+ for_each_ofldrxq(&adap->sge, i)
+ sync_txq_pidx(adap, &adap->sge.ofldtxq[i].q);
+ for_each_port(adap, i)
+ sync_txq_pidx(adap, &adap->sge.ctrlq[i].q);
+}
+
static void notify_rdma_uld(struct adapter *adap, enum cxgb4_control cmd)
{
mutex_lock(&uld_mutex);
@@ -2468,55 +2612,41 @@ static void notify_rdma_uld(struct adapter *adap, enum cxgb4_control cmd)
static void process_db_full(struct work_struct *work)
{
struct adapter *adap;
- static int delay = 1000;
- u32 v;
adap = container_of(work, struct adapter, db_full_task);
-
- /* stop LLD queues */
-
notify_rdma_uld(adap, CXGB4_CONTROL_DB_FULL);
- do {
- set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(usecs_to_jiffies(delay));
- v = t4_read_reg(adap, A_SGE_DBFIFO_STATUS);
- if (G_LP_COUNT(v) == 0 && G_HP_COUNT(v) == 0)
- break;
- } while (1);
+ drain_db_fifo(adap, dbfifo_drain_delay);
+ t4_set_reg_field(adap, A_SGE_INT_ENABLE3,
+ F_DBFIFO_HP_INT | F_DBFIFO_LP_INT,
+ F_DBFIFO_HP_INT | F_DBFIFO_LP_INT);
notify_rdma_uld(adap, CXGB4_CONTROL_DB_EMPTY);
-
-
- /*
- * The more we get db full interrupts, the more we'll delay
- * in re-enabling db rings on queues, capped off at 200ms.
- */
- delay = min(delay << 1, 200000);
-
- /* resume LLD queues */
}
static void process_db_drop(struct work_struct *work)
{
struct adapter *adap;
- adap = container_of(work, struct adapter, db_drop_task);
+ adap = container_of(work, struct adapter, db_drop_task);
- /*
- * sync the PIDX values in HW and SW for LLD queues.
- */
-
+ t4_set_reg_field(adap, A_SGE_DOORBELL_CONTROL, F_DROPPED_DB, 0);
+ disable_dbs(adap);
notify_rdma_uld(adap, CXGB4_CONTROL_DB_DROP);
+ drain_db_fifo(adap, 1);
+ recover_all_queues(adap);
+ enable_dbs(adap);
}
void t4_db_full(struct adapter *adap)
{
- schedule_work(&adap->db_full_task);
+ t4_set_reg_field(adap, A_SGE_INT_ENABLE3,
+ F_DBFIFO_HP_INT | F_DBFIFO_LP_INT, 0);
+ queue_work(workq, &adap->db_full_task);
}
void t4_db_dropped(struct adapter *adap)
{
- schedule_work(&adap->db_drop_task);
+ queue_work(workq, &adap->db_drop_task);
}
static void uld_attach(struct adapter *adap, unsigned int uld)
@@ -2552,6 +2682,7 @@ static void uld_attach(struct adapter *adap, unsigned int uld)
lli.gts_reg = adap->regs + MYPF_REG(SGE_PF_GTS);
lli.db_reg = adap->regs + MYPF_REG(SGE_PF_KDOORBELL);
lli.fw_vers = adap->params.fw_vers;
+ lli.dbfifo_int_thresh = dbfifo_int_thresh;
handle = ulds[uld].add(&lli);
if (IS_ERR(handle)) {
@@ -3668,6 +3799,7 @@ static int __devinit init_one(struct pci_dev *pdev,
adapter->pdev = pdev;
adapter->pdev_dev = &pdev->dev;
+ adapter->mbox = func;
adapter->fn = func;
adapter->msg_enable = dflt_msg_enable;
memset(adapter->chan_map, 0xff, sizeof(adapter->chan_map));
@@ -3865,6 +3997,10 @@ static int __init cxgb4_init_module(void)
{
int ret;
+ workq = create_singlethread_workqueue("cxgb4");
+ if (!workq)
+ return -ENOMEM;
+
/* Debugfs support is optional, just warn if this fails */
cxgb4_debugfs_root = debugfs_create_dir(KBUILD_MODNAME, NULL);
if (!cxgb4_debugfs_root)
@@ -3880,6 +4016,8 @@ static void __exit cxgb4_cleanup_module(void)
{
pci_unregister_driver(&cxgb4_driver);
debugfs_remove(cxgb4_debugfs_root); /* NULL ok */
+ flush_workqueue(workq);
+ destroy_workqueue(workq);
}
module_init(cxgb4_init_module);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
index 5cc2f27..d79980c 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h
@@ -218,6 +218,7 @@ struct cxgb4_lld_info {
unsigned short ucq_density; /* # of user CQs/page */
void __iomem *gts_reg; /* address of GTS register */
void __iomem *db_reg; /* address of kernel doorbell */
+ int dbfifo_int_thresh; /* doorbell fifo int threshold */
};
struct cxgb4_uld_info {
@@ -226,6 +227,7 @@ struct cxgb4_uld_info {
int (*rx_handler)(void *handle, const __be64 *rsp,
const struct pkt_gl *gl);
int (*state_change)(void *handle, enum cxgb4_state new_state);
+ int (*control)(void *handle, enum cxgb4_control control, ...);
};
int cxgb4_register_uld(enum cxgb4_uld type, const struct cxgb4_uld_info *p);
@@ -243,4 +245,6 @@ void cxgb4_iscsi_init(struct net_device *dev, unsigned int tag_mask,
const unsigned int *pgsz_order);
struct sk_buff *cxgb4_pktgl_to_skb(const struct pkt_gl *gl,
unsigned int skb_len, unsigned int pull_len);
+int cxgb4_sync_txq_pidx(struct net_device *dev, u16 qid, u16 pidx, u16 size);
+int cxgb4_flush_eq_cache(struct net_device *dev);
#endif /* !__CXGB4_OFLD_H */
diff --git a/drivers/net/ethernet/chelsio/cxgb4/sge.c b/drivers/net/ethernet/chelsio/cxgb4/sge.c
index 234c157..e111d97 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/sge.c
+++ b/drivers/net/ethernet/chelsio/cxgb4/sge.c
@@ -767,8 +767,13 @@ static void write_sgl(const struct sk_buff *skb, struct sge_txq *q,
static inline void ring_tx_db(struct adapter *adap, struct sge_txq *q, int n)
{
wmb(); /* write descriptors before telling HW */
- t4_write_reg(adap, MYPF_REG(SGE_PF_KDOORBELL),
- QID(q->cntxt_id) | PIDX(n));
+ spin_lock(&q->db_lock);
+ if (!q->db_disabled) {
+ t4_write_reg(adap, MYPF_REG(A_SGE_PF_KDOORBELL),
+ V_QID(q->cntxt_id) | V_PIDX(n));
+ }
+ q->db_pidx = q->pidx;
+ spin_unlock(&q->db_lock);
}
/**
@@ -2081,6 +2086,7 @@ static void init_txq(struct adapter *adap, struct sge_txq *q, unsigned int id)
q->stops = q->restarts = 0;
q->stat = (void *)&q->desc[q->size];
q->cntxt_id = id;
+ spin_lock_init(&q->db_lock);
adap->sge.egr_map[id - adap->sge.egr_start] = q;
}
@@ -2415,9 +2421,15 @@ void t4_sge_init(struct adapter *adap)
RXPKTCPLMODE |
(STAT_LEN == 128 ? EGRSTATUSPAGESIZE : 0));
+ /*
+ * Set up to drop DOORBELL writes when the DOORBELL FIFO overflows
+ * and generate an interrupt when this occurs so we can recover.
+ */
t4_set_reg_field(adap, A_SGE_DBFIFO_STATUS,
- V_HP_INT_THRESH(5) | V_LP_INT_THRESH(5),
- V_HP_INT_THRESH(5) | V_LP_INT_THRESH(5));
+ V_HP_INT_THRESH(M_HP_INT_THRESH) |
+ V_LP_INT_THRESH(M_LP_INT_THRESH),
+ V_HP_INT_THRESH(dbfifo_int_thresh) |
+ V_LP_INT_THRESH(dbfifo_int_thresh));
t4_set_reg_field(adap, A_SGE_DOORBELL_CONTROL, F_ENABLE_DROP,
F_ENABLE_DROP);
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4_regs.h b/drivers/net/ethernet/chelsio/cxgb4/t4_regs.h
index 0adc5bc..111fc32 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4_regs.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4_regs.h
@@ -190,6 +190,59 @@
#define SGE_DEBUG_DATA_LOW 0x10d4
#define SGE_INGRESS_QUEUES_PER_PAGE_PF 0x10f4
+#define S_LP_INT_THRESH 12
+#define V_LP_INT_THRESH(x) ((x) << S_LP_INT_THRESH)
+#define S_HP_INT_THRESH 28
+#define V_HP_INT_THRESH(x) ((x) << S_HP_INT_THRESH)
+#define A_SGE_DBFIFO_STATUS 0x10a4
+
+#define S_ENABLE_DROP 13
+#define V_ENABLE_DROP(x) ((x) << S_ENABLE_DROP)
+#define F_ENABLE_DROP V_ENABLE_DROP(1U)
+#define A_SGE_DOORBELL_CONTROL 0x10a8
+
+#define A_SGE_CTXT_CMD 0x11fc
+#define A_SGE_DBQ_CTXT_BADDR 0x1084
+
+#define A_SGE_PF_KDOORBELL 0x0
+
+#define S_QID 15
+#define V_QID(x) ((x) << S_QID)
+
+#define S_PIDX 0
+#define V_PIDX(x) ((x) << S_PIDX)
+
+#define M_LP_COUNT 0x7ffU
+#define S_LP_COUNT 0
+#define G_LP_COUNT(x) (((x) >> S_LP_COUNT) & M_LP_COUNT)
+
+#define M_HP_COUNT 0x7ffU
+#define S_HP_COUNT 16
+#define G_HP_COUNT(x) (((x) >> S_HP_COUNT) & M_HP_COUNT)
+
+#define A_SGE_INT_ENABLE3 0x1040
+
+#define S_DBFIFO_HP_INT 8
+#define V_DBFIFO_HP_INT(x) ((x) << S_DBFIFO_HP_INT)
+#define F_DBFIFO_HP_INT V_DBFIFO_HP_INT(1U)
+
+#define S_DBFIFO_LP_INT 7
+#define V_DBFIFO_LP_INT(x) ((x) << S_DBFIFO_LP_INT)
+#define F_DBFIFO_LP_INT V_DBFIFO_LP_INT(1U)
+
+#define S_DROPPED_DB 0
+#define V_DROPPED_DB(x) ((x) << S_DROPPED_DB)
+#define F_DROPPED_DB V_DROPPED_DB(1U)
+
+#define S_ERR_DROPPED_DB 18
+#define V_ERR_DROPPED_DB(x) ((x) << S_ERR_DROPPED_DB)
+#define F_ERR_DROPPED_DB V_ERR_DROPPED_DB(1U)
+
+#define A_PCIE_MEM_ACCESS_OFFSET 0x306c
+
+#define M_HP_INT_THRESH 0xfU
+#define M_LP_INT_THRESH 0xfU
+
#define PCIE_PF_CLI 0x44
#define PCIE_INT_CAUSE 0x3004
#define UNXSPLCPLERR 0x20000000U
diff --git a/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h b/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
index edcfd7e..ad53f79 100644
--- a/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
+++ b/drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h
@@ -1620,4 +1620,19 @@ struct fw_hdr {
#define FW_HDR_FW_VER_MINOR_GET(x) (((x) >> 16) & 0xff)
#define FW_HDR_FW_VER_MICRO_GET(x) (((x) >> 8) & 0xff)
#define FW_HDR_FW_VER_BUILD_GET(x) (((x) >> 0) & 0xff)
+
+#define S_FW_CMD_OP 24
+#define V_FW_CMD_OP(x) ((x) << S_FW_CMD_OP)
+
+#define S_FW_CMD_REQUEST 23
+#define V_FW_CMD_REQUEST(x) ((x) << S_FW_CMD_REQUEST)
+#define F_FW_CMD_REQUEST V_FW_CMD_REQUEST(1U)
+
+#define S_FW_CMD_WRITE 21
+#define V_FW_CMD_WRITE(x) ((x) << S_FW_CMD_WRITE)
+#define F_FW_CMD_WRITE V_FW_CMD_WRITE(1U)
+
+#define S_FW_LDST_CMD_ADDRSPACE 0
+#define V_FW_LDST_CMD_ADDRSPACE(x) ((x) << S_FW_LDST_CMD_ADDRSPACE)
+
#endif /* _T4FW_INTERFACE_H_ */
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 06/10] RDMA/cxgb4: disable interrupts in c4iw_ev_dispatch().
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: roland-BHEL68pLQRGGvPXPguhicg, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
divy-ut6Up61K2wZBDgjK7y7TUQ, dm-ut6Up61K2wZBDgjK7y7TUQ,
kumaras-ut6Up61K2wZBDgjK7y7TUQ,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Use GFP_ATOMIC in _insert_handle() if ints are disabled.
Don't panic if we get an abort with no endpoint found. Just log a
warning.
Signed-off-by: Vipul Pandya <vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/hw/cxgb4/cm.c | 5 ++++-
drivers/infiniband/hw/cxgb4/ev.c | 8 ++++----
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 2 +-
3 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index 4c7c62f..6ce401a 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -1362,7 +1362,10 @@ static int abort_rpl(struct c4iw_dev *dev, struct sk_buff *skb)
ep = lookup_tid(t, tid);
PDBG("%s ep %p tid %u\n", __func__, ep, ep->hwtid);
- BUG_ON(!ep);
+ if (!ep) {
+ printk(KERN_WARNING MOD "Abort rpl to freed endpoint\n");
+ return 0;
+ }
mutex_lock(&ep->com.mutex);
switch (ep->com.state) {
case ABORTING:
diff --git a/drivers/infiniband/hw/cxgb4/ev.c b/drivers/infiniband/hw/cxgb4/ev.c
index 397cb36..cf2f6b4 100644
--- a/drivers/infiniband/hw/cxgb4/ev.c
+++ b/drivers/infiniband/hw/cxgb4/ev.c
@@ -84,7 +84,7 @@ void c4iw_ev_dispatch(struct c4iw_dev *dev, struct t4_cqe *err_cqe)
struct c4iw_qp *qhp;
u32 cqid;
- spin_lock(&dev->lock);
+ spin_lock_irq(&dev->lock);
qhp = get_qhp(dev, CQE_QPID(err_cqe));
if (!qhp) {
printk(KERN_ERR MOD "BAD AE qpid 0x%x opcode %d "
@@ -93,7 +93,7 @@ void c4iw_ev_dispatch(struct c4iw_dev *dev, struct t4_cqe *err_cqe)
CQE_OPCODE(err_cqe), CQE_STATUS(err_cqe),
CQE_TYPE(err_cqe), CQE_WRID_HI(err_cqe),
CQE_WRID_LOW(err_cqe));
- spin_unlock(&dev->lock);
+ spin_unlock_irq(&dev->lock);
goto out;
}
@@ -109,13 +109,13 @@ void c4iw_ev_dispatch(struct c4iw_dev *dev, struct t4_cqe *err_cqe)
CQE_OPCODE(err_cqe), CQE_STATUS(err_cqe),
CQE_TYPE(err_cqe), CQE_WRID_HI(err_cqe),
CQE_WRID_LOW(err_cqe));
- spin_unlock(&dev->lock);
+ spin_unlock_irq(&dev->lock);
goto out;
}
c4iw_qp_add_ref(&qhp->ibqp);
atomic_inc(&chp->refcnt);
- spin_unlock(&dev->lock);
+ spin_unlock_irq(&dev->lock);
/* Bad incoming write */
if (RQ_TYPE(err_cqe) &&
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index a11ed5c..e8b88a0 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -246,7 +246,7 @@ static inline int _insert_handle(struct c4iw_dev *rhp, struct idr *idr,
int newid;
do {
- if (!idr_pre_get(idr, GFP_KERNEL))
+ if (!idr_pre_get(idr, lock ? GFP_KERNEL : GFP_ATOMIC))
return -ENOMEM;
if (lock)
spin_lock_irq(&rhp->lock);
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* [PATCH 09/10] RDMA/cxgb4: remove kfifo usage
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma-u79uwXL29TY76Z2rM5mHXA, netdev-u79uwXL29TY76Z2rM5mHXA
Cc: roland-BHEL68pLQRGGvPXPguhicg, davem-fT/PcQaiUtIeIZ0/mPfg9Q,
divy-ut6Up61K2wZBDgjK7y7TUQ, dm-ut6Up61K2wZBDgjK7y7TUQ,
kumaras-ut6Up61K2wZBDgjK7y7TUQ,
swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Using kfifos for ID management was limiting the number of QPs and
preventing NP384 MPI jobs. So replace it with a simple bitmap
allocator.
Remove IDs from the IDR tables before deallocating them. This bug was
causing the BUG_ON() in insert_handle() to fire because the ID was
getting reused before being removed from the IDR table.
Signed-off-by: Vipul Pandya <vipul-ut6Up61K2wZBDgjK7y7TUQ@public.gmane.org>
Signed-off-by: Steve Wise <swise-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org>
---
drivers/infiniband/hw/cxgb4/Makefile | 2 +-
drivers/infiniband/hw/cxgb4/device.c | 37 +++++---
drivers/infiniband/hw/cxgb4/id_table.c | 112 ++++++++++++++++++++++++
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 35 ++++++--
drivers/infiniband/hw/cxgb4/mem.c | 10 +--
drivers/infiniband/hw/cxgb4/provider.c | 9 +--
drivers/infiniband/hw/cxgb4/resource.c | 148 ++++++++------------------------
7 files changed, 203 insertions(+), 150 deletions(-)
create mode 100644 drivers/infiniband/hw/cxgb4/id_table.c
diff --git a/drivers/infiniband/hw/cxgb4/Makefile b/drivers/infiniband/hw/cxgb4/Makefile
index 46b878c..e11cf72 100644
--- a/drivers/infiniband/hw/cxgb4/Makefile
+++ b/drivers/infiniband/hw/cxgb4/Makefile
@@ -2,4 +2,4 @@ ccflags-y := -Idrivers/net/ethernet/chelsio/cxgb4
obj-$(CONFIG_INFINIBAND_CXGB4) += iw_cxgb4.o
-iw_cxgb4-y := device.o cm.o provider.o mem.o cq.o qp.o resource.o ev.o
+iw_cxgb4-y := device.o cm.o provider.o mem.o cq.o qp.o resource.o ev.o id_table.o
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index 8545629..c8fd1d8 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -252,25 +252,26 @@ static int stats_show(struct seq_file *seq, void *v)
{
struct c4iw_dev *dev = seq->private;
- seq_printf(seq, " Object: %10s %10s %10s\n", "Total", "Current", "Max");
- seq_printf(seq, " PDID: %10llu %10llu %10llu\n",
+ seq_printf(seq, " Object: %10s %10s %10s %10s\n", "Total", "Current",
+ "Max", "Fail");
+ seq_printf(seq, " PDID: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.pd.total, dev->rdev.stats.pd.cur,
- dev->rdev.stats.pd.max);
- seq_printf(seq, " QID: %10llu %10llu %10llu\n",
+ dev->rdev.stats.pd.max, dev->rdev.stats.pd.fail);
+ seq_printf(seq, " QID: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.qid.total, dev->rdev.stats.qid.cur,
- dev->rdev.stats.qid.max);
- seq_printf(seq, " TPTMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.qid.max, dev->rdev.stats.qid.fail);
+ seq_printf(seq, " TPTMEM: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.stag.total, dev->rdev.stats.stag.cur,
- dev->rdev.stats.stag.max);
- seq_printf(seq, " PBLMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.stag.max, dev->rdev.stats.stag.fail);
+ seq_printf(seq, " PBLMEM: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.pbl.total, dev->rdev.stats.pbl.cur,
- dev->rdev.stats.pbl.max);
- seq_printf(seq, " RQTMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.pbl.max, dev->rdev.stats.pbl.fail);
+ seq_printf(seq, " RQTMEM: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.rqt.total, dev->rdev.stats.rqt.cur,
- dev->rdev.stats.rqt.max);
- seq_printf(seq, " OCQPMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.rqt.max, dev->rdev.stats.rqt.fail);
+ seq_printf(seq, " OCQPMEM: %10llu %10llu %10llu %10llu\n",
dev->rdev.stats.ocqp.total, dev->rdev.stats.ocqp.cur,
- dev->rdev.stats.ocqp.max);
+ dev->rdev.stats.ocqp.max, dev->rdev.stats.ocqp.fail);
seq_printf(seq, " DB FULL: %10llu\n", dev->rdev.stats.db_full);
seq_printf(seq, " DB EMPTY: %10llu\n", dev->rdev.stats.db_empty);
seq_printf(seq, " DB DROP: %10llu\n", dev->rdev.stats.db_drop);
@@ -292,11 +293,17 @@ static ssize_t stats_clear(struct file *file, const char __user *buf,
mutex_lock(&dev->rdev.stats.lock);
dev->rdev.stats.pd.max = 0;
+ dev->rdev.stats.pd.fail = 0;
dev->rdev.stats.qid.max = 0;
+ dev->rdev.stats.qid.fail = 0;
dev->rdev.stats.stag.max = 0;
+ dev->rdev.stats.stag.fail = 0;
dev->rdev.stats.pbl.max = 0;
+ dev->rdev.stats.pbl.fail = 0;
dev->rdev.stats.rqt.max = 0;
+ dev->rdev.stats.rqt.fail = 0;
dev->rdev.stats.ocqp.max = 0;
+ dev->rdev.stats.ocqp.fail = 0;
dev->rdev.stats.db_full = 0;
dev->rdev.stats.db_empty = 0;
dev->rdev.stats.db_drop = 0;
@@ -350,8 +357,8 @@ void c4iw_release_dev_ucontext(struct c4iw_rdev *rdev,
entry = list_entry(pos, struct c4iw_qid_list, entry);
list_del_init(&entry->entry);
if (!(entry->qid & rdev->qpmask)) {
- c4iw_put_resource(&rdev->resource.qid_fifo, entry->qid,
- &rdev->resource.qid_fifo_lock);
+ c4iw_put_resource(&rdev->resource.qid_table,
+ entry->qid);
mutex_lock(&rdev->stats.lock);
rdev->stats.qid.cur -= rdev->qpmask + 1;
mutex_unlock(&rdev->stats.lock);
diff --git a/drivers/infiniband/hw/cxgb4/id_table.c b/drivers/infiniband/hw/cxgb4/id_table.c
new file mode 100644
index 0000000..f95e5df
--- /dev/null
+++ b/drivers/infiniband/hw/cxgb4/id_table.c
@@ -0,0 +1,112 @@
+/*
+ * Copyright (c) 2011 Chelsio Communications. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ * copyright notice, this list of conditions and the following
+ * disclaimer in the documentation and/or other materials
+ * provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+#include <linux/kernel.h>
+#include <linux/random.h>
+#include "iw_cxgb4.h"
+
+#define RANDOM_SKIP 16
+
+/*
+ * Trivial bitmap-based allocator. If the random flag is set, the
+ * allocator is designed to:
+ * - pseudo-randomize the id returned such that it is not trivially predictable.
+ * - avoid reuse of recently used id (at the expense of predictability)
+ */
+u32 c4iw_id_alloc(struct c4iw_id_table *alloc)
+{
+ unsigned long flags;
+ u32 obj;
+
+ spin_lock_irqsave(&alloc->lock, flags);
+
+ obj = find_next_zero_bit(alloc->table, alloc->max, alloc->last);
+ if (obj >= alloc->max)
+ obj = find_first_zero_bit(alloc->table, alloc->max);
+
+ if (obj < alloc->max) {
+ if (alloc->flags & C4IW_ID_TABLE_F_RANDOM)
+ alloc->last += random32() % RANDOM_SKIP;
+ else
+ alloc->last = obj + 1;
+ if (alloc->last >= alloc->max)
+ alloc->last = 0;
+ set_bit(obj, alloc->table);
+ obj += alloc->start;
+ } else
+ obj = -1;
+
+ spin_unlock_irqrestore(&alloc->lock, flags);
+ return obj;
+}
+
+void c4iw_id_free(struct c4iw_id_table *alloc, u32 obj)
+{
+ unsigned long flags;
+
+ obj -= alloc->start;
+ BUG_ON((int)obj < 0);
+
+ spin_lock_irqsave(&alloc->lock, flags);
+ clear_bit(obj, alloc->table);
+ spin_unlock_irqrestore(&alloc->lock, flags);
+}
+
+int c4iw_id_table_alloc(struct c4iw_id_table *alloc, u32 start, u32 num,
+ u32 reserved, u32 flags)
+{
+ int i;
+
+ alloc->start = start;
+ alloc->flags = flags;
+ if (flags & C4IW_ID_TABLE_F_RANDOM)
+ alloc->last = random32() % RANDOM_SKIP;
+ else
+ alloc->last = 0;
+ alloc->max = num;
+ spin_lock_init(&alloc->lock);
+ alloc->table = kmalloc(BITS_TO_LONGS(num) * sizeof(long),
+ GFP_KERNEL);
+ if (!alloc->table)
+ return -ENOMEM;
+
+ bitmap_zero(alloc->table, num);
+ if (!(alloc->flags & C4IW_ID_TABLE_F_EMPTY))
+ for (i = 0; i < reserved; ++i)
+ set_bit(i, alloc->table);
+
+ return 0;
+}
+
+void c4iw_id_table_free(struct c4iw_id_table *alloc)
+{
+ kfree(alloc->table);
+}
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index 6818659..2d5b06b 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -45,7 +45,6 @@
#include <linux/kref.h>
#include <linux/timer.h>
#include <linux/io.h>
-#include <linux/kfifo.h>
#include <asm/byteorder.h>
@@ -79,13 +78,22 @@ static inline void *cplhdr(struct sk_buff *skb)
return skb->data;
}
+#define C4IW_ID_TABLE_F_RANDOM 1 /* Pseudo-randomize the id's returned */
+#define C4IW_ID_TABLE_F_EMPTY 2 /* Table is initially empty */
+
+struct c4iw_id_table {
+ u32 flags;
+ u32 start; /* logical minimal id */
+ u32 last; /* hint for find */
+ u32 max;
+ spinlock_t lock;
+ unsigned long *table;
+};
+
struct c4iw_resource {
- struct kfifo tpt_fifo;
- spinlock_t tpt_fifo_lock;
- struct kfifo qid_fifo;
- spinlock_t qid_fifo_lock;
- struct kfifo pdid_fifo;
- spinlock_t pdid_fifo_lock;
+ struct c4iw_id_table tpt_table;
+ struct c4iw_id_table qid_table;
+ struct c4iw_id_table pdid_table;
};
struct c4iw_qid_list {
@@ -107,6 +115,7 @@ struct c4iw_stat {
u64 total;
u64 cur;
u64 max;
+ u64 fail;
};
struct c4iw_stats {
@@ -253,7 +262,7 @@ static inline int _insert_handle(struct c4iw_dev *rhp, struct idr *idr,
if (lock)
spin_lock_irq(&rhp->lock);
ret = idr_get_new_above(idr, handle, id, &newid);
- BUG_ON(newid != id);
+ BUG_ON(!ret && newid != id);
if (lock)
spin_unlock_irq(&rhp->lock);
} while (ret == -EAGAIN);
@@ -755,14 +764,20 @@ static inline int compute_wscale(int win)
return wscale;
}
+u32 c4iw_id_alloc(struct c4iw_id_table *alloc);
+void c4iw_id_free(struct c4iw_id_table *alloc, u32 obj);
+int c4iw_id_table_alloc(struct c4iw_id_table *alloc, u32 start, u32 num,
+ u32 reserved, u32 flags);
+void c4iw_id_table_free(struct c4iw_id_table *alloc);
+
typedef int (*c4iw_handler_func)(struct c4iw_dev *dev, struct sk_buff *skb);
int c4iw_ep_redirect(void *ctx, struct dst_entry *old, struct dst_entry *new,
struct l2t_entry *l2t);
void c4iw_put_qpid(struct c4iw_rdev *rdev, u32 qpid,
struct c4iw_dev_ucontext *uctx);
-u32 c4iw_get_resource(struct kfifo *fifo, spinlock_t *lock);
-void c4iw_put_resource(struct kfifo *fifo, u32 entry, spinlock_t *lock);
+u32 c4iw_get_resource(struct c4iw_id_table *id_table);
+void c4iw_put_resource(struct c4iw_id_table *id_table, u32 entry);
int c4iw_init_resource(struct c4iw_rdev *rdev, u32 nr_tpt, u32 nr_pdid);
int c4iw_init_ctrl_qp(struct c4iw_rdev *rdev);
int c4iw_pblpool_create(struct c4iw_rdev *rdev);
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 2a87379..57e07c6 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -131,8 +131,7 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
stag_idx = (*stag) >> 8;
if ((!reset_tpt_entry) && (*stag == T4_STAG_UNSET)) {
- stag_idx = c4iw_get_resource(&rdev->resource.tpt_fifo,
- &rdev->resource.tpt_fifo_lock);
+ stag_idx = c4iw_get_resource(&rdev->resource.tpt_table);
if (!stag_idx)
return -ENOMEM;
mutex_lock(&rdev->stats.lock);
@@ -171,8 +170,7 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
sizeof(tpt), &tpt);
if (reset_tpt_entry) {
- c4iw_put_resource(&rdev->resource.tpt_fifo, stag_idx,
- &rdev->resource.tpt_fifo_lock);
+ c4iw_put_resource(&rdev->resource.tpt_table, stag_idx);
mutex_lock(&rdev->stats.lock);
rdev->stats.stag.cur -= 32;
mutex_unlock(&rdev->stats.lock);
@@ -695,8 +693,8 @@ int c4iw_dealloc_mw(struct ib_mw *mw)
mhp = to_c4iw_mw(mw);
rhp = mhp->rhp;
mmid = (mw->rkey) >> 8;
- deallocate_window(&rhp->rdev, mhp->attr.stag);
remove_handle(rhp, &rhp->mmidr, mmid);
+ deallocate_window(&rhp->rdev, mhp->attr.stag);
kfree(mhp);
PDBG("%s ib_mw %p mmid 0x%x ptr %p\n", __func__, mw, mmid, mhp);
return 0;
@@ -798,12 +796,12 @@ int c4iw_dereg_mr(struct ib_mr *ib_mr)
mhp = to_c4iw_mr(ib_mr);
rhp = mhp->rhp;
mmid = mhp->attr.stag >> 8;
+ remove_handle(rhp, &rhp->mmidr, mmid);
dereg_mem(&rhp->rdev, mhp->attr.stag, mhp->attr.pbl_size,
mhp->attr.pbl_addr);
if (mhp->attr.pbl_size)
c4iw_pblpool_free(&mhp->rhp->rdev, mhp->attr.pbl_addr,
mhp->attr.pbl_size << 3);
- remove_handle(rhp, &rhp->mmidr, mmid);
if (mhp->kva)
kfree((void *) (unsigned long) mhp->kva);
if (mhp->umem)
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index 8d58736..fe98a0a 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -188,8 +188,7 @@ static int c4iw_deallocate_pd(struct ib_pd *pd)
php = to_c4iw_pd(pd);
rhp = php->rhp;
PDBG("%s ibpd %p pdid 0x%x\n", __func__, pd, php->pdid);
- c4iw_put_resource(&rhp->rdev.resource.pdid_fifo, php->pdid,
- &rhp->rdev.resource.pdid_fifo_lock);
+ c4iw_put_resource(&rhp->rdev.resource.pdid_table, php->pdid);
mutex_lock(&rhp->rdev.stats.lock);
rhp->rdev.stats.pd.cur--;
mutex_unlock(&rhp->rdev.stats.lock);
@@ -207,14 +206,12 @@ static struct ib_pd *c4iw_allocate_pd(struct ib_device *ibdev,
PDBG("%s ibdev %p\n", __func__, ibdev);
rhp = (struct c4iw_dev *) ibdev;
- pdid = c4iw_get_resource(&rhp->rdev.resource.pdid_fifo,
- &rhp->rdev.resource.pdid_fifo_lock);
+ pdid = c4iw_get_resource(&rhp->rdev.resource.pdid_table);
if (!pdid)
return ERR_PTR(-EINVAL);
php = kzalloc(sizeof(*php), GFP_KERNEL);
if (!php) {
- c4iw_put_resource(&rhp->rdev.resource.pdid_fifo, pdid,
- &rhp->rdev.resource.pdid_fifo_lock);
+ c4iw_put_resource(&rhp->rdev.resource.pdid_table, pdid);
return ERR_PTR(-ENOMEM);
}
php->pdid = pdid;
diff --git a/drivers/infiniband/hw/cxgb4/resource.c b/drivers/infiniband/hw/cxgb4/resource.c
index 1b948d1..cdef4d7 100644
--- a/drivers/infiniband/hw/cxgb4/resource.c
+++ b/drivers/infiniband/hw/cxgb4/resource.c
@@ -30,96 +30,25 @@
* SOFTWARE.
*/
/* Crude resource management */
-#include <linux/kernel.h>
-#include <linux/random.h>
-#include <linux/slab.h>
-#include <linux/kfifo.h>
#include <linux/spinlock.h>
-#include <linux/errno.h>
#include <linux/genalloc.h>
#include <linux/ratelimit.h>
#include "iw_cxgb4.h"
-#define RANDOM_SIZE 16
-
-static int __c4iw_init_resource_fifo(struct kfifo *fifo,
- spinlock_t *fifo_lock,
- u32 nr, u32 skip_low,
- u32 skip_high,
- int random)
-{
- u32 i, j, entry = 0, idx;
- u32 random_bytes;
- u32 rarray[16];
- spin_lock_init(fifo_lock);
-
- if (kfifo_alloc(fifo, nr * sizeof(u32), GFP_KERNEL))
- return -ENOMEM;
-
- for (i = 0; i < skip_low + skip_high; i++)
- kfifo_in(fifo, (unsigned char *) &entry, sizeof(u32));
- if (random) {
- j = 0;
- random_bytes = random32();
- for (i = 0; i < RANDOM_SIZE; i++)
- rarray[i] = i + skip_low;
- for (i = skip_low + RANDOM_SIZE; i < nr - skip_high; i++) {
- if (j >= RANDOM_SIZE) {
- j = 0;
- random_bytes = random32();
- }
- idx = (random_bytes >> (j * 2)) & 0xF;
- kfifo_in(fifo,
- (unsigned char *) &rarray[idx],
- sizeof(u32));
- rarray[idx] = i;
- j++;
- }
- for (i = 0; i < RANDOM_SIZE; i++)
- kfifo_in(fifo,
- (unsigned char *) &rarray[i],
- sizeof(u32));
- } else
- for (i = skip_low; i < nr - skip_high; i++)
- kfifo_in(fifo, (unsigned char *) &i, sizeof(u32));
-
- for (i = 0; i < skip_low + skip_high; i++)
- if (kfifo_out_locked(fifo, (unsigned char *) &entry,
- sizeof(u32), fifo_lock))
- break;
- return 0;
-}
-
-static int c4iw_init_resource_fifo(struct kfifo *fifo, spinlock_t * fifo_lock,
- u32 nr, u32 skip_low, u32 skip_high)
-{
- return __c4iw_init_resource_fifo(fifo, fifo_lock, nr, skip_low,
- skip_high, 0);
-}
-
-static int c4iw_init_resource_fifo_random(struct kfifo *fifo,
- spinlock_t *fifo_lock,
- u32 nr, u32 skip_low, u32 skip_high)
-{
- return __c4iw_init_resource_fifo(fifo, fifo_lock, nr, skip_low,
- skip_high, 1);
-}
-
-static int c4iw_init_qid_fifo(struct c4iw_rdev *rdev)
+static int c4iw_init_qid_table(struct c4iw_rdev *rdev)
{
u32 i;
- spin_lock_init(&rdev->resource.qid_fifo_lock);
-
- if (kfifo_alloc(&rdev->resource.qid_fifo, rdev->lldi.vr->qp.size *
- sizeof(u32), GFP_KERNEL))
+ if (c4iw_id_table_alloc(&rdev->resource.qid_table,
+ rdev->lldi.vr->qp.start,
+ rdev->lldi.vr->qp.size,
+ rdev->lldi.vr->qp.size, 0))
return -ENOMEM;
for (i = rdev->lldi.vr->qp.start;
- i < rdev->lldi.vr->qp.start + rdev->lldi.vr->qp.size; i++)
+ i < rdev->lldi.vr->qp.start + rdev->lldi.vr->qp.size; i++)
if (!(i & rdev->qpmask))
- kfifo_in(&rdev->resource.qid_fifo,
- (unsigned char *) &i, sizeof(u32));
+ c4iw_id_free(&rdev->resource.qid_table, i);
return 0;
}
@@ -127,44 +56,42 @@ static int c4iw_init_qid_fifo(struct c4iw_rdev *rdev)
int c4iw_init_resource(struct c4iw_rdev *rdev, u32 nr_tpt, u32 nr_pdid)
{
int err = 0;
- err = c4iw_init_resource_fifo_random(&rdev->resource.tpt_fifo,
- &rdev->resource.tpt_fifo_lock,
- nr_tpt, 1, 0);
+ err = c4iw_id_table_alloc(&rdev->resource.tpt_table, 0, nr_tpt, 1,
+ C4IW_ID_TABLE_F_RANDOM);
if (err)
goto tpt_err;
- err = c4iw_init_qid_fifo(rdev);
+ err = c4iw_init_qid_table(rdev);
if (err)
goto qid_err;
- err = c4iw_init_resource_fifo(&rdev->resource.pdid_fifo,
- &rdev->resource.pdid_fifo_lock,
- nr_pdid, 1, 0);
+ err = c4iw_id_table_alloc(&rdev->resource.pdid_table, 0,
+ nr_pdid, 1, 0);
if (err)
goto pdid_err;
return 0;
-pdid_err:
- kfifo_free(&rdev->resource.qid_fifo);
-qid_err:
- kfifo_free(&rdev->resource.tpt_fifo);
-tpt_err:
+ pdid_err:
+ c4iw_id_table_free(&rdev->resource.qid_table);
+ qid_err:
+ c4iw_id_table_free(&rdev->resource.tpt_table);
+ tpt_err:
return -ENOMEM;
}
/*
* returns 0 if no resource available
*/
-u32 c4iw_get_resource(struct kfifo *fifo, spinlock_t *lock)
+u32 c4iw_get_resource(struct c4iw_id_table *id_table)
{
u32 entry;
- if (kfifo_out_locked(fifo, (unsigned char *) &entry, sizeof(u32), lock))
- return entry;
- else
+ entry = c4iw_id_alloc(id_table);
+ if (entry == (u32)(-1))
return 0;
+ return entry;
}
-void c4iw_put_resource(struct kfifo *fifo, u32 entry, spinlock_t *lock)
+void c4iw_put_resource(struct c4iw_id_table *id_table, u32 entry)
{
PDBG("%s entry 0x%x\n", __func__, entry);
- kfifo_in_locked(fifo, (unsigned char *) &entry, sizeof(u32), lock);
+ c4iw_id_free(id_table, entry);
}
u32 c4iw_get_cqid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
@@ -181,8 +108,7 @@ u32 c4iw_get_cqid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
qid = entry->qid;
kfree(entry);
} else {
- qid = c4iw_get_resource(&rdev->resource.qid_fifo,
- &rdev->resource.qid_fifo_lock);
+ qid = c4iw_get_resource(&rdev->resource.qid_table);
if (!qid)
goto out;
mutex_lock(&rdev->stats.lock);
@@ -252,8 +178,7 @@ u32 c4iw_get_qpid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
qid = entry->qid;
kfree(entry);
} else {
- qid = c4iw_get_resource(&rdev->resource.qid_fifo,
- &rdev->resource.qid_fifo_lock);
+ qid = c4iw_get_resource(&rdev->resource.qid_table);
if (!qid)
goto out;
mutex_lock(&rdev->stats.lock);
@@ -311,9 +236,9 @@ void c4iw_put_qpid(struct c4iw_rdev *rdev, u32 qid,
void c4iw_destroy_resource(struct c4iw_resource *rscp)
{
- kfifo_free(&rscp->tpt_fifo);
- kfifo_free(&rscp->qid_fifo);
- kfifo_free(&rscp->pdid_fifo);
+ c4iw_id_table_free(&rscp->tpt_table);
+ c4iw_id_table_free(&rscp->qid_table);
+ c4iw_id_table_free(&rscp->pdid_table);
}
/*
@@ -326,16 +251,14 @@ u32 c4iw_pblpool_alloc(struct c4iw_rdev *rdev, int size)
{
unsigned long addr = gen_pool_alloc(rdev->pbl_pool, size);
PDBG("%s addr 0x%x size %d\n", __func__, (u32)addr, size);
- if (!addr)
- printk_ratelimited(KERN_WARNING MOD "%s: Out of PBL memory\n",
- pci_name(rdev->lldi.pdev));
+ mutex_lock(&rdev->stats.lock);
if (addr) {
- mutex_lock(&rdev->stats.lock);
rdev->stats.pbl.cur += roundup(size, 1 << MIN_PBL_SHIFT);
if (rdev->stats.pbl.cur > rdev->stats.pbl.max)
rdev->stats.pbl.max = rdev->stats.pbl.cur;
- mutex_unlock(&rdev->stats.lock);
- }
+ } else
+ rdev->stats.pbl.fail++;
+ mutex_unlock(&rdev->stats.lock);
return (u32)addr;
}
@@ -401,13 +324,14 @@ u32 c4iw_rqtpool_alloc(struct c4iw_rdev *rdev, int size)
if (!addr)
printk_ratelimited(KERN_WARNING MOD "%s: Out of RQT memory\n",
pci_name(rdev->lldi.pdev));
+ mutex_lock(&rdev->stats.lock);
if (addr) {
- mutex_lock(&rdev->stats.lock);
rdev->stats.rqt.cur += roundup(size << 6, 1 << MIN_RQT_SHIFT);
if (rdev->stats.rqt.cur > rdev->stats.rqt.max)
rdev->stats.rqt.max = rdev->stats.rqt.cur;
- mutex_unlock(&rdev->stats.lock);
- }
+ } else
+ rdev->stats.rqt.fail++;
+ mutex_unlock(&rdev->stats.lock);
return (u32)addr;
}
--
1.7.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related
* Re: [V2 PATCH 2/9] macvtap: zerocopy: fix truesize underestimation
From: Jason Wang @ 2012-05-18 10:10 UTC (permalink / raw)
To: Shirley Ma; +Cc: eric.dumazet, mst, netdev, linux-kernel, ebiederm, davem
In-Reply-To: <1337268512.10741.53.camel@oc3660625478.ibm.com>
On 05/17/2012 11:28 PM, Shirley Ma wrote:
> On Thu, 2012-05-17 at 10:59 +0800, Jason Wang wrote:
>> Didn't see how this affact skb->len. And for truesize, I think they
>> are
>> different, when the offset were not zero, the data in this vector
>> were
>> divided into two parts. First part is copied into skb directly, and
>> the
>> second were pinned from a whole userspace page by
>> get_user_pages_fast(),
>> so we need count the whole page to the socket limit to prevent evil
>> application.
> What I meant that the code for skb->truesize has double added the first
> offset if any left from that vector (partically copied into skb
> directly, and then count pagesize which includes the offset (truesize +=
> PAGE_SIZE)).
Yes, I get you mean. There's no difference between first frag and
others: it's also possible for other frags that didn't occupy the whole
page. Since we pin the whole user page, better to count the whole page
size to prevent evil application.
> Thanks
> Shirley
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply
* [PATCH 08/10] RDMA/cxgb4: Use vmalloc for debugfs qp dump. Allows dumping thousands of qps.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul@chelsio.com>
Log active open failures of interest.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
drivers/infiniband/hw/cxgb4/cm.c | 18 ++++++++++++++++++
drivers/infiniband/hw/cxgb4/device.c | 4 ++--
2 files changed, 20 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/cm.c b/drivers/infiniband/hw/cxgb4/cm.c
index 6ce401a..55ab284 100644
--- a/drivers/infiniband/hw/cxgb4/cm.c
+++ b/drivers/infiniband/hw/cxgb4/cm.c
@@ -1413,6 +1413,24 @@ static int act_open_rpl(struct c4iw_dev *dev, struct sk_buff *skb)
return 0;
}
+ /*
+ * Log interesting failures.
+ */
+ switch (status) {
+ case CPL_ERR_CONN_RESET:
+ case CPL_ERR_CONN_TIMEDOUT:
+ break;
+ default:
+ printk(KERN_INFO MOD "Active open failure - "
+ "atid %u status %u errno %d %pI4:%u->%pI4:%u\n",
+ atid, status, status2errno(status),
+ &ep->com.local_addr.sin_addr.s_addr,
+ ntohs(ep->com.local_addr.sin_port),
+ &ep->com.remote_addr.sin_addr.s_addr,
+ ntohs(ep->com.remote_addr.sin_port));
+ break;
+ }
+
connect_reply_upcall(ep, status2errno(status));
state_set(&ep->com, DEAD);
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index bdb398f..8545629 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -121,7 +121,7 @@ static int qp_release(struct inode *inode, struct file *file)
printk(KERN_INFO "%s null qpd?\n", __func__);
return 0;
}
- kfree(qpd->buf);
+ vfree(qpd->buf);
kfree(qpd);
return 0;
}
@@ -145,7 +145,7 @@ static int qp_open(struct inode *inode, struct file *file)
spin_unlock_irq(&qpd->devp->lock);
qpd->bufsize = count * 128;
- qpd->buf = kmalloc(qpd->bufsize, GFP_KERNEL);
+ qpd->buf = vmalloc(qpd->bufsize);
if (!qpd->buf) {
ret = -ENOMEM;
goto err1;
--
1.7.1
^ permalink raw reply related
* [PATCH 00/10] Doorbell drop recovery for Chelsio T4 iWARP
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
This patch-series implements doorbell drop recovery for Chelsio T4 iWARP driver.
In the event where DBs are dropped application can get stalled for one or more
reasons. So, we recover RDMA and LLD queues in such an event.
We also take care for handling DB overflow events.
The patch-series also has some bug fixes, adds RDMA debugfs stats and removes
kfifo usage for ID mangement.
The patch-series is based on Roland's infiniband tree (for-next branch),
and involves changes on drivers/net/ethernet/chelsio/cxgb4 and
drivers/infiniband/hw/cxgb4.
The changes on drivers/infiniband/hw/cxgb4 are dependent on the changes of
drivers/net/ethernet/chelsio/cxgb4 for the T4-iWARP driver to build correctly.
So, we request to merge the entire patch-series through Roland's tree.
Both linux-rdma and netdev are included in this post for review.
The earlier posting of this series was reviewed and can be found at below link.
http://www.mail-archive.com/linux-rdma@vger.kernel.org/msg09606.html
Below is a link where Roland advised to re-post the series.
http://www.spinics.net/lists/netdev/msg187997.html
Vipul Pandya (10):
cxgb4: Detect DB FULL events and notify RDMA ULD.
cxgb4: Common platform specific changes for DB Drop Recovery
cxgb4: DB Drop Recovery for RDMA and LLD queues.
RDMA/cxgb4: Add debugfs rdma memory stats
RDMA/cxgb4: Add DB Overflow Avoidance.
RDMA/cxgb4: disable interrupts in c4iw_ev_dispatch().
RDMA/cxgb4: DB Drop Recovery for RDMA and LLD queues.
RDMA/cxgb4: Use vmalloc for debugfs qp dump. Allows dumping thousands
of qps.
RDMA/cxgb4: remove kfifo usage
RDMA/cxgb4: Add query_qp support in driver to query the qp state
before flushing.
drivers/infiniband/hw/cxgb4/Makefile | 2 +-
drivers/infiniband/hw/cxgb4/cm.c | 23 ++-
drivers/infiniband/hw/cxgb4/device.c | 339 ++++++++++++++++++++++-
drivers/infiniband/hw/cxgb4/ev.c | 8 +-
drivers/infiniband/hw/cxgb4/id_table.c | 112 ++++++++
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 134 ++++++++--
drivers/infiniband/hw/cxgb4/mem.c | 21 +-
drivers/infiniband/hw/cxgb4/provider.c | 19 +-
drivers/infiniband/hw/cxgb4/qp.c | 105 +++++++-
drivers/infiniband/hw/cxgb4/resource.c | 180 +++++-------
drivers/infiniband/hw/cxgb4/t4.h | 24 ++
drivers/infiniband/hw/cxgb4/user.h | 2 +-
drivers/net/ethernet/chelsio/cxgb4/cxgb4.h | 23 ++
drivers/net/ethernet/chelsio/cxgb4/cxgb4_main.c | 235 +++++++++++++++-
drivers/net/ethernet/chelsio/cxgb4/cxgb4_uld.h | 11 +
drivers/net/ethernet/chelsio/cxgb4/sge.c | 22 ++-
drivers/net/ethernet/chelsio/cxgb4/t4_hw.c | 62 ++++-
drivers/net/ethernet/chelsio/cxgb4/t4_regs.h | 53 ++++
drivers/net/ethernet/chelsio/cxgb4/t4fw_api.h | 15 +
19 files changed, 1218 insertions(+), 172 deletions(-)
create mode 100644 drivers/infiniband/hw/cxgb4/id_table.c
^ permalink raw reply
* [PATCH 10/10] RDMA/cxgb4: Add query_qp support in driver to query the qp state before flushing.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 19 +++++++++++++++++++
drivers/infiniband/hw/cxgb4/provider.c | 2 ++
drivers/infiniband/hw/cxgb4/qp.c | 11 +++++++++++
3 files changed, 32 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index 2d5b06b..9beb3a9 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -551,6 +551,23 @@ static inline int c4iw_convert_state(enum ib_qp_state ib_state)
}
}
+static inline int to_ib_qp_state(int c4iw_qp_state)
+{
+ switch (c4iw_qp_state) {
+ case C4IW_QP_STATE_IDLE:
+ return IB_QPS_INIT;
+ case C4IW_QP_STATE_RTS:
+ return IB_QPS_RTS;
+ case C4IW_QP_STATE_CLOSING:
+ return IB_QPS_SQD;
+ case C4IW_QP_STATE_TERMINATE:
+ return IB_QPS_SQE;
+ case C4IW_QP_STATE_ERROR:
+ return IB_QPS_ERR;
+ }
+ return IB_QPS_ERR;
+}
+
static inline u32 c4iw_ib_to_tpt_access(int a)
{
return (a & IB_ACCESS_REMOTE_WRITE ? FW_RI_MEM_ACCESS_REM_WRITE : 0) |
@@ -846,6 +863,8 @@ struct ib_qp *c4iw_create_qp(struct ib_pd *pd,
struct ib_udata *udata);
int c4iw_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
int attr_mask, struct ib_udata *udata);
+int c4iw_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_qp_init_attr *init_attr);
struct ib_qp *c4iw_get_qp(struct ib_device *dev, int qpn);
u32 c4iw_rqtpool_alloc(struct c4iw_rdev *rdev, int size);
void c4iw_rqtpool_free(struct c4iw_rdev *rdev, u32 addr, int size);
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index fe98a0a..e084fdc 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -443,6 +443,7 @@ int c4iw_register_device(struct c4iw_dev *dev)
(1ull << IB_USER_VERBS_CMD_REQ_NOTIFY_CQ) |
(1ull << IB_USER_VERBS_CMD_CREATE_QP) |
(1ull << IB_USER_VERBS_CMD_MODIFY_QP) |
+ (1ull << IB_USER_VERBS_CMD_QUERY_QP) |
(1ull << IB_USER_VERBS_CMD_POLL_CQ) |
(1ull << IB_USER_VERBS_CMD_DESTROY_QP) |
(1ull << IB_USER_VERBS_CMD_POST_SEND) |
@@ -465,6 +466,7 @@ int c4iw_register_device(struct c4iw_dev *dev)
dev->ibdev.destroy_ah = c4iw_ah_destroy;
dev->ibdev.create_qp = c4iw_create_qp;
dev->ibdev.modify_qp = c4iw_ib_modify_qp;
+ dev->ibdev.query_qp = c4iw_ib_query_qp;
dev->ibdev.destroy_qp = c4iw_destroy_qp;
dev->ibdev.create_cq = c4iw_create_cq;
dev->ibdev.destroy_cq = c4iw_destroy_cq;
diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
index ba1343e..45aedf1 100644
--- a/drivers/infiniband/hw/cxgb4/qp.c
+++ b/drivers/infiniband/hw/cxgb4/qp.c
@@ -1711,3 +1711,14 @@ struct ib_qp *c4iw_get_qp(struct ib_device *dev, int qpn)
PDBG("%s ib_dev %p qpn 0x%x\n", __func__, dev, qpn);
return (struct ib_qp *)get_qhp(to_c4iw_dev(dev), qpn);
}
+
+int c4iw_ib_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
+ int attr_mask, struct ib_qp_init_attr *init_attr)
+{
+ struct c4iw_qp *qhp = to_c4iw_qp(ibqp);
+
+ memset(attr, 0, sizeof *attr);
+ memset(init_attr, 0, sizeof *init_attr);
+ attr->qp_state = to_ib_qp_state(qhp->attr.state);
+ return 0;
+}
--
1.7.1
^ permalink raw reply related
* [PATCH 05/10] RDMA/cxgb4: Add DB Overflow Avoidance.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul@chelsio.com>
get FULL/EMPTY/DROP events from LLD
on FULL event, disable normal user mode DB rings.
add modify_qp semantics to allow user processes to call into
the kernel to ring doobells without overflowing.
Add DB Full/Empty/Drop stats.
Mark queues when created indicating the doorbell state.
If we're in the middle of db overflow avoidance, then newly created
queues should start out in this mode.
Bump the C4IW_UVERBS_ABI_VERSION to 2 so the user mode library can
know if the driver supports the kernel mode db ringing.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
drivers/infiniband/hw/cxgb4/device.c | 84 +++++++++++++++++++++++++++++--
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 37 ++++++++++++--
drivers/infiniband/hw/cxgb4/qp.c | 51 +++++++++++++++++++-
drivers/infiniband/hw/cxgb4/user.h | 2 +-
4 files changed, 162 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index 8483111..9062ed9 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -44,6 +44,12 @@ MODULE_DESCRIPTION("Chelsio T4 RDMA Driver");
MODULE_LICENSE("Dual BSD/GPL");
MODULE_VERSION(DRV_VERSION);
+struct uld_ctx {
+ struct list_head entry;
+ struct cxgb4_lld_info lldi;
+ struct c4iw_dev *dev;
+};
+
static LIST_HEAD(uld_ctx_list);
static DEFINE_MUTEX(dev_mutex);
@@ -263,6 +269,9 @@ static int stats_show(struct seq_file *seq, void *v)
seq_printf(seq, " OCQPMEM: %10llu %10llu %10llu\n",
dev->rdev.stats.ocqp.total, dev->rdev.stats.ocqp.cur,
dev->rdev.stats.ocqp.max);
+ seq_printf(seq, " DB FULL: %10llu\n", dev->rdev.stats.db_full);
+ seq_printf(seq, " DB EMPTY: %10llu\n", dev->rdev.stats.db_empty);
+ seq_printf(seq, " DB DROP: %10llu\n", dev->rdev.stats.db_drop);
return 0;
}
@@ -283,6 +292,9 @@ static ssize_t stats_clear(struct file *file, const char __user *buf,
dev->rdev.stats.pbl.max = 0;
dev->rdev.stats.rqt.max = 0;
dev->rdev.stats.ocqp.max = 0;
+ dev->rdev.stats.db_full = 0;
+ dev->rdev.stats.db_empty = 0;
+ dev->rdev.stats.db_drop = 0;
mutex_unlock(&dev->rdev.stats.lock);
return count;
}
@@ -443,12 +455,6 @@ static void c4iw_rdev_close(struct c4iw_rdev *rdev)
c4iw_destroy_resource(&rdev->resource);
}
-struct uld_ctx {
- struct list_head entry;
- struct cxgb4_lld_info lldi;
- struct c4iw_dev *dev;
-};
-
static void c4iw_dealloc(struct uld_ctx *ctx)
{
c4iw_rdev_close(&ctx->dev->rdev);
@@ -514,6 +520,7 @@ static struct c4iw_dev *c4iw_alloc(const struct cxgb4_lld_info *infop)
idr_init(&devp->mmidr);
spin_lock_init(&devp->lock);
mutex_init(&devp->rdev.stats.lock);
+ mutex_init(&devp->db_mutex);
if (c4iw_debugfs_root) {
devp->debugfs_root = debugfs_create_dir(
@@ -659,11 +666,76 @@ static int c4iw_uld_state_change(void *handle, enum cxgb4_state new_state)
return 0;
}
+static int disable_qp_db(int id, void *p, void *data)
+{
+ struct c4iw_qp *qp = p;
+
+ t4_disable_wq_db(&qp->wq);
+ return 0;
+}
+
+static void stop_queues(struct uld_ctx *ctx)
+{
+ spin_lock_irq(&ctx->dev->lock);
+ ctx->dev->db_state = FLOW_CONTROL;
+ idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
+ spin_unlock_irq(&ctx->dev->lock);
+}
+
+static int enable_qp_db(int id, void *p, void *data)
+{
+ struct c4iw_qp *qp = p;
+
+ t4_enable_wq_db(&qp->wq);
+ return 0;
+}
+
+static void resume_queues(struct uld_ctx *ctx)
+{
+ spin_lock_irq(&ctx->dev->lock);
+ ctx->dev->db_state = NORMAL;
+ idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
+ spin_unlock_irq(&ctx->dev->lock);
+}
+
+static int c4iw_uld_control(void *handle, enum cxgb4_control control, ...)
+{
+ struct uld_ctx *ctx = handle;
+
+ switch (control) {
+ case CXGB4_CONTROL_DB_FULL:
+ stop_queues(ctx);
+ mutex_lock(&ctx->dev->rdev.stats.lock);
+ ctx->dev->rdev.stats.db_full++;
+ mutex_unlock(&ctx->dev->rdev.stats.lock);
+ break;
+ case CXGB4_CONTROL_DB_EMPTY:
+ resume_queues(ctx);
+ mutex_lock(&ctx->dev->rdev.stats.lock);
+ ctx->dev->rdev.stats.db_empty++;
+ mutex_unlock(&ctx->dev->rdev.stats.lock);
+ break;
+ case CXGB4_CONTROL_DB_DROP:
+ printk(KERN_WARNING MOD "%s: Fatal DB DROP\n",
+ pci_name(ctx->lldi.pdev));
+ mutex_lock(&ctx->dev->rdev.stats.lock);
+ ctx->dev->rdev.stats.db_drop++;
+ mutex_unlock(&ctx->dev->rdev.stats.lock);
+ break;
+ default:
+ printk(KERN_WARNING MOD "%s: unknown control cmd %u\n",
+ pci_name(ctx->lldi.pdev), control);
+ break;
+ }
+ return 0;
+}
+
static struct cxgb4_uld_info c4iw_uld_info = {
.name = DRV_NAME,
.add = c4iw_uld_add,
.rx_handler = c4iw_uld_rx_handler,
.state_change = c4iw_uld_state_change,
+ .control = c4iw_uld_control,
};
static int __init c4iw_init_module(void)
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index a849074..a11ed5c 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -117,6 +117,9 @@ struct c4iw_stats {
struct c4iw_stat pbl;
struct c4iw_stat rqt;
struct c4iw_stat ocqp;
+ u64 db_full;
+ u64 db_empty;
+ u64 db_drop;
};
struct c4iw_rdev {
@@ -192,6 +195,12 @@ static inline int c4iw_wait_for_reply(struct c4iw_rdev *rdev,
return wr_waitp->ret;
}
+enum db_state {
+ NORMAL = 0,
+ FLOW_CONTROL = 1,
+ RECOVERY = 2
+};
+
struct c4iw_dev {
struct ib_device ibdev;
struct c4iw_rdev rdev;
@@ -200,7 +209,9 @@ struct c4iw_dev {
struct idr qpidr;
struct idr mmidr;
spinlock_t lock;
+ struct mutex db_mutex;
struct dentry *debugfs_root;
+ enum db_state db_state;
};
static inline struct c4iw_dev *to_c4iw_dev(struct ib_device *ibdev)
@@ -228,8 +239,8 @@ static inline struct c4iw_mr *get_mhp(struct c4iw_dev *rhp, u32 mmid)
return idr_find(&rhp->mmidr, mmid);
}
-static inline int insert_handle(struct c4iw_dev *rhp, struct idr *idr,
- void *handle, u32 id)
+static inline int _insert_handle(struct c4iw_dev *rhp, struct idr *idr,
+ void *handle, u32 id, int lock)
{
int ret;
int newid;
@@ -237,15 +248,29 @@ static inline int insert_handle(struct c4iw_dev *rhp, struct idr *idr,
do {
if (!idr_pre_get(idr, GFP_KERNEL))
return -ENOMEM;
- spin_lock_irq(&rhp->lock);
+ if (lock)
+ spin_lock_irq(&rhp->lock);
ret = idr_get_new_above(idr, handle, id, &newid);
BUG_ON(newid != id);
- spin_unlock_irq(&rhp->lock);
+ if (lock)
+ spin_unlock_irq(&rhp->lock);
} while (ret == -EAGAIN);
return ret;
}
+static inline int insert_handle(struct c4iw_dev *rhp, struct idr *idr,
+ void *handle, u32 id)
+{
+ return _insert_handle(rhp, idr, handle, id, 1);
+}
+
+static inline int insert_handle_nolock(struct c4iw_dev *rhp, struct idr *idr,
+ void *handle, u32 id)
+{
+ return _insert_handle(rhp, idr, handle, id, 0);
+}
+
static inline void remove_handle(struct c4iw_dev *rhp, struct idr *idr, u32 id)
{
spin_lock_irq(&rhp->lock);
@@ -370,6 +395,8 @@ struct c4iw_qp_attributes {
struct c4iw_ep *llp_stream_handle;
u8 layer_etype;
u8 ecode;
+ u16 sq_db_inc;
+ u16 rq_db_inc;
};
struct c4iw_qp {
@@ -444,6 +471,8 @@ static inline void insert_mmap(struct c4iw_ucontext *ucontext,
enum c4iw_qp_attr_mask {
C4IW_QP_ATTR_NEXT_STATE = 1 << 0,
+ C4IW_QP_ATTR_SQ_DB = 1<<1,
+ C4IW_QP_ATTR_RQ_DB = 1<<2,
C4IW_QP_ATTR_ENABLE_RDMA_READ = 1 << 7,
C4IW_QP_ATTR_ENABLE_RDMA_WRITE = 1 << 8,
C4IW_QP_ATTR_ENABLE_RDMA_BIND = 1 << 9,
diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
index 5f940ae..beec667 100644
--- a/drivers/infiniband/hw/cxgb4/qp.c
+++ b/drivers/infiniband/hw/cxgb4/qp.c
@@ -34,6 +34,10 @@
#include "iw_cxgb4.h"
+static int db_delay_usecs = 1;
+module_param(db_delay_usecs, int, 0644);
+MODULE_PARM_DESC(db_delay_usecs, "Usecs to delay awaiting db fifo to drain");
+
static int ocqp_support = 1;
module_param(ocqp_support, int, 0644);
MODULE_PARM_DESC(ocqp_support, "Support on-chip SQs (default=1)");
@@ -1128,6 +1132,29 @@ out:
return ret;
}
+/*
+ * Called by the library when the qp has user dbs disabled due to
+ * a DB_FULL condition. This function will single-thread all user
+ * DB rings to avoid overflowing the hw db-fifo.
+ */
+static int ring_kernel_db(struct c4iw_qp *qhp, u32 qid, u16 inc)
+{
+ int delay = db_delay_usecs;
+
+ mutex_lock(&qhp->rhp->db_mutex);
+ do {
+ if (cxgb4_dbfifo_count(qhp->rhp->rdev.lldi.ports[0], 1) < 768) {
+ writel(V_QID(qid) | V_PIDX(inc), qhp->wq.db);
+ break;
+ }
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(delay));
+ delay = min(delay << 1, 200000);
+ } while (1);
+ mutex_unlock(&qhp->rhp->db_mutex);
+ return 0;
+}
+
int c4iw_modify_qp(struct c4iw_dev *rhp, struct c4iw_qp *qhp,
enum c4iw_qp_attr_mask mask,
struct c4iw_qp_attributes *attrs,
@@ -1176,6 +1203,15 @@ int c4iw_modify_qp(struct c4iw_dev *rhp, struct c4iw_qp *qhp,
qhp->attr = newattr;
}
+ if (mask & C4IW_QP_ATTR_SQ_DB) {
+ ret = ring_kernel_db(qhp, qhp->wq.sq.qid, attrs->sq_db_inc);
+ goto out;
+ }
+ if (mask & C4IW_QP_ATTR_RQ_DB) {
+ ret = ring_kernel_db(qhp, qhp->wq.rq.qid, attrs->rq_db_inc);
+ goto out;
+ }
+
if (!(mask & C4IW_QP_ATTR_NEXT_STATE))
goto out;
if (qhp->attr.state == attrs->next_state)
@@ -1469,7 +1505,11 @@ struct ib_qp *c4iw_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *attrs,
init_waitqueue_head(&qhp->wait);
atomic_set(&qhp->refcnt, 1);
- ret = insert_handle(rhp, &rhp->qpidr, qhp, qhp->wq.sq.qid);
+ spin_lock_irq(&rhp->lock);
+ if (rhp->db_state != NORMAL)
+ t4_disable_wq_db(&qhp->wq);
+ ret = insert_handle_nolock(rhp, &rhp->qpidr, qhp, qhp->wq.sq.qid);
+ spin_unlock_irq(&rhp->lock);
if (ret)
goto err2;
@@ -1613,6 +1653,15 @@ int c4iw_ib_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr,
C4IW_QP_ATTR_ENABLE_RDMA_WRITE |
C4IW_QP_ATTR_ENABLE_RDMA_BIND) : 0;
+ /*
+ * Use SQ_PSN and RQ_PSN to pass in IDX_INC values for
+ * ringing the queue db when we're in DB_FULL mode.
+ */
+ attrs.sq_db_inc = attr->sq_psn;
+ attrs.rq_db_inc = attr->rq_psn;
+ mask |= (attr_mask & IB_QP_SQ_PSN) ? C4IW_QP_ATTR_SQ_DB : 0;
+ mask |= (attr_mask & IB_QP_RQ_PSN) ? C4IW_QP_ATTR_RQ_DB : 0;
+
return c4iw_modify_qp(rhp, qhp, mask, &attrs, 0);
}
diff --git a/drivers/infiniband/hw/cxgb4/user.h b/drivers/infiniband/hw/cxgb4/user.h
index e6669d5..32b754c 100644
--- a/drivers/infiniband/hw/cxgb4/user.h
+++ b/drivers/infiniband/hw/cxgb4/user.h
@@ -32,7 +32,7 @@
#ifndef __C4IW_USER_H__
#define __C4IW_USER_H__
-#define C4IW_UVERBS_ABI_VERSION 1
+#define C4IW_UVERBS_ABI_VERSION 2
/*
* Make sure that all structs defined in this file remain laid out so
--
1.7.1
^ permalink raw reply related
* [PATCH 04/10] RDMA/cxgb4: Add debugfs rdma memory stats
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul@chelsio.com>
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
drivers/infiniband/hw/cxgb4/device.c | 78 +++++++++++++++++++++++++++++++-
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 17 +++++++
drivers/infiniband/hw/cxgb4/mem.c | 11 ++++-
drivers/infiniband/hw/cxgb4/provider.c | 8 +++
drivers/infiniband/hw/cxgb4/resource.c | 44 ++++++++++++++++++
5 files changed, 155 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index 6d0df6e..8483111 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -240,6 +240,62 @@ static const struct file_operations stag_debugfs_fops = {
.llseek = default_llseek,
};
+static int stats_show(struct seq_file *seq, void *v)
+{
+ struct c4iw_dev *dev = seq->private;
+
+ seq_printf(seq, " Object: %10s %10s %10s\n", "Total", "Current", "Max");
+ seq_printf(seq, " PDID: %10llu %10llu %10llu\n",
+ dev->rdev.stats.pd.total, dev->rdev.stats.pd.cur,
+ dev->rdev.stats.pd.max);
+ seq_printf(seq, " QID: %10llu %10llu %10llu\n",
+ dev->rdev.stats.qid.total, dev->rdev.stats.qid.cur,
+ dev->rdev.stats.qid.max);
+ seq_printf(seq, " TPTMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.stag.total, dev->rdev.stats.stag.cur,
+ dev->rdev.stats.stag.max);
+ seq_printf(seq, " PBLMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.pbl.total, dev->rdev.stats.pbl.cur,
+ dev->rdev.stats.pbl.max);
+ seq_printf(seq, " RQTMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.rqt.total, dev->rdev.stats.rqt.cur,
+ dev->rdev.stats.rqt.max);
+ seq_printf(seq, " OCQPMEM: %10llu %10llu %10llu\n",
+ dev->rdev.stats.ocqp.total, dev->rdev.stats.ocqp.cur,
+ dev->rdev.stats.ocqp.max);
+ return 0;
+}
+
+static int stats_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, stats_show, inode->i_private);
+}
+
+static ssize_t stats_clear(struct file *file, const char __user *buf,
+ size_t count, loff_t *pos)
+{
+ struct c4iw_dev *dev = ((struct seq_file *)file->private_data)->private;
+
+ mutex_lock(&dev->rdev.stats.lock);
+ dev->rdev.stats.pd.max = 0;
+ dev->rdev.stats.qid.max = 0;
+ dev->rdev.stats.stag.max = 0;
+ dev->rdev.stats.pbl.max = 0;
+ dev->rdev.stats.rqt.max = 0;
+ dev->rdev.stats.ocqp.max = 0;
+ mutex_unlock(&dev->rdev.stats.lock);
+ return count;
+}
+
+static const struct file_operations stats_debugfs_fops = {
+ .owner = THIS_MODULE,
+ .open = stats_open,
+ .release = single_release,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .write = stats_clear,
+};
+
static int setup_debugfs(struct c4iw_dev *devp)
{
struct dentry *de;
@@ -256,6 +312,12 @@ static int setup_debugfs(struct c4iw_dev *devp)
(void *)devp, &stag_debugfs_fops);
if (de && de->d_inode)
de->d_inode->i_size = 4096;
+
+ de = debugfs_create_file("stats", S_IWUSR, devp->debugfs_root,
+ (void *)devp, &stats_debugfs_fops);
+ if (de && de->d_inode)
+ de->d_inode->i_size = 4096;
+
return 0;
}
@@ -269,9 +331,13 @@ void c4iw_release_dev_ucontext(struct c4iw_rdev *rdev,
list_for_each_safe(pos, nxt, &uctx->qpids) {
entry = list_entry(pos, struct c4iw_qid_list, entry);
list_del_init(&entry->entry);
- if (!(entry->qid & rdev->qpmask))
+ if (!(entry->qid & rdev->qpmask)) {
c4iw_put_resource(&rdev->resource.qid_fifo, entry->qid,
- &rdev->resource.qid_fifo_lock);
+ &rdev->resource.qid_fifo_lock);
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.qid.cur -= rdev->qpmask + 1;
+ mutex_unlock(&rdev->stats.lock);
+ }
kfree(entry);
}
@@ -332,6 +398,13 @@ static int c4iw_rdev_open(struct c4iw_rdev *rdev)
goto err1;
}
+ rdev->stats.pd.total = T4_MAX_NUM_PD;
+ rdev->stats.stag.total = rdev->lldi.vr->stag.size;
+ rdev->stats.pbl.total = rdev->lldi.vr->pbl.size;
+ rdev->stats.rqt.total = rdev->lldi.vr->rq.size;
+ rdev->stats.ocqp.total = rdev->lldi.vr->ocq.size;
+ rdev->stats.qid.total = rdev->lldi.vr->qp.size;
+
err = c4iw_init_resource(rdev, c4iw_num_stags(rdev), T4_MAX_NUM_PD);
if (err) {
printk(KERN_ERR MOD "error %d initializing resources\n", err);
@@ -440,6 +513,7 @@ static struct c4iw_dev *c4iw_alloc(const struct cxgb4_lld_info *infop)
idr_init(&devp->qpidr);
idr_init(&devp->mmidr);
spin_lock_init(&devp->lock);
+ mutex_init(&devp->rdev.stats.lock);
if (c4iw_debugfs_root) {
devp->debugfs_root = debugfs_create_dir(
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index 1357c5b..a849074 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -103,6 +103,22 @@ enum c4iw_rdev_flags {
T4_FATAL_ERROR = (1<<0),
};
+struct c4iw_stat {
+ u64 total;
+ u64 cur;
+ u64 max;
+};
+
+struct c4iw_stats {
+ struct mutex lock;
+ struct c4iw_stat qid;
+ struct c4iw_stat pd;
+ struct c4iw_stat stag;
+ struct c4iw_stat pbl;
+ struct c4iw_stat rqt;
+ struct c4iw_stat ocqp;
+};
+
struct c4iw_rdev {
struct c4iw_resource resource;
unsigned long qpshift;
@@ -117,6 +133,7 @@ struct c4iw_rdev {
struct cxgb4_lld_info lldi;
unsigned long oc_mw_pa;
void __iomem *oc_mw_kva;
+ struct c4iw_stats stats;
};
static inline int c4iw_fatal_error(struct c4iw_rdev *rdev)
diff --git a/drivers/infiniband/hw/cxgb4/mem.c b/drivers/infiniband/hw/cxgb4/mem.c
index 40c8353..2a87379 100644
--- a/drivers/infiniband/hw/cxgb4/mem.c
+++ b/drivers/infiniband/hw/cxgb4/mem.c
@@ -135,6 +135,11 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
&rdev->resource.tpt_fifo_lock);
if (!stag_idx)
return -ENOMEM;
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.stag.cur += 32;
+ if (rdev->stats.stag.cur > rdev->stats.stag.max)
+ rdev->stats.stag.max = rdev->stats.stag.cur;
+ mutex_unlock(&rdev->stats.lock);
*stag = (stag_idx << 8) | (atomic_inc_return(&key) & 0xff);
}
PDBG("%s stag_state 0x%0x type 0x%0x pdid 0x%0x, stag_idx 0x%x\n",
@@ -165,9 +170,13 @@ static int write_tpt_entry(struct c4iw_rdev *rdev, u32 reset_tpt_entry,
(rdev->lldi.vr->stag.start >> 5),
sizeof(tpt), &tpt);
- if (reset_tpt_entry)
+ if (reset_tpt_entry) {
c4iw_put_resource(&rdev->resource.tpt_fifo, stag_idx,
&rdev->resource.tpt_fifo_lock);
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.stag.cur -= 32;
+ mutex_unlock(&rdev->stats.lock);
+ }
return err;
}
diff --git a/drivers/infiniband/hw/cxgb4/provider.c b/drivers/infiniband/hw/cxgb4/provider.c
index be1c18f..8d58736 100644
--- a/drivers/infiniband/hw/cxgb4/provider.c
+++ b/drivers/infiniband/hw/cxgb4/provider.c
@@ -190,6 +190,9 @@ static int c4iw_deallocate_pd(struct ib_pd *pd)
PDBG("%s ibpd %p pdid 0x%x\n", __func__, pd, php->pdid);
c4iw_put_resource(&rhp->rdev.resource.pdid_fifo, php->pdid,
&rhp->rdev.resource.pdid_fifo_lock);
+ mutex_lock(&rhp->rdev.stats.lock);
+ rhp->rdev.stats.pd.cur--;
+ mutex_unlock(&rhp->rdev.stats.lock);
kfree(php);
return 0;
}
@@ -222,6 +225,11 @@ static struct ib_pd *c4iw_allocate_pd(struct ib_device *ibdev,
return ERR_PTR(-EFAULT);
}
}
+ mutex_lock(&rhp->rdev.stats.lock);
+ rhp->rdev.stats.pd.cur++;
+ if (rhp->rdev.stats.pd.cur > rhp->rdev.stats.pd.max)
+ rhp->rdev.stats.pd.max = rhp->rdev.stats.pd.cur;
+ mutex_unlock(&rhp->rdev.stats.lock);
PDBG("%s pdid 0x%0x ptr 0x%p\n", __func__, pdid, php);
return &php->ibpd;
}
diff --git a/drivers/infiniband/hw/cxgb4/resource.c b/drivers/infiniband/hw/cxgb4/resource.c
index 407ff39..1b948d1 100644
--- a/drivers/infiniband/hw/cxgb4/resource.c
+++ b/drivers/infiniband/hw/cxgb4/resource.c
@@ -185,6 +185,9 @@ u32 c4iw_get_cqid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
&rdev->resource.qid_fifo_lock);
if (!qid)
goto out;
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.qid.cur += rdev->qpmask + 1;
+ mutex_unlock(&rdev->stats.lock);
for (i = qid+1; i & rdev->qpmask; i++) {
entry = kmalloc(sizeof *entry, GFP_KERNEL);
if (!entry)
@@ -213,6 +216,10 @@ u32 c4iw_get_cqid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
out:
mutex_unlock(&uctx->lock);
PDBG("%s qid 0x%x\n", __func__, qid);
+ mutex_lock(&rdev->stats.lock);
+ if (rdev->stats.qid.cur > rdev->stats.qid.max)
+ rdev->stats.qid.max = rdev->stats.qid.cur;
+ mutex_unlock(&rdev->stats.lock);
return qid;
}
@@ -249,6 +256,9 @@ u32 c4iw_get_qpid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
&rdev->resource.qid_fifo_lock);
if (!qid)
goto out;
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.qid.cur += rdev->qpmask + 1;
+ mutex_unlock(&rdev->stats.lock);
for (i = qid+1; i & rdev->qpmask; i++) {
entry = kmalloc(sizeof *entry, GFP_KERNEL);
if (!entry)
@@ -277,6 +287,10 @@ u32 c4iw_get_qpid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx)
out:
mutex_unlock(&uctx->lock);
PDBG("%s qid 0x%x\n", __func__, qid);
+ mutex_lock(&rdev->stats.lock);
+ if (rdev->stats.qid.cur > rdev->stats.qid.max)
+ rdev->stats.qid.max = rdev->stats.qid.cur;
+ mutex_unlock(&rdev->stats.lock);
return qid;
}
@@ -315,12 +329,22 @@ u32 c4iw_pblpool_alloc(struct c4iw_rdev *rdev, int size)
if (!addr)
printk_ratelimited(KERN_WARNING MOD "%s: Out of PBL memory\n",
pci_name(rdev->lldi.pdev));
+ if (addr) {
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.pbl.cur += roundup(size, 1 << MIN_PBL_SHIFT);
+ if (rdev->stats.pbl.cur > rdev->stats.pbl.max)
+ rdev->stats.pbl.max = rdev->stats.pbl.cur;
+ mutex_unlock(&rdev->stats.lock);
+ }
return (u32)addr;
}
void c4iw_pblpool_free(struct c4iw_rdev *rdev, u32 addr, int size)
{
PDBG("%s addr 0x%x size %d\n", __func__, addr, size);
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.pbl.cur -= roundup(size, 1 << MIN_PBL_SHIFT);
+ mutex_unlock(&rdev->stats.lock);
gen_pool_free(rdev->pbl_pool, (unsigned long)addr, size);
}
@@ -377,12 +401,22 @@ u32 c4iw_rqtpool_alloc(struct c4iw_rdev *rdev, int size)
if (!addr)
printk_ratelimited(KERN_WARNING MOD "%s: Out of RQT memory\n",
pci_name(rdev->lldi.pdev));
+ if (addr) {
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.rqt.cur += roundup(size << 6, 1 << MIN_RQT_SHIFT);
+ if (rdev->stats.rqt.cur > rdev->stats.rqt.max)
+ rdev->stats.rqt.max = rdev->stats.rqt.cur;
+ mutex_unlock(&rdev->stats.lock);
+ }
return (u32)addr;
}
void c4iw_rqtpool_free(struct c4iw_rdev *rdev, u32 addr, int size)
{
PDBG("%s addr 0x%x size %d\n", __func__, addr, size << 6);
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.rqt.cur -= roundup(size << 6, 1 << MIN_RQT_SHIFT);
+ mutex_unlock(&rdev->stats.lock);
gen_pool_free(rdev->rqt_pool, (unsigned long)addr, size << 6);
}
@@ -433,12 +467,22 @@ u32 c4iw_ocqp_pool_alloc(struct c4iw_rdev *rdev, int size)
{
unsigned long addr = gen_pool_alloc(rdev->ocqp_pool, size);
PDBG("%s addr 0x%x size %d\n", __func__, (u32)addr, size);
+ if (addr) {
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.ocqp.cur += roundup(size, 1 << MIN_OCQP_SHIFT);
+ if (rdev->stats.ocqp.cur > rdev->stats.ocqp.max)
+ rdev->stats.ocqp.max = rdev->stats.ocqp.cur;
+ mutex_unlock(&rdev->stats.lock);
+ }
return (u32)addr;
}
void c4iw_ocqp_pool_free(struct c4iw_rdev *rdev, u32 addr, int size)
{
PDBG("%s addr 0x%x size %d\n", __func__, addr, size);
+ mutex_lock(&rdev->stats.lock);
+ rdev->stats.ocqp.cur -= roundup(size, 1 << MIN_OCQP_SHIFT);
+ mutex_unlock(&rdev->stats.lock);
gen_pool_free(rdev->ocqp_pool, (unsigned long)addr, size);
}
--
1.7.1
^ permalink raw reply related
* [PATCH 07/10] RDMA/cxgb4: DB Drop Recovery for RDMA and LLD queues.
From: Vipul Pandya @ 2012-05-18 9:59 UTC (permalink / raw)
To: linux-rdma, netdev; +Cc: roland, davem, divy, dm, kumaras, swise, Vipul Pandya
In-Reply-To: <1337335173-3226-1-git-send-email-vipul@chelsio.com>
add module option db_fc_threshold which is the count of active QPs
that trigger automatic db flow control mode.
automatically transition to/from flow control mode when the active qp
count crosses db_fc_theshold.
add more db debugfs stats
on DB DROP event from the LLD, recover all the iwarp queues.
Signed-off-by: Vipul Pandya <vipul@chelsio.com>
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
drivers/infiniband/hw/cxgb4/device.c | 176 ++++++++++++++++++++++++++++++-
drivers/infiniband/hw/cxgb4/iw_cxgb4.h | 24 ++++-
drivers/infiniband/hw/cxgb4/qp.c | 47 ++++++++-
drivers/infiniband/hw/cxgb4/t4.h | 24 +++++
4 files changed, 259 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb4/device.c b/drivers/infiniband/hw/cxgb4/device.c
index 9062ed9..bdb398f 100644
--- a/drivers/infiniband/hw/cxgb4/device.c
+++ b/drivers/infiniband/hw/cxgb4/device.c
@@ -246,6 +246,8 @@ static const struct file_operations stag_debugfs_fops = {
.llseek = default_llseek,
};
+static char *db_state_str[] = {"NORMAL", "FLOW_CONTROL", "RECOVERY"};
+
static int stats_show(struct seq_file *seq, void *v)
{
struct c4iw_dev *dev = seq->private;
@@ -272,6 +274,9 @@ static int stats_show(struct seq_file *seq, void *v)
seq_printf(seq, " DB FULL: %10llu\n", dev->rdev.stats.db_full);
seq_printf(seq, " DB EMPTY: %10llu\n", dev->rdev.stats.db_empty);
seq_printf(seq, " DB DROP: %10llu\n", dev->rdev.stats.db_drop);
+ seq_printf(seq, " DB State: %s Transitions %llu\n",
+ db_state_str[dev->db_state],
+ dev->rdev.stats.db_state_transitions);
return 0;
}
@@ -295,6 +300,7 @@ static ssize_t stats_clear(struct file *file, const char __user *buf,
dev->rdev.stats.db_full = 0;
dev->rdev.stats.db_empty = 0;
dev->rdev.stats.db_drop = 0;
+ dev->rdev.stats.db_state_transitions = 0;
mutex_unlock(&dev->rdev.stats.lock);
return count;
}
@@ -677,8 +683,11 @@ static int disable_qp_db(int id, void *p, void *data)
static void stop_queues(struct uld_ctx *ctx)
{
spin_lock_irq(&ctx->dev->lock);
- ctx->dev->db_state = FLOW_CONTROL;
- idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
+ if (ctx->dev->db_state == NORMAL) {
+ ctx->dev->rdev.stats.db_state_transitions++;
+ ctx->dev->db_state = FLOW_CONTROL;
+ idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
+ }
spin_unlock_irq(&ctx->dev->lock);
}
@@ -693,9 +702,165 @@ static int enable_qp_db(int id, void *p, void *data)
static void resume_queues(struct uld_ctx *ctx)
{
spin_lock_irq(&ctx->dev->lock);
- ctx->dev->db_state = NORMAL;
- idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
+ if (ctx->dev->qpcnt <= db_fc_threshold &&
+ ctx->dev->db_state == FLOW_CONTROL) {
+ ctx->dev->db_state = NORMAL;
+ ctx->dev->rdev.stats.db_state_transitions++;
+ idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
+ }
+ spin_unlock_irq(&ctx->dev->lock);
+}
+
+struct qp_list {
+ unsigned idx;
+ struct c4iw_qp **qps;
+};
+
+static int add_and_ref_qp(int id, void *p, void *data)
+{
+ struct qp_list *qp_listp = data;
+ struct c4iw_qp *qp = p;
+
+ c4iw_qp_add_ref(&qp->ibqp);
+ qp_listp->qps[qp_listp->idx++] = qp;
+ return 0;
+}
+
+static int count_qps(int id, void *p, void *data)
+{
+ unsigned *countp = data;
+ (*countp)++;
+ return 0;
+}
+
+static void deref_qps(struct qp_list qp_list)
+{
+ int idx;
+
+ for (idx = 0; idx < qp_list.idx; idx++)
+ c4iw_qp_rem_ref(&qp_list.qps[idx]->ibqp);
+}
+
+static void recover_lost_dbs(struct uld_ctx *ctx, struct qp_list *qp_list)
+{
+ int idx;
+ int ret;
+
+ for (idx = 0; idx < qp_list->idx; idx++) {
+ struct c4iw_qp *qp = qp_list->qps[idx];
+
+ ret = cxgb4_sync_txq_pidx(qp->rhp->rdev.lldi.ports[0],
+ qp->wq.sq.qid,
+ t4_sq_host_wq_pidx(&qp->wq),
+ t4_sq_wq_size(&qp->wq));
+ if (ret) {
+ printk(KERN_ERR MOD "%s: Fatal error - "
+ "DB overflow recovery failed - "
+ "error syncing SQ qid %u\n",
+ pci_name(ctx->lldi.pdev), qp->wq.sq.qid);
+ return;
+ }
+
+ ret = cxgb4_sync_txq_pidx(qp->rhp->rdev.lldi.ports[0],
+ qp->wq.rq.qid,
+ t4_rq_host_wq_pidx(&qp->wq),
+ t4_rq_wq_size(&qp->wq));
+
+ if (ret) {
+ printk(KERN_ERR MOD "%s: Fatal error - "
+ "DB overflow recovery failed - "
+ "error syncing RQ qid %u\n",
+ pci_name(ctx->lldi.pdev), qp->wq.rq.qid);
+ return;
+ }
+
+ /* Wait for the dbfifo to drain */
+ while (cxgb4_dbfifo_count(qp->rhp->rdev.lldi.ports[0], 1) > 0) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(10));
+ }
+ }
+}
+
+static void recover_queues(struct uld_ctx *ctx)
+{
+ int count = 0;
+ struct qp_list qp_list;
+ int ret;
+
+ /* lock out kernel db ringers */
+ mutex_lock(&ctx->dev->db_mutex);
+
+ /* put all queues in to recovery mode */
+ spin_lock_irq(&ctx->dev->lock);
+ ctx->dev->db_state = RECOVERY;
+ ctx->dev->rdev.stats.db_state_transitions++;
+ idr_for_each(&ctx->dev->qpidr, disable_qp_db, NULL);
+ spin_unlock_irq(&ctx->dev->lock);
+
+ /* slow everybody down */
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(1000));
+
+ /* Wait for the dbfifo to completely drain. */
+ while (cxgb4_dbfifo_count(ctx->dev->rdev.lldi.ports[0], 1) > 0) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(10));
+ }
+
+ /* flush the SGE contexts */
+ ret = cxgb4_flush_eq_cache(ctx->dev->rdev.lldi.ports[0]);
+ if (ret) {
+ printk(KERN_ERR MOD "%s: Fatal error - DB overflow recovery failed\n",
+ pci_name(ctx->lldi.pdev));
+ goto out;
+ }
+
+ /* Count active queues so we can build a list of queues to recover */
+ spin_lock_irq(&ctx->dev->lock);
+ idr_for_each(&ctx->dev->qpidr, count_qps, &count);
+
+ qp_list.qps = kzalloc(count * sizeof *qp_list.qps, GFP_ATOMIC);
+ if (!qp_list.qps) {
+ printk(KERN_ERR MOD "%s: Fatal error - DB overflow recovery failed\n",
+ pci_name(ctx->lldi.pdev));
+ spin_unlock_irq(&ctx->dev->lock);
+ goto out;
+ }
+ qp_list.idx = 0;
+
+ /* add and ref each qp so it doesn't get freed */
+ idr_for_each(&ctx->dev->qpidr, add_and_ref_qp, &qp_list);
+
spin_unlock_irq(&ctx->dev->lock);
+
+ /* now traverse the list in a safe context to recover the db state*/
+ recover_lost_dbs(ctx, &qp_list);
+
+ /* we're almost done! deref the qps and clean up */
+ deref_qps(qp_list);
+ kfree(qp_list.qps);
+
+ /* Wait for the dbfifo to completely drain again */
+ while (cxgb4_dbfifo_count(ctx->dev->rdev.lldi.ports[0], 1) > 0) {
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(usecs_to_jiffies(10));
+ }
+
+ /* resume the queues */
+ spin_lock_irq(&ctx->dev->lock);
+ if (ctx->dev->qpcnt > db_fc_threshold)
+ ctx->dev->db_state = FLOW_CONTROL;
+ else {
+ ctx->dev->db_state = NORMAL;
+ idr_for_each(&ctx->dev->qpidr, enable_qp_db, NULL);
+ }
+ ctx->dev->rdev.stats.db_state_transitions++;
+ spin_unlock_irq(&ctx->dev->lock);
+
+out:
+ /* start up kernel db ringers again */
+ mutex_unlock(&ctx->dev->db_mutex);
}
static int c4iw_uld_control(void *handle, enum cxgb4_control control, ...)
@@ -716,8 +881,7 @@ static int c4iw_uld_control(void *handle, enum cxgb4_control control, ...)
mutex_unlock(&ctx->dev->rdev.stats.lock);
break;
case CXGB4_CONTROL_DB_DROP:
- printk(KERN_WARNING MOD "%s: Fatal DB DROP\n",
- pci_name(ctx->lldi.pdev));
+ recover_queues(ctx);
mutex_lock(&ctx->dev->rdev.stats.lock);
ctx->dev->rdev.stats.db_drop++;
mutex_unlock(&ctx->dev->rdev.stats.lock);
diff --git a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
index e8b88a0..6818659 100644
--- a/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
+++ b/drivers/infiniband/hw/cxgb4/iw_cxgb4.h
@@ -120,6 +120,7 @@ struct c4iw_stats {
u64 db_full;
u64 db_empty;
u64 db_drop;
+ u64 db_state_transitions;
};
struct c4iw_rdev {
@@ -212,6 +213,7 @@ struct c4iw_dev {
struct mutex db_mutex;
struct dentry *debugfs_root;
enum db_state db_state;
+ int qpcnt;
};
static inline struct c4iw_dev *to_c4iw_dev(struct ib_device *ibdev)
@@ -271,11 +273,25 @@ static inline int insert_handle_nolock(struct c4iw_dev *rhp, struct idr *idr,
return _insert_handle(rhp, idr, handle, id, 0);
}
-static inline void remove_handle(struct c4iw_dev *rhp, struct idr *idr, u32 id)
+static inline void _remove_handle(struct c4iw_dev *rhp, struct idr *idr,
+ u32 id, int lock)
{
- spin_lock_irq(&rhp->lock);
+ if (lock)
+ spin_lock_irq(&rhp->lock);
idr_remove(idr, id);
- spin_unlock_irq(&rhp->lock);
+ if (lock)
+ spin_unlock_irq(&rhp->lock);
+}
+
+static inline void remove_handle(struct c4iw_dev *rhp, struct idr *idr, u32 id)
+{
+ _remove_handle(rhp, idr, id, 1);
+}
+
+static inline void remove_handle_nolock(struct c4iw_dev *rhp,
+ struct idr *idr, u32 id)
+{
+ _remove_handle(rhp, idr, id, 0);
}
struct c4iw_pd {
@@ -843,5 +859,7 @@ void c4iw_ev_dispatch(struct c4iw_dev *dev, struct t4_cqe *err_cqe);
extern struct cxgb4_client t4c_client;
extern c4iw_handler_func c4iw_handlers[NUM_CPL_CMDS];
extern int c4iw_max_read_depth;
+extern int db_fc_threshold;
+
#endif
diff --git a/drivers/infiniband/hw/cxgb4/qp.c b/drivers/infiniband/hw/cxgb4/qp.c
index beec667..ba1343e 100644
--- a/drivers/infiniband/hw/cxgb4/qp.c
+++ b/drivers/infiniband/hw/cxgb4/qp.c
@@ -42,6 +42,11 @@ static int ocqp_support = 1;
module_param(ocqp_support, int, 0644);
MODULE_PARM_DESC(ocqp_support, "Support on-chip SQs (default=1)");
+int db_fc_threshold = 2000;
+module_param(db_fc_threshold, int, 0644);
+MODULE_PARM_DESC(db_fc_threshold, "QP count/threshold that triggers automatic "
+ "db flow control mode (default = 2000)");
+
static void set_state(struct c4iw_qp *qhp, enum c4iw_qp_state state)
{
unsigned long flag;
@@ -1143,13 +1148,19 @@ static int ring_kernel_db(struct c4iw_qp *qhp, u32 qid, u16 inc)
mutex_lock(&qhp->rhp->db_mutex);
do {
- if (cxgb4_dbfifo_count(qhp->rhp->rdev.lldi.ports[0], 1) < 768) {
+
+ /*
+ * The interrupt threshold is dbfifo_int_thresh << 6. So
+ * make sure we don't cross that and generate an interrupt.
+ */
+ if (cxgb4_dbfifo_count(qhp->rhp->rdev.lldi.ports[0], 1) <
+ (qhp->rhp->rdev.lldi.dbfifo_int_thresh << 5)) {
writel(V_QID(qid) | V_PIDX(inc), qhp->wq.db);
break;
}
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(usecs_to_jiffies(delay));
- delay = min(delay << 1, 200000);
+ delay = min(delay << 1, 2000);
} while (1);
mutex_unlock(&qhp->rhp->db_mutex);
return 0;
@@ -1388,6 +1399,14 @@ out:
return ret;
}
+static int enable_qp_db(int id, void *p, void *data)
+{
+ struct c4iw_qp *qp = p;
+
+ t4_enable_wq_db(&qp->wq);
+ return 0;
+}
+
int c4iw_destroy_qp(struct ib_qp *ib_qp)
{
struct c4iw_dev *rhp;
@@ -1405,7 +1424,16 @@ int c4iw_destroy_qp(struct ib_qp *ib_qp)
c4iw_modify_qp(rhp, qhp, C4IW_QP_ATTR_NEXT_STATE, &attrs, 0);
wait_event(qhp->wait, !qhp->ep);
- remove_handle(rhp, &rhp->qpidr, qhp->wq.sq.qid);
+ spin_lock_irq(&rhp->lock);
+ remove_handle_nolock(rhp, &rhp->qpidr, qhp->wq.sq.qid);
+ rhp->qpcnt--;
+ BUG_ON(rhp->qpcnt < 0);
+ if (rhp->qpcnt <= db_fc_threshold && rhp->db_state == FLOW_CONTROL) {
+ rhp->rdev.stats.db_state_transitions++;
+ rhp->db_state = NORMAL;
+ idr_for_each(&rhp->qpidr, enable_qp_db, NULL);
+ }
+ spin_unlock_irq(&rhp->lock);
atomic_dec(&qhp->refcnt);
wait_event(qhp->wait, !atomic_read(&qhp->refcnt));
@@ -1419,6 +1447,14 @@ int c4iw_destroy_qp(struct ib_qp *ib_qp)
return 0;
}
+static int disable_qp_db(int id, void *p, void *data)
+{
+ struct c4iw_qp *qp = p;
+
+ t4_disable_wq_db(&qp->wq);
+ return 0;
+}
+
struct ib_qp *c4iw_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *attrs,
struct ib_udata *udata)
{
@@ -1508,6 +1544,11 @@ struct ib_qp *c4iw_create_qp(struct ib_pd *pd, struct ib_qp_init_attr *attrs,
spin_lock_irq(&rhp->lock);
if (rhp->db_state != NORMAL)
t4_disable_wq_db(&qhp->wq);
+ if (++rhp->qpcnt > db_fc_threshold && rhp->db_state == NORMAL) {
+ rhp->rdev.stats.db_state_transitions++;
+ rhp->db_state = FLOW_CONTROL;
+ idr_for_each(&rhp->qpidr, disable_qp_db, NULL);
+ }
ret = insert_handle_nolock(rhp, &rhp->qpidr, qhp, qhp->wq.sq.qid);
spin_unlock_irq(&rhp->lock);
if (ret)
diff --git a/drivers/infiniband/hw/cxgb4/t4.h b/drivers/infiniband/hw/cxgb4/t4.h
index c0221ee..16f26ab 100644
--- a/drivers/infiniband/hw/cxgb4/t4.h
+++ b/drivers/infiniband/hw/cxgb4/t4.h
@@ -62,6 +62,10 @@ struct t4_status_page {
__be16 pidx;
u8 qp_err; /* flit 1 - sw owns */
u8 db_off;
+ u8 pad;
+ u16 host_wq_pidx;
+ u16 host_cidx;
+ u16 host_pidx;
};
#define T4_EQ_ENTRY_SIZE 64
@@ -375,6 +379,16 @@ static inline void t4_rq_consume(struct t4_wq *wq)
wq->rq.cidx = 0;
}
+static inline u16 t4_rq_host_wq_pidx(struct t4_wq *wq)
+{
+ return wq->rq.queue[wq->rq.size].status.host_wq_pidx;
+}
+
+static inline u16 t4_rq_wq_size(struct t4_wq *wq)
+{
+ return wq->rq.size * T4_RQ_NUM_SLOTS;
+}
+
static inline int t4_sq_onchip(struct t4_sq *sq)
{
return sq->flags & T4_SQ_ONCHIP;
@@ -412,6 +426,16 @@ static inline void t4_sq_consume(struct t4_wq *wq)
wq->sq.cidx = 0;
}
+static inline u16 t4_sq_host_wq_pidx(struct t4_wq *wq)
+{
+ return wq->sq.queue[wq->sq.size].status.host_wq_pidx;
+}
+
+static inline u16 t4_sq_wq_size(struct t4_wq *wq)
+{
+ return wq->sq.size * T4_SQ_NUM_SLOTS;
+}
+
static inline void t4_ring_sq_db(struct t4_wq *wq, u16 inc)
{
wmb();
--
1.7.1
^ permalink raw reply related
* Re: Kernel consistently panicing on br_parse_ip_options
From: Massimo Cetra @ 2012-05-18 10:40 UTC (permalink / raw)
To: linux-kernel, netdev; +Cc: Eric Dumazet, David Miller
In-Reply-To: <4FB0FC82.70003@navynet.it>
[-- Attachment #1: Type: text/plain, Size: 939 bytes --]
On 14/05/2012 14:37, Massimo Cetra wrote:
> Hello,
>
> I had already filed similar panics a month ago.
> Today i upgraded to 3.2.16 and nothing seems to be changed (and i don't
> see anything related in .17).
>
> The server (a Dell R410 with a couple of bnx2 ethernet cards) has two
> bridges onboard.
>
> Each bridge is connected to a different switch and has 2 uses:
> - one bridge is connecting an internal network and the KVM hosts that
> run on the same machine
> - one bridge connects the server to the public network along with
> another bunch of kvm servers whose interfaces bridges
>
> The bug can be easily triggered adding or removing (with heartbeat) a
> virtual address (br0:1, for example) .
>
> Is there any known fix or patch ?
As an attachment you may find another panic that seems compatible with
the previous one.
The crash seems consistent and reproduceable. Is there anyone worrying
about this ?
Thanks
Massimo
[-- Attachment #2: panic2.txt --]
[-- Type: text/plain, Size: 14705 bytes --]
May 18 11:24:05 172.30.1.2 [335035.342152] BUG: unable to handle kernel
May 18 11:24:05 172.30.1.2 [335035.357985] IP:
May 18 11:24:05 172.30.1.2 [335035.372242] PGD 0
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.376433] Oops: 0000 [#1]
May 18 11:24:05 172.30.1.2 SMP
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.383065] CPU 0
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.386893] Modules linked in:
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.526015]
May 18 11:24:05 172.30.1.2 [335035.529151] Pid: 4321, comm: kvm Not tainted 3.2.0-2-amd64 #1
May 18 11:24:05 172.30.1.2 /0N051F
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.546372] RIP: 0010:[<ffffffffa0294336>]
May 18 11:24:05 172.30.1.2 [335035.565482] RSP: 0018:ffff88042fc03b18 EFLAGS: 00010293
May 18 11:24:05 172.30.1.2 [335035.576265] RAX: 0000000000000000 RBX: ffff8803b5395dc0 RCX: 0000000000000007
May 18 11:24:05 172.30.1.2 [335035.590705] RDX: ffffffffa0294308 RSI: 0000000104fdde80 RDI: ffff8803b5395dc0
May 18 11:24:05 172.30.1.2 [335035.605145] RBP: ffff8804261ba000 R08: 0000000000000000 R09: ffff88042fc03ad0
May 18 11:24:05 172.30.1.2 [335035.619585] R10: ffffffff8165aac0 R11: ffffffff8165aac0 R12: 0000000000000000
May 18 11:24:05 172.30.1.2 [335035.634022] R13: ffff880425158002 R14: ffff880425c21f00 R15: ffff880425158000
May 18 11:24:05 172.30.1.2 [335035.648462] FS: 00007f6793d51900(0000) GS:ffff88042fc00000(0000) knlGS:0000000000000000
May 18 11:24:05 172.30.1.2 [335035.664808] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
May 18 11:24:05 172.30.1.2 [335035.676457] CR2: 0000000000000018 CR3: 0000000211fcf000 CR4: 00000000000026e0
May 18 11:24:05 172.30.1.2 [335035.690897] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
May 18 11:24:05 172.30.1.2 [335035.705336] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
May 18 11:24:05 172.30.1.2 [335035.719776] Process kvm (pid: 4321, threadinfo ffff880211b98000, task ffff88022594c870)
May 18 11:24:05 172.30.1.2 [335035.735948] Stack:
May 18 11:24:05 172.30.1.2 [335035.740140] ffffffff80000000
May 18 11:24:05 172.30.1.2
May 18 11:24:05 172.30.1.2 [335035.755147] ffff8802279f7000
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 [335035.770154] ffff8803b5395dc0
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 [335035.785170] Call Trace:
May 18 11:24:06 172.30.1.2 [335035.790229] <IRQ>
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 [335035.794611] [<ffffffffa02946db>] ? br_parse_ip_options+0x3d/0x19a [bridge]
May 18 11:24:06 172.30.1.2 [335035.808693] [<ffffffffa0294a67>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
May 18 11:24:06 172.30.1.2 [335035.822428] [<ffffffff812ac039>] ? nf_iterate+0x41/0x77
May 18 11:24:06 172.30.1.2 [335035.833213] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335035.845906] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335035.858597] [<ffffffff812ac0d7>] ? nf_hook_slow+0x68/0x101
May 18 11:24:06 172.30.1.2 [335035.869902] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335035.882594] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335035.896500] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335035.909193] [<ffffffffa028f85e>] ? NF_HOOK.constprop.8+0x3c/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335035.923099] [<ffffffffa028f9f2>] ? br_forward+0x16/0x5a [bridge]
May 18 11:24:06 172.30.1.2 [335035.935446] [<ffffffffa029051b>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
May 18 11:24:06 172.30.1.2 [335035.950235] [<ffffffffa02945ff>] ? br_nf_pre_routing_finish+0x1d0/0x1dd [bridge]
May 18 11:24:06 172.30.1.2 [335035.965371] [<ffffffffa0293ff0>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
May 18 11:24:06 172.30.1.2 [335035.978410] [<ffffffffa0294f58>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
May 18 11:24:06 172.30.1.2 [335035.992315] [<ffffffff812ac039>] ? nf_iterate+0x41/0x77
May 18 11:24:06 172.30.1.2 [335036.003100] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.017004] [<ffffffff812ac0d7>] ? nf_hook_slow+0x68/0x101
May 18 11:24:06 172.30.1.2 [335036.028309] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.042216] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.056123] [<ffffffffa0290360>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.070031] [<ffffffffa029073c>] ? br_handle_frame+0x1b3/0x1cb [bridge]
May 18 11:24:06 172.30.1.2 [335036.083591] [<ffffffffa0290589>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
May 18 11:24:06 172.30.1.2 [335036.098382] [<ffffffff812892c0>] ? __netif_receive_skb+0x324/0x41f
May 18 11:24:06 172.30.1.2 [335036.111071] [<ffffffff81289427>] ? process_backlog+0x6c/0x123
May 18 11:24:06 172.30.1.2 [335036.122896] [<ffffffff8128b30d>] ? net_rx_action+0xa1/0x1af
May 18 11:24:06 172.30.1.2 [335036.134374] [<ffffffff8104b298>] ? __local_bh_enable+0x40/0x77
May 18 11:24:06 172.30.1.2 [335036.146371] [<ffffffff8104be30>] ? __do_softirq+0xb9/0x177
May 18 11:24:06 172.30.1.2 [335036.157678] [<ffffffff8135046c>] ? call_softirq+0x1c/0x30
May 18 11:24:06 172.30.1.2 [335036.168805] <EOI>
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 [335036.173187] [<ffffffff8100f8e5>] ? do_softirq+0x3c/0x7b
May 18 11:24:06 172.30.1.2 [335036.183969] [<ffffffff8128b5fd>] ? netif_rx_ni+0x1e/0x27
May 18 11:24:06 172.30.1.2 [335036.194927] [<ffffffffa02a1721>] ? tun_get_user+0x39a/0x3c2 [tun]
May 18 11:24:06 172.30.1.2 [335036.207445] [<ffffffffa02a1a66>] ? tun_chr_poll+0xcd/0xcd [tun]
May 18 11:24:06 172.30.1.2 [335036.219615] [<ffffffffa02a1ac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
May 18 11:24:06 172.30.1.2 [335036.232656] [<ffffffff810f9594>] ? do_sync_readv_writev+0x9a/0xd7
May 18 11:24:06 172.30.1.2 [335036.245174] [<ffffffff810363c7>] ? should_resched+0x5/0x23
May 18 11:24:06 172.30.1.2 [335036.256478] [<ffffffff810f8c16>] ? do_sync_read+0xab/0xe3
May 18 11:24:06 172.30.1.2 [335036.267608] [<ffffffff810363c7>] ? should_resched+0x5/0x23
May 18 11:24:06 172.30.1.2 [335036.278915] [<ffffffff811626a1>] ? security_file_permission+0x16/0x2d
May 18 11:24:06 172.30.1.2 [335036.292124] [<ffffffff810f97f8>] ? do_readv_writev+0xaf/0x11c
May 18 11:24:06 172.30.1.2 [335036.303950] [<ffffffff8112ab7e>] ? eventfd_ctx_read+0x162/0x174
May 18 11:24:06 172.30.1.2 [335036.316124] [<ffffffff8103f3ff>] ? try_to_wake_up+0x197/0x197
May 18 11:24:06 172.30.1.2 [335036.327949] [<ffffffff810f99cd>] ? sys_writev+0x45/0x90
May 18 11:24:06 172.30.1.2 [335036.338733] [<ffffffff8134e212>] ? system_call_fastpath+0x16/0x1b
May 18 11:24:06 172.30.1.2 [335036.351249] Code:
May 18 11:24:06 172.30.1.2 53
May 18 11:24:06 172.30.1.2 48
May 18 11:24:06 172.30.1.2 89
May 18 11:24:06 172.30.1.2 fb
May 18 11:24:06 172.30.1.2 48
May 18 11:24:06 172.30.1.2 83
May 18 11:24:06 172.30.1.2 ec
May 18 11:24:06 172.30.1.2 10
May 18 11:24:06 172.30.1.2 66
May 18 11:24:06 172.30.1.2 81
May 18 11:24:06 172.30.1.2 7f
May 18 11:24:06 172.30.1.2 7e
May 18 11:24:06 172.30.1.2 08
May 18 11:24:06 172.30.1.2 06
May 18 11:24:06 172.30.1.2 4c
May 18 11:24:06 172.30.1.2 8b
May 18 11:24:06 172.30.1.2 a7
May 18 11:24:06 172.30.1.2 98
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 74
May 18 11:24:06 172.30.1.2 3d
May 18 11:24:06 172.30.1.2 e8
May 18 11:24:06 172.30.1.2 07
May 18 11:24:06 172.30.1.2 fe
May 18 11:24:06 172.30.1.2 ff
May 18 11:24:06 172.30.1.2 ff
May 18 11:24:06 172.30.1.2 66
May 18 11:24:06 172.30.1.2 3d
May 18 11:24:06 172.30.1.2 08
May 18 11:24:06 172.30.1.2 06
May 18 11:24:06 172.30.1.2 75
May 18 11:24:06 172.30.1.2 09
May 18 11:24:06 172.30.1.2 83
May 18 11:24:06 172.30.1.2 3d
May 18 11:24:06 172.30.1.2 98
May 18 11:24:06 172.30.1.2 6a
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 00
May 18 11:24:06 172.30.1.2 75
May 18 11:24:06 172.30.1.2 29
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 f6
May 18 11:24:06 172.30.1.2 44
May 18 11:24:06 172.30.1.2 24
May 18 11:24:06 172.30.1.2 18
May 18 11:24:06 172.30.1.2 01
May 18 11:24:06 172.30.1.2 49
May 18 11:24:06 172.30.1.2 8b
May 18 11:24:06 172.30.1.2 6c
May 18 11:24:06 172.30.1.2 24
May 18 11:24:06 172.30.1.2 08
May 18 11:24:06 172.30.1.2 74
May 18 11:24:06 172.30.1.2 12
May 18 11:24:06 172.30.1.2 8a
May 18 11:24:06 172.30.1.2 43
May 18 11:24:06 172.30.1.2 7d
May 18 11:24:06 172.30.1.2 83
May 18 11:24:06 172.30.1.2 e0
May 18 11:24:06 172.30.1.2 f8
May 18 11:24:06 172.30.1.2 83
May 18 11:24:06 172.30.1.2 c8
May 18 11:24:06 172.30.1.2
May 18 11:24:06 172.30.1.2 [335036.390118] RIP
May 18 11:24:06 172.30.1.2 [335036.404573] RSP <ffff88042fc03b18>
May 18 11:24:06 172.30.1.2 [335036.411711] CR2: 0000000000000018
May 18 11:24:06 172.30.1.2 [335036.418997] ---[ end trace 3272a02392487fe9 ]---
May 18 11:24:06 172.30.1.2 [335036.428496] Kernel panic - not syncing: Fatal exception in interrupt
May 18 11:24:06 172.30.1.2 [335036.441519] Pid: 4321, comm: kvm Tainted: G D 3.2.0-2-amd64 #1
May 18 11:24:06 172.30.1.2 [335036.455226] Call Trace:
May 18 11:24:06 172.30.1.2 [335036.460431] <IRQ>
May 18 11:24:06 172.30.1.2 [335036.472087] [<ffffffff8134a086>] ? oops_end+0xa9/0xb6
May 18 11:24:06 172.30.1.2 [335036.482697] [<ffffffff81342487>] ? no_context+0x1ff/0x20e
May 18 11:24:06 172.30.1.2 [335036.494029] [<ffffffff810e9c30>] ? virt_to_slab+0x6/0x16
May 18 11:24:06 172.30.1.2 [335036.505173] [<ffffffff8134c099>] ? do_page_fault+0x1a8/0x337
May 18 11:24:06 172.30.1.2 [335036.517001] [<ffffffffa0398f06>] ? ip_vs_conn_put+0x28/0x32 [ip_vs]
May 18 11:24:06 172.30.1.2 [335036.530050] [<ffffffffa039b0e0>] ? ip_vs_out+0x2bd/0x432 [ip_vs]
May 18 11:24:06 172.30.1.2 [335036.542621] [<ffffffff812ac0d7>] ? nf_hook_slow+0x68/0x101
May 18 11:24:06 172.30.1.2 [335036.554228] [<ffffffff813497f5>] ? page_fault+0x25/0x30
May 18 11:24:06 172.30.1.2 [335036.565318] [<ffffffffa0294308>] ? nf_bridge_update_protocol+0x20/0x20 [bridge]
May 18 11:24:06 172.30.1.2 [335036.580577] [<ffffffffa0294336>] ? br_nf_forward_finish+0x2e/0x95 [bridge]
May 18 11:24:06 172.30.1.2 [335036.594754] [<ffffffffa0294327>] ? br_nf_forward_finish+0x1f/0x95 [bridge]
May 18 11:24:06 172.30.1.2 [335036.608979] [<ffffffffa02946db>] ? br_parse_ip_options+0x3d/0x19a [bridge]
May 18 11:24:06 172.30.1.2 [335036.623296] [<ffffffffa0294a67>] ? br_nf_forward_ip+0x1c0/0x1d4 [bridge]
May 18 11:24:06 172.30.1.2 [335036.637308] [<ffffffff812ac039>] ? nf_iterate+0x41/0x77
May 18 11:24:06 172.30.1.2 [335036.648326] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335036.661291] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335036.674283] [<ffffffff812ac0d7>] ? nf_hook_slow+0x68/0x101
May 18 11:24:06 172.30.1.2 [335036.685892] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335036.698883] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.713091] [<ffffffffa028f918>] ? __br_deliver+0xa0/0xa0 [bridge]
May 18 11:24:06 172.30.1.2 [335036.726088] [<ffffffffa028f85e>] ? NF_HOOK.constprop.8+0x3c/0x56 [bridge]
May 18 11:24:06 172.30.1.2 [335036.740298] [<ffffffffa028f9f2>] ? br_forward+0x16/0x5a [bridge]
May 18 11:24:07 172.30.1.2 [335036.752942] [<ffffffffa029051b>] ? br_handle_frame_finish+0x1a1/0x20f [bridge]
May 18 11:24:07 172.30.1.2 [335036.767996] [<ffffffffa02945ff>] ? br_nf_pre_routing_finish+0x1d0/0x1dd [bridge]
May 18 11:24:07 172.30.1.2 [335036.783410] [<ffffffffa0293ff0>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
May 18 11:24:07 172.30.1.2 [335036.796757] [<ffffffffa0294f58>] ? br_nf_pre_routing+0x3e8/0x3f5 [bridge]
May 18 11:24:07 172.30.1.2 [335036.810959] [<ffffffff812ac039>] ? nf_iterate+0x41/0x77
May 18 11:24:07 172.30.1.2 [335036.822043] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:07 172.30.1.2 [335036.836248] [<ffffffff812ac0d7>] ? nf_hook_slow+0x68/0x101
May 18 11:24:07 172.30.1.2 [335036.847849] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:07 172.30.1.2 [335036.862053] [<ffffffffa029037a>] ? NF_HOOK.constprop.4+0x56/0x56 [bridge]
May 18 11:24:07 172.30.1.2 [335036.876264] [<ffffffffa0290360>] ? NF_HOOK.constprop.4+0x3c/0x56 [bridge]
May 18 11:24:07 172.30.1.2 [335036.890457] [<ffffffffa029073c>] ? br_handle_frame+0x1b3/0x1cb [bridge]
May 18 11:24:07 172.30.1.2 [335036.904319] [<ffffffffa0290589>] ? br_handle_frame_finish+0x20f/0x20f [bridge]
May 18 11:24:07 172.30.1.2 [335036.919409] [<ffffffff812892c0>] ? __netif_receive_skb+0x324/0x41f
May 18 11:24:07 172.30.1.2 [335036.932395] [<ffffffff81289427>] ? process_backlog+0x6c/0x123
May 18 11:24:07 172.30.1.2 [335036.944520] [<ffffffff8128b30d>] ? net_rx_action+0xa1/0x1af
May 18 11:24:07 172.30.1.2 [335036.956298] [<ffffffff8104b298>] ? __local_bh_enable+0x40/0x77
May 18 11:24:07 172.30.1.2 [335036.968594] [<ffffffff8104be30>] ? __do_softirq+0xb9/0x177
May 18 11:24:07 172.30.1.2 [335036.980197] [<ffffffff8135046c>] ? call_softirq+0x1c/0x30
May 18 11:24:07 172.30.1.2 [335036.991624] <EOI>
May 18 11:24:07 172.30.1.2 [335037.004236] [<ffffffff8128b5fd>] ? netif_rx_ni+0x1e/0x27
May 18 11:24:07 172.30.1.2 [335037.015493] [<ffffffffa02a1721>] ? tun_get_user+0x39a/0x3c2 [tun]
May 18 11:24:07 172.30.1.2 [335037.028308] [<ffffffffa02a1a66>] ? tun_chr_poll+0xcd/0xcd [tun]
May 18 11:24:07 172.30.1.2 [335037.040765] [<ffffffffa02a1ac4>] ? tun_chr_aio_write+0x5e/0x79 [tun]
May 18 11:24:07 172.30.1.2 [335037.054111] [<ffffffff810f9594>] ? do_sync_readv_writev+0x9a/0xd7
May 18 11:24:07 172.30.1.2 [335037.066925] [<ffffffff810363c7>] ? should_resched+0x5/0x23
May 18 11:24:07 172.30.1.2 [335037.078534] [<ffffffff810f8c16>] ? do_sync_read+0xab/0xe3
May 18 11:24:07 172.30.1.2 [335037.089957] [<ffffffff810363c7>] ? should_resched+0x5/0x23
May 18 11:24:07 172.30.1.2 [335037.101560] [<ffffffff811626a1>] ? security_file_permission+0x16/0x2d
May 18 11:24:07 172.30.1.2 [335037.115072] [<ffffffff810f97f8>] ? do_readv_writev+0xaf/0x11c
May 18 11:24:07 172.30.1.2 [335037.127196] [<ffffffff8112ab7e>] ? eventfd_ctx_read+0x162/0x174
May 18 11:24:07 172.30.1.2 [335037.139673] [<ffffffff8103f3ff>] ? try_to_wake_up+0x197/0x197
May 18 11:24:07 172.30.1.2 [335037.151782] [<ffffffff810f99cd>] ? sys_writev+0x45/0x90
May 18 11:24:07 172.30.1.2 [335037.162870] [<ffffffff8134e212>] ? system_call_fastpath+0x16/0x1b
^ permalink raw reply
* Regarding: Bug 43242 - Kconfig: Unable to disable Broadcom and Chelsio devices
From: Paul Menzel @ 2012-05-18 11:37 UTC (permalink / raw)
To: netdev
[-- Attachment #1: Type: text/plain, Size: 250 bytes --]
Dear netdev folks,
I created the ticket for bug #43242 [1]. Is that a Kconfig problem or a
problem with the Kconfig files in the Broadcom and Chelsio directories?
Thanks,
Paul
[1] https://bugzilla.kernel.org/show_bug.cgi?id=43242
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply
* e1000 rx emulation bug (Was: [PATCH] e1000: Reset rx ring index on receive overrun)
From: Samuel Thibault @ 2012-05-18 13:51 UTC (permalink / raw)
To: Brandeburg, Jesse, qemu-devel, dlaor
Cc: Jiri Pirko, e1000-devel@lists.sourceforge.net, Dean Nelson,
Allan, Bruce W, linux-kernel@vger.kernel.org, David S. Miller,
Ronciak, John, netdev@vger.kernel.org
In-Reply-To: <alpine.WNT.2.00.1205171718160.7900@jbrandeb-mobl2.amr.corp.intel.com>
Hello,
There seems to be a bug in qemu in the e1000 emulation, which triggers
an issue with the Linux driver.
What happens in Linux is the following:
- e1000_open
- e1000_configure
- e1000_setup_rctl
- enables E1000_RCTL_EN
- e1000_configure_rx
- sets RDT/RDH to 0
- alloc_rx_buf
- pushes buffers to the ring
with bad luck, or on high traffic of small packets, what is observed is
that between setting RDT/RDH and pushing buffers, the ring fills up in
qemu. Here is what happens there on the qemu side:
- e1000_receive
- e1000_has_rxbufs
- total_size <= s->rxbuf_size (because it's small)
return s->mac_reg[RDH] != s->mac_reg[RDT] || !s->check_rxov;
although RDH == RDT == 0, it returns 1, because since RDT/RDH have
just been set to 0, set_rdt has cleared check_rxov. e1000_receive
thus believes there is room, and proceeds with filling the ring.
Unfortunately, since no buffer was pushed, desc.buffer_addr is NULL, and
thus the do loop skips all these nul rx descriptors of the ring, but
marking each of them with E1000_RXD_STAT_DD, and eventually wrapping
around. From then on, since check_rxov has been set by the do loop,
nothing more is pushed, until the linux driver pushes buffers to the
ring. qemu can then fill some descriptors, and Linux read them, but
since the whole ring was filled with E1000_RXD_STAT_DD, Linux goes on
reading, and thus gets completely desynchronized with the device.
That raises two questions:
- what is the role of the check_rxov flag? Is hardware really allowed
to push in some cases, even when RDH==RDT? Removing it makes things
work just fine.
- BTW, when skipping a descriptor because of NULL address, does
E1000_RXD_STAT_DD have to be set?
Samuel
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
E1000-devel mailing list
E1000-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/e1000-devel
To learn more about Intel® Ethernet, visit http://communities.intel.com/community/wired
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox