* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: David Miller @ 2011-10-06 20:00 UTC (permalink / raw)
To: Ian.Campbell; +Cc: mirqus, netdev, jdmason
In-Reply-To: <1317916822.21903.252.camel@zakaz.uk.xensource.com>
From: Ian Campbell <Ian.Campbell@citrix.com>
Date: Thu, 6 Oct 2011 17:00:22 +0100
> Here it is. David, if you want N separate patches (or a git pull
> request?) let me know.
No, this is fine, applied.
Thanks.
^ permalink raw reply
* [GIT] Networking
From: David Miller @ 2011-10-06 20:23 UTC (permalink / raw)
To: torvalds; +Cc: akpm, netdev, linux-kernel
1) MD5 signature pool leak fix in TCP, from Zheng Yan.
2) Fix carrier state regression in bridging, from Stephen Hemminger.
3) Unicast forwards between macvtap interfaces should use
dev_forward_skb() not the vlan->forward() method. Fix from David
Ward.
4) TCP's lost_cnt_hint is updated one too many times in some situations,
fix from Zheng Yan.
5) netfilter needs to use rwlock_init(), from Thomas Gleixner.
Please pull, thanks a lot!
The following changes since commit 6367f1775ebb66b0f0e9e3512159f3257a6fde0e:
Merge branch 'for-linus' of http://people.redhat.com/agk/git/linux-dm (2011-10-06 08:31:47 -0700)
are available in the git repository at:
git://github.com/davem330/net.git master
Benjamin Poirier (1):
net: fix typos in Documentation/networking/scaling.txt
David Ward (1):
macvlan/macvtap: Fix unicast between macvtap interfaces in bridge mode
Thomas Gleixner (1):
netfilter: Use proper rwlock init function
Yan, Zheng (2):
tcp: properly handle md5sig_pool references
tcp: properly update lost_cnt_hint during shifting
stephen hemminger (1):
bridge: leave carrier on for empty bridge
Documentation/networking/scaling.txt | 10 +++++-----
drivers/net/macvlan.c | 2 +-
net/bridge/br_device.c | 3 ---
net/ipv4/tcp_input.c | 4 +---
net/ipv4/tcp_ipv4.c | 11 +++++++----
net/ipv6/tcp_ipv6.c | 8 +++++---
net/netfilter/ipvs/ip_vs_ctl.c | 2 +-
7 files changed, 20 insertions(+), 20 deletions(-)
^ permalink raw reply
* [PATCH net-next] macvlan: handle fragmented multicast frames
From: Eric Dumazet @ 2011-10-06 20:28 UTC (permalink / raw)
To: Ben Greear; +Cc: netdev
In-Reply-To: <4E8CDB9B.6010900@candelatech.com>
Le mercredi 05 octobre 2011 à 15:35 -0700, Ben Greear a écrit :
> If someone wants to cook up macvlan-ip-defrag patch I'll be happy
> to test it. But, as far as I can tell, this problem can happen on
> any two interfaces. The reason that some of mine work (.1q vlans)
> and macvlan didn't is probably because those were separated by
> some virtual network links that imparted extra delay...so the
> vlan consumed all its fragments and passed the complete pkt up
> the stack before the mac-vlan ever saw the initial frame.
>
> With this in mind, it seems that using multiple udp multicast
> sockets bound to specific devices is fundamentally broken for
> fragmented packets.
>
> I have no pressing need for this feature, so now that I better understand
> the problem I can just document it and move on to other things.
>
> Thanks for all the help.
>
Please test following patch (note I had no time to test it, sorry !)
Based on net-next tree, might apply on 3.0 kernel...
[PATCH net-next] macvlan: handle fragmented multicast frames
Fragmented multicast frames are delivered to a single macvlan port,
because ip defrag logic considers other samples are redundant.
Implement a defrag step before trying to send the multicast frame.
Reported-by: Ben Greear <greearb@candelatech.com>
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
drivers/net/macvlan.c | 3 +++
include/net/ip.h | 9 +++++++++
net/ipv4/ip_fragment.c | 36 ++++++++++++++++++++++++++++++++++++
net/packet/af_packet.c | 39 +--------------------------------------
diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index b100c90..40366eb 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -169,6 +169,9 @@ static rx_handler_result_t macvlan_handle_frame(struct sk_buff **pskb)
port = macvlan_port_get_rcu(skb->dev);
if (is_multicast_ether_addr(eth->h_dest)) {
+ skb = ip_check_defrag(skb, IP_DEFRAG_MACVLAN);
+ if (!skb)
+ return RX_HANDLER_CONSUMED;
src = macvlan_hash_lookup(port, eth->h_source);
if (!src)
/* frame comes from an external address */
diff --git a/include/net/ip.h b/include/net/ip.h
index aa76c7a..c7e066a 100644
--- a/include/net/ip.h
+++ b/include/net/ip.h
@@ -406,9 +406,18 @@ enum ip_defrag_users {
IP_DEFRAG_VS_OUT,
IP_DEFRAG_VS_FWD,
IP_DEFRAG_AF_PACKET,
+ IP_DEFRAG_MACVLAN,
};
int ip_defrag(struct sk_buff *skb, u32 user);
+#ifdef CONFIG_INET
+struct sk_buff *ip_check_defrag(struct sk_buff *skb, u32 user);
+#else
+static inline struct sk_buff *ip_check_defrag(struct sk_buff *skb, u32 user)
+{
+ return skb;
+}
+#endif
int ip_frag_mem(struct net *net);
int ip_frag_nqueues(struct net *net);
diff --git a/net/ipv4/ip_fragment.c b/net/ipv4/ip_fragment.c
index 0e0ab98..763589a 100644
--- a/net/ipv4/ip_fragment.c
+++ b/net/ipv4/ip_fragment.c
@@ -682,6 +682,42 @@ int ip_defrag(struct sk_buff *skb, u32 user)
}
EXPORT_SYMBOL(ip_defrag);
+struct sk_buff *ip_check_defrag(struct sk_buff *skb, u32 user)
+{
+ const struct iphdr *iph;
+ u32 len;
+
+ if (skb->protocol != htons(ETH_P_IP))
+ return skb;
+
+ if (!pskb_may_pull(skb, sizeof(struct iphdr)))
+ return skb;
+
+ iph = ip_hdr(skb);
+ if (iph->ihl < 5 || iph->version != 4)
+ return skb;
+ if (!pskb_may_pull(skb, iph->ihl*4))
+ return skb;
+ iph = ip_hdr(skb);
+ len = ntohs(iph->tot_len);
+ if (skb->len < len || len < (iph->ihl * 4))
+ return skb;
+
+ if (ip_is_fragment(ip_hdr(skb))) {
+ skb = skb_share_check(skb, GFP_ATOMIC);
+ if (skb) {
+ if (pskb_trim_rcsum(skb, len))
+ return skb;
+ memset(IPCB(skb), 0, sizeof(struct inet_skb_parm));
+ if (ip_defrag(skb, user))
+ return NULL;
+ skb->rxhash = 0;
+ }
+ }
+ return skb;
+}
+EXPORT_SYMBOL(ip_check_defrag);
+
#ifdef CONFIG_SYSCTL
static int zero;
diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c
index 25e68f5..ff9eed7 100644
--- a/net/packet/af_packet.c
+++ b/net/packet/af_packet.c
@@ -1213,43 +1213,6 @@ static struct sock *fanout_demux_cpu(struct packet_fanout *f, struct sk_buff *sk
return f->arr[cpu % num];
}
-static struct sk_buff *fanout_check_defrag(struct sk_buff *skb)
-{
-#ifdef CONFIG_INET
- const struct iphdr *iph;
- u32 len;
-
- if (skb->protocol != htons(ETH_P_IP))
- return skb;
-
- if (!pskb_may_pull(skb, sizeof(struct iphdr)))
- return skb;
-
- iph = ip_hdr(skb);
- if (iph->ihl < 5 || iph->version != 4)
- return skb;
- if (!pskb_may_pull(skb, iph->ihl*4))
- return skb;
- iph = ip_hdr(skb);
- len = ntohs(iph->tot_len);
- if (skb->len < len || len < (iph->ihl * 4))
- return skb;
-
- if (ip_is_fragment(ip_hdr(skb))) {
- skb = skb_share_check(skb, GFP_ATOMIC);
- if (skb) {
- if (pskb_trim_rcsum(skb, len))
- return skb;
- memset(IPCB(skb), 0, sizeof(struct inet_skb_parm));
- if (ip_defrag(skb, IP_DEFRAG_AF_PACKET))
- return NULL;
- skb->rxhash = 0;
- }
- }
-#endif
- return skb;
-}
-
static int packet_rcv_fanout(struct sk_buff *skb, struct net_device *dev,
struct packet_type *pt, struct net_device *orig_dev)
{
@@ -1268,7 +1231,7 @@ static int packet_rcv_fanout(struct sk_buff *skb, struct net_device *dev,
case PACKET_FANOUT_HASH:
default:
if (f->defrag) {
- skb = fanout_check_defrag(skb);
+ skb = ip_check_defrag(skb, IP_DEFRAG_AF_PACKET);
if (!skb)
return 0;
}
^ permalink raw reply related
* Re: IPv4 multicast and mac-vlans acting weird on 3.0.4+
From: Eric Dumazet @ 2011-10-06 20:42 UTC (permalink / raw)
To: Ben Greear; +Cc: netdev
In-Reply-To: <1317850603.3457.21.camel@edumazet-laptop>
Le mercredi 05 octobre 2011 à 23:36 +0200, Eric Dumazet a écrit :
> Le mercredi 05 octobre 2011 à 13:56 -0700, Ben Greear a écrit :
>
> > Wouldn't you have the same problem with two real Ethernet interfaces on
> > the same LAN, or two 802.1Q devices for that matter? The addrs will all
> > be the same in that case too?
> >
>
> Usually multicast is coupled with routing.
>
> A JOIN message from your app wont be sent on all interfaces...
>
> But yes, we might have a similar issue with regular vlans.
>
> Probably nobody noticed yet. Just say no to fragments :)
>
> > Also, if I have just a single mac-vlan active (the other 3 are 'ifconfig foo down'),
> > I still see the problem with mcast.
> >
>
> Thats another bug : macvlan doesnt test IFF_UP on broadcasts, only for
> unicast messages. Please test following patch.
>
> > From what you describe, I am thinking I may be hitting a different
> > issue. Any ideas on how to figure out why exactly the NF_HOOK isn't
> > calling the ip_rcv_finish method?
> >
>
> Really I believe I tried to explain the thing already...
>
> ip_local_deliver() -> ip_defrag() :
>
>
> [PATCH] macvlan: dont send frames on DOWN devices
>
> Reported-by: Ben Greear <greearb@candelatech.com>
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
> ---
> diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
> index b100c90..94a0282 100644
> --- a/drivers/net/macvlan.c
> +++ b/drivers/net/macvlan.c
> @@ -145,7 +145,8 @@ static void macvlan_broadcast(struct sk_buff *skb,
> hlist_for_each_entry_rcu(vlan, n, &port->vlan_hash[i], hlist) {
> if (vlan->dev == src || !(vlan->mode & mode))
> continue;
> -
> + if (!(vlan->dev->flags & IFF_UP))
> + continue;
> nskb = skb_clone(skb, GFP_ATOMIC);
> err = macvlan_broadcast_one(nskb, vlan, eth,
> mode == MACVLAN_MODE_BRIDGE);
>
This one is not needed.
When a port is down, its not in vlan_hash[] table anymore.
(Not sure why we perform the IFF_UP test for unicast frames.)
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: Michał Mirosław @ 2011-10-06 20:45 UTC (permalink / raw)
To: Ian Campbell; +Cc: David Miller, netdev@vger.kernel.org, Jon Mason
In-Reply-To: <1317916822.21903.252.camel@zakaz.uk.xensource.com>
W dniu 6 października 2011 18:00 użytkownik Ian Campbell
<Ian.Campbell@citrix.com> napisał:
> On Thu, 2011-10-06 at 08:05 +0100, Ian Campbell wrote:
>> On Wed, 2011-10-05 at 22:03 +0100, Michał Mirosław wrote:
>> > 2011/10/5 Ian Campbell <ian.campbell@citrix.com>:
>> > [...]
>> > > --- a/drivers/net/ethernet/neterion/vxge/vxge-main.c
>> > > +++ b/drivers/net/ethernet/neterion/vxge/vxge-main.c
>> > > @@ -923,9 +923,9 @@ vxge_xmit(struct sk_buff *skb, struct net_device *dev)
>> > > if (!frag->size)
>> > > continue;
>> > >
>> > > - dma_pointer = (u64) pci_map_page(fifo->pdev, frag->page,
>> > > - frag->page_offset, frag->size,
>> > > - PCI_DMA_TODEVICE);
>> > > + dma_pointer = (u64)skb_frag_dma_map(&fifo->pdev->dev, frag,
>> > > + 0, frag->size,
>> > > + PCI_DMA_TODEVICE);
>> >
>> > This should be DMA_TO_DEVICE instead of PCI_DMA_TODEVICE.
>> > > if (unlikely(pci_dma_mapping_error(fifo->pdev, dma_pointer)))
>> > > goto _exit2;
>> > I would also change this to dma_mapping_error() in one go.
>> > Just a random patch check.
>> Thanks Michał.
>> I'm sure I must have made the same mistakes in a whole bunch of patches
>> which have already been applied. I'll knock up a fixup patch.
> Here it is. David, if you want N separate patches (or a git pull
> request?) let me know.
There's a catch there, though:
[...]
> mapping = skb_frag_dma_map(&tp->pdev->dev, frag, 0,
> - len, PCI_DMA_TODEVICE);
> + len, DMA_TO_DEVICE);
>
> tnapi->tx_buffers[entry].skb = NULL;
> dma_unmap_addr_set(&tnapi->tx_buffers[entry], mapping,
> mapping);
> - if (pci_dma_mapping_error(tp->pdev, mapping))
> + if (dma_mapping_error(tp->pdev, mapping))
dma_mapping_error() takes struct device *, so those changes should be:
dma_mapping_error(&tp->pdev->dev, mapping)
(Like skb_frag_dma_map()'s first argument).
Best Regards,
Michał Mirosław
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: Ian Campbell @ 2011-10-06 21:08 UTC (permalink / raw)
To: Michał Mirosław; +Cc: David Miller, netdev@vger.kernel.org, Jon Mason
In-Reply-To: <CAHXqBFJsGcAdH0zFS_Gd34oR3Ov6ssbXRMzK-KrnSxq8c3=WNg@mail.gmail.com>
On Thu, 2011-10-06 at 21:45 +0100, Michał Mirosław wrote:
> W dniu 6 października 2011 18:00 użytkownik Ian Campbell
> <Ian.Campbell@citrix.com> napisał:
> > On Thu, 2011-10-06 at 08:05 +0100, Ian Campbell wrote:
> >> On Wed, 2011-10-05 at 22:03 +0100, Michał Mirosław wrote:
> >> > 2011/10/5 Ian Campbell <ian.campbell@citrix.com>:
> >> > [...]
> >> > > --- a/drivers/net/ethernet/neterion/vxge/vxge-main.c
> >> > > +++ b/drivers/net/ethernet/neterion/vxge/vxge-main.c
> >> > > @@ -923,9 +923,9 @@ vxge_xmit(struct sk_buff *skb, struct net_device *dev)
> >> > > if (!frag->size)
> >> > > continue;
> >> > >
> >> > > - dma_pointer = (u64) pci_map_page(fifo->pdev, frag->page,
> >> > > - frag->page_offset, frag->size,
> >> > > - PCI_DMA_TODEVICE);
> >> > > + dma_pointer = (u64)skb_frag_dma_map(&fifo->pdev->dev, frag,
> >> > > + 0, frag->size,
> >> > > + PCI_DMA_TODEVICE);
> >> >
> >> > This should be DMA_TO_DEVICE instead of PCI_DMA_TODEVICE.
> >> > > if (unlikely(pci_dma_mapping_error(fifo->pdev, dma_pointer)))
> >> > > goto _exit2;
> >> > I would also change this to dma_mapping_error() in one go.
> >> > Just a random patch check.
> >> Thanks Michał.
> >> I'm sure I must have made the same mistakes in a whole bunch of patches
> >> which have already been applied. I'll knock up a fixup patch.
> > Here it is. David, if you want N separate patches (or a git pull
> > request?) let me know.
>
> There's a catch there, though:
>
> [...]
> > mapping = skb_frag_dma_map(&tp->pdev->dev, frag, 0,
> > - len, PCI_DMA_TODEVICE);
> > + len, DMA_TO_DEVICE);
> >
> > tnapi->tx_buffers[entry].skb = NULL;
> > dma_unmap_addr_set(&tnapi->tx_buffers[entry], mapping,
> > mapping);
> > - if (pci_dma_mapping_error(tp->pdev, mapping))
> > + if (dma_mapping_error(tp->pdev, mapping))
>
> dma_mapping_error() takes struct device *, so those changes should be:
>
> dma_mapping_error(&tp->pdev->dev, mapping)
>
> (Like skb_frag_dma_map()'s first argument).
You are absolutely right, I've no idea how I missed the very obvious
warning this produces. Incremental patch is below, sorry about this!
8<-------------------------------------------------------
From 5be2edc6eec5c66b58f4287f1d3ba3637afa7ad6 Mon Sep 17 00:00:00 2001
From: Ian Campbell <ian.campbell@citrix.com>
Date: Thu, 6 Oct 2011 22:05:41 +0100
Subject: [PATCH] net: fix argument to dma_mapping_error after conversion to skb_frag_dma_map
The recent conversion from pci_dma_mapping_error to dma_mapping_error missed
the change in the exact parameter, which needs to be the struct device * not
the struct pci_device *.
Signed-off-by: Ian Campbell <ian.campbell@citrix.com>
---
drivers/net/ethernet/broadcom/tg3.c | 2 +-
drivers/net/ethernet/marvell/sky2.c | 4 ++--
drivers/net/ethernet/pasemi/pasemi_mac.c | 2 +-
.../net/ethernet/qlogic/netxen/netxen_nic_main.c | 2 +-
drivers/net/ethernet/qlogic/qla3xxx.c | 2 +-
drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c | 2 +-
drivers/net/ethernet/qlogic/qlge/qlge_main.c | 2 +-
drivers/net/ethernet/sfc/tx.c | 2 +-
8 files changed, 9 insertions(+), 9 deletions(-)
diff --git a/drivers/net/ethernet/broadcom/tg3.c b/drivers/net/ethernet/broadcom/tg3.c
index 3abcb4d..9dbd1af 100644
--- a/drivers/net/ethernet/broadcom/tg3.c
+++ b/drivers/net/ethernet/broadcom/tg3.c
@@ -6784,7 +6784,7 @@ static netdev_tx_t tg3_start_xmit(struct sk_buff *skb, struct net_device *dev)
tnapi->tx_buffers[entry].skb = NULL;
dma_unmap_addr_set(&tnapi->tx_buffers[entry], mapping,
mapping);
- if (dma_mapping_error(tp->pdev, mapping))
+ if (dma_mapping_error(&tp->pdev->dev, mapping))
goto dma_error;
if (tg3_tx_frag_set(tnapi, &entry, &budget, mapping,
diff --git a/drivers/net/ethernet/marvell/sky2.c b/drivers/net/ethernet/marvell/sky2.c
index 7baff3e..a3ce9b6 100644
--- a/drivers/net/ethernet/marvell/sky2.c
+++ b/drivers/net/ethernet/marvell/sky2.c
@@ -1231,7 +1231,7 @@ static int sky2_rx_map_skb(struct pci_dev *pdev, struct rx_ring_info *re,
frag->size,
DMA_FROM_DEVICE);
- if (dma_mapping_error(pdev, re->frag_addr[i]))
+ if (dma_mapping_error(&pdev->dev, re->frag_addr[i]))
goto map_page_error;
}
return 0;
@@ -1938,7 +1938,7 @@ static netdev_tx_t sky2_xmit_frame(struct sk_buff *skb,
mapping = skb_frag_dma_map(&hw->pdev->dev, frag, 0,
frag->size, DMA_TO_DEVICE);
- if (dma_mapping_error(hw->pdev, mapping))
+ if (dma_mapping_error(&hw->pdev->dev, mapping))
goto mapping_unwind;
upper = upper_32_bits(mapping);
diff --git a/drivers/net/ethernet/pasemi/pasemi_mac.c b/drivers/net/ethernet/pasemi/pasemi_mac.c
index d247030..c6f0056 100644
--- a/drivers/net/ethernet/pasemi/pasemi_mac.c
+++ b/drivers/net/ethernet/pasemi/pasemi_mac.c
@@ -1508,7 +1508,7 @@ static int pasemi_mac_start_tx(struct sk_buff *skb, struct net_device *dev)
map[i + 1] = skb_frag_dma_map(&mac->dma_pdev->dev, frag, 0,
frag->size, DMA_TO_DEVICE);
map_size[i+1] = frag->size;
- if (dma_mapping_error(mac->dma_pdev, map[i + 1])) {
+ if (dma_mapping_error(&mac->dma_pdev->dev, map[i + 1])) {
nfrags = i;
goto out_err_nolock;
}
diff --git a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
index b061c07..e2ba78b 100644
--- a/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
+++ b/drivers/net/ethernet/qlogic/netxen/netxen_nic_main.c
@@ -1907,7 +1907,7 @@ netxen_map_tx_skb(struct pci_dev *pdev,
map = skb_frag_dma_map(&pdev->dev, frag, 0, frag->size,
DMA_TO_DEVICE);
- if (dma_mapping_error(pdev, map))
+ if (dma_mapping_error(&pdev->dev, map))
goto unwind;
nf->dma = map;
diff --git a/drivers/net/ethernet/qlogic/qla3xxx.c b/drivers/net/ethernet/qlogic/qla3xxx.c
index 8932265..46f9b64 100644
--- a/drivers/net/ethernet/qlogic/qla3xxx.c
+++ b/drivers/net/ethernet/qlogic/qla3xxx.c
@@ -2391,7 +2391,7 @@ static int ql_send_map(struct ql3_adapter *qdev,
map = skb_frag_dma_map(&qdev->pdev->dev, frag, 0, frag->size,
DMA_TO_DEVICE);
- err = dma_mapping_error(qdev->pdev, map);
+ err = dma_mapping_error(&qdev->pdev->dev, map);
if (err) {
netdev_err(qdev->ndev,
"PCI mapping frags failed with error: %d\n",
diff --git a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
index c9756e7..eac19e7d 100644
--- a/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
+++ b/drivers/net/ethernet/qlogic/qlcnic/qlcnic_main.c
@@ -2137,7 +2137,7 @@ qlcnic_map_tx_skb(struct pci_dev *pdev,
map = skb_frag_dma_map(&pdev->dev, frag, 0, frag->size,
DMA_TO_DEVICE);
- if (dma_mapping_error(pdev, map))
+ if (dma_mapping_error(&pdev->dev, map))
goto unwind;
nf->dma = map;
diff --git a/drivers/net/ethernet/qlogic/qlge/qlge_main.c b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
index 094ac22..f2d9bb7 100644
--- a/drivers/net/ethernet/qlogic/qlge/qlge_main.c
+++ b/drivers/net/ethernet/qlogic/qlge/qlge_main.c
@@ -1434,7 +1434,7 @@ static int ql_map_send(struct ql_adapter *qdev,
map = skb_frag_dma_map(&qdev->pdev->dev, frag, 0, frag->size,
DMA_TO_DEVICE);
- err = dma_mapping_error(qdev->pdev, map);
+ err = dma_mapping_error(&qdev->pdev->dev, map);
if (err) {
netif_err(qdev, tx_queued, qdev->ndev,
"PCI mapping frags failed with error: %d.\n",
diff --git a/drivers/net/ethernet/sfc/tx.c b/drivers/net/ethernet/sfc/tx.c
index 7f47efc..3964a62 100644
--- a/drivers/net/ethernet/sfc/tx.c
+++ b/drivers/net/ethernet/sfc/tx.c
@@ -927,7 +927,7 @@ static int tso_get_fragment(struct tso_state *st, struct efx_nic *efx,
{
st->unmap_addr = skb_frag_dma_map(&efx->pci_dev->dev, frag, 0,
frag->size, DMA_TO_DEVICE);
- if (likely(!dma_mapping_error(efx->pci_dev, st->unmap_addr))) {
+ if (likely(!dma_mapping_error(&efx->pci_dev->dev, st->unmap_addr))) {
st->unmap_single = false;
st->unmap_len = frag->size;
st->in_len = frag->size;
--
1.7.2.5
Ian.
^ permalink raw reply related
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: David Miller @ 2011-10-06 21:15 UTC (permalink / raw)
To: mirqus; +Cc: Ian.Campbell, netdev, jdmason
In-Reply-To: <CAHXqBFJsGcAdH0zFS_Gd34oR3Ov6ssbXRMzK-KrnSxq8c3=WNg@mail.gmail.com>
From: Michał Mirosław <mirqus@gmail.com>
Date: Thu, 6 Oct 2011 22:45:57 +0200
> There's a catch there, though:
..
> dma_mapping_error() takes struct device *, so those changes should be:
>
> dma_mapping_error(&tp->pdev->dev, mapping)
>
> (Like skb_frag_dma_map()'s first argument).
Why don't you take a look at what I committed and pushed out
(hint: I fixed it up when I saw the build warnings)?
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: David Miller @ 2011-10-06 21:16 UTC (permalink / raw)
To: Ian.Campbell; +Cc: mirqus, netdev, jdmason
In-Reply-To: <1317935310.24742.18.camel@dagon.hellion.org.uk>
From: Ian Campbell <Ian.Campbell@eu.citrix.com>
Date: Thu, 6 Oct 2011 22:08:30 +0100
> You are absolutely right, I've no idea how I missed the very obvious
> warning this produces. Incremental patch is below, sorry about this!
Would really love to know what your patch is against, since I
fixed this problem when I commited your original patch.
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: Ian Campbell @ 2011-10-06 21:19 UTC (permalink / raw)
To: David Miller; +Cc: mirqus@gmail.com, netdev@vger.kernel.org, jdmason@kudzu.us
In-Reply-To: <20111006.171519.797156146863050743.davem@davemloft.net>
On Thu, 2011-10-06 at 22:15 +0100, David Miller wrote:
> From: Michał Mirosław <mirqus@gmail.com>
> Date: Thu, 6 Oct 2011 22:45:57 +0200
>
> > There's a catch there, though:
> ..
> > dma_mapping_error() takes struct device *, so those changes should be:
> >
> > dma_mapping_error(&tp->pdev->dev, mapping)
> >
> > (Like skb_frag_dma_map()'s first argument).
>
> Why don't you take a look at what I committed and pushed out
> (hint: I fixed it up when I saw the build warnings)?
I pulled your tree and it wasn't there yet and it still isn't, I guess
it's delayed in mirroring?
Ian.
^ permalink raw reply
* [PATCH] bridge: fix hang on removal of bridge via netlink
From: Stephen Hemminger @ 2011-10-06 21:19 UTC (permalink / raw)
To: Sridhar Samudrala, David Miller; +Cc: netdev
In-Reply-To: <1317921532.6433.13.camel@w-sridhar.beaverton.ibm.com>
Need to cleanup bridge device timers and ports when being bridge
device is being removed via netlink.
This fixes the problem of observed when doing:
ip link add br0 type bridge
ip link set dev eth1 master br0
ip link set br0 up
ip link del br0
which would cause br0 to hang in unregister_netdev because
of leftover reference count.
Reported-by: Sridhar Samudrala <sri@us.ibm.com>
Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
---
Patch is against net-next but should go to net and stable trees
since it is an observable hang on 3.0 and later kernels.
--- a/net/bridge/br_if.c 2011-10-03 11:08:36.304168386 -0700
+++ b/net/bridge/br_if.c 2011-10-06 11:27:47.682488755 -0700
@@ -160,9 +160,10 @@ static void del_nbp(struct net_bridge_po
call_rcu(&p->rcu, destroy_nbp_rcu);
}
-/* called with RTNL */
-static void del_br(struct net_bridge *br, struct list_head *head)
+/* Delete bridge device */
+void br_dev_delete(struct net_device *dev, struct list_head *head)
{
+ struct net_bridge *br = netdev_priv(dev);
struct net_bridge_port *p, *n;
list_for_each_entry_safe(p, n, &br->port_list, list) {
@@ -267,7 +268,7 @@ int br_del_bridge(struct net *net, const
}
else
- del_br(netdev_priv(dev), NULL);
+ br_dev_delete(dev, NULL);
rtnl_unlock();
return ret;
@@ -446,7 +447,7 @@ void __net_exit br_net_exit(struct net *
rtnl_lock();
for_each_netdev(net, dev)
if (dev->priv_flags & IFF_EBRIDGE)
- del_br(netdev_priv(dev), &list);
+ br_dev_delete(dev, &list);
unregister_netdevice_many(&list);
rtnl_unlock();
--- a/net/bridge/br_netlink.c 2011-09-16 13:12:58.061369744 -0700
+++ b/net/bridge/br_netlink.c 2011-10-06 11:20:21.808911679 -0700
@@ -210,6 +210,7 @@ static struct rtnl_link_ops br_link_ops
.priv_size = sizeof(struct net_bridge),
.setup = br_dev_setup,
.validate = br_validate,
+ .dellink = br_dev_delete,
};
int __init br_netlink_init(void)
--- a/net/bridge/br_private.h 2011-10-06 08:42:27.353044954 -0700
+++ b/net/bridge/br_private.h 2011-10-06 11:25:17.845118817 -0700
@@ -301,6 +301,7 @@ static inline int br_is_root_bridge(cons
/* br_device.c */
extern void br_dev_setup(struct net_device *dev);
+extern void br_dev_delete(struct net_device *dev, struct list_head *list);
extern netdev_tx_t br_dev_xmit(struct sk_buff *skb,
struct net_device *dev);
#ifdef CONFIG_NET_POLL_CONTROLLER
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: Ian Campbell @ 2011-10-06 21:25 UTC (permalink / raw)
To: David Miller; +Cc: mirqus@gmail.com, netdev@vger.kernel.org, jdmason@kudzu.us
In-Reply-To: <20111006.171645.1154998655978830692.davem@davemloft.net>
On Thu, 2011-10-06 at 22:16 +0100, David Miller wrote:
> From: Ian Campbell <Ian.Campbell@eu.citrix.com>
> Date: Thu, 6 Oct 2011 22:08:30 +0100
>
> > You are absolutely right, I've no idea how I missed the very obvious
> > warning this produces. Incremental patch is below, sorry about this!
>
> Would really love to know what your patch is against, since I
> fixed this problem when I commited your original patch.
It was against e878d78b9a74 + the original bad patch, i.e.:
$ git log --pretty=oneline net-next/master^..HEAD
80f4b53b3d2c009689178d12de0bc108ddd580cd net: fix argument to dma_mapping_error after conversion to skb_frag_dma_map
669b1ce22eedd0f7bac048299feb06c67804ed83 net: use DMA_x_DEVICE and dma_mapping_error with skb_frag_dma_map
e878d78b9a7403fabc89ecc93c56928b74d14f01 virtio-net: Verify page list size before fitting into skb
AFAICT e878d78b9a7403fabc89ecc93c56928b74d14f01 is still the head of the
public git://github.com/davem330/net-next master.
Ian.
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: Michał Mirosław @ 2011-10-06 21:28 UTC (permalink / raw)
To: David Miller; +Cc: Ian.Campbell, netdev, jdmason
In-Reply-To: <20111006.171519.797156146863050743.davem@davemloft.net>
2011/10/6 David Miller <davem@davemloft.net>:
> From: Michał Mirosław <mirqus@gmail.com>
> Date: Thu, 6 Oct 2011 22:45:57 +0200
>> There's a catch there, though:
> ..
>> dma_mapping_error() takes struct device *, so those changes should be:
>>
>> dma_mapping_error(&tp->pdev->dev, mapping)
>>
>> (Like skb_frag_dma_map()'s first argument).
> Why don't you take a look at what I committed and pushed out
> (hint: I fixed it up when I saw the build warnings)?
You were just quicker than me or Ian on this one. We'll be faster next time.
Best Regards,
Michał Mirosław
^ permalink raw reply
* Re: [PATCH 1/8] vxge: convert to SKB paged frag API.
From: David Miller @ 2011-10-06 21:56 UTC (permalink / raw)
To: Ian.Campbell; +Cc: mirqus, netdev, jdmason
In-Reply-To: <1317936302.24742.23.camel@dagon.hellion.org.uk>
From: Ian Campbell <Ian.Campbell@eu.citrix.com>
Date: Thu, 6 Oct 2011 22:25:02 +0100
> On Thu, 2011-10-06 at 22:16 +0100, David Miller wrote:
>> From: Ian Campbell <Ian.Campbell@eu.citrix.com>
>> Date: Thu, 6 Oct 2011 22:08:30 +0100
>>
>> > You are absolutely right, I've no idea how I missed the very obvious
>> > warning this produces. Incremental patch is below, sorry about this!
>>
>> Would really love to know what your patch is against, since I
>> fixed this problem when I commited your original patch.
>
> It was against e878d78b9a74 + the original bad patch, i.e.:
>
> $ git log --pretty=oneline net-next/master^..HEAD
> 80f4b53b3d2c009689178d12de0bc108ddd580cd net: fix argument to dma_mapping_error after conversion to skb_frag_dma_map
> 669b1ce22eedd0f7bac048299feb06c67804ed83 net: use DMA_x_DEVICE and dma_mapping_error with skb_frag_dma_map
> e878d78b9a7403fabc89ecc93c56928b74d14f01 virtio-net: Verify page list size before fitting into skb
>
> AFAICT e878d78b9a7403fabc89ecc93c56928b74d14f01 is still the head of the
> public git://github.com/davem330/net-next master.
I'm and idiot, I didn't push it out when I left the office :-/
Sorry.
But when your commit appears it will have the dma_mapping_error() stuff
fixed up, so don't worry about it. :-)
^ permalink raw reply
* Re: [PATCH] IPv6: DAD from bonding iface is treated as dup address from others
From: Yinglin Sun @ 2011-10-06 22:17 UTC (permalink / raw)
To: Jay Vosburgh
Cc: Neil Horman, David S. Miller, Alexey Kuznetsov, James Morris,
Hideaki YOSHIFUJI, Patrick McHardy, netdev
In-Reply-To: <27199.1317927933@death>
On Thu, Oct 6, 2011 at 12:05 PM, Jay Vosburgh <fubar@us.ibm.com> wrote:
>
> Neil Horman <nhorman@tuxdriver.com> wrote:
>
> >On Wed, Oct 05, 2011 at 08:59:10PM -0700, Yinglin Sun wrote:
> >> Steps to reproduce this issue:
> >> 1. create bond0 over eth0 and eth1, set the mode to balance-xor
> >> 2. add an IPv6 address to bond0
> >> 3. DAD packet is sent out from one slave and then is looped back from
> >> the other slave. Therefore, it is treated as a duplicate address and
> >> stays tentative afterwards:
> >> kern.info:
> >> Oct 5 11:50:18 testvm1 kernel: [ 129.224353] bond0: IPv6 duplicate address 1234::1 detected!
> >>
> >> Signed-off-by: Yinglin Sun <Yinglin.Sun@emc.com>
> >> ---
> >> net/ipv6/ndisc.c | 15 +++++++++++++--
> >> 1 files changed, 13 insertions(+), 2 deletions(-)
> >>
> >> diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
> >> index 9da6e02..c82f4c7 100644
> >> --- a/net/ipv6/ndisc.c
> >> +++ b/net/ipv6/ndisc.c
> >> @@ -809,9 +809,10 @@ static void ndisc_recv_ns(struct sk_buff *skb)
> >>
> >> if (ifp->flags & (IFA_F_TENTATIVE|IFA_F_OPTIMISTIC)) {
> >> if (dad) {
> >> + const unsigned char *sadr;
> >> + sadr = skb_mac_header(skb);
> >> +
> >> if (dev->type == ARPHRD_IEEE802_TR) {
> >> - const unsigned char *sadr;
> >> - sadr = skb_mac_header(skb);
> >> if (((sadr[8] ^ dev->dev_addr[0]) & 0x7f) == 0 &&
> >> sadr[9] == dev->dev_addr[1] &&
> >> sadr[10] == dev->dev_addr[2] &&
> >> @@ -821,6 +822,16 @@ static void ndisc_recv_ns(struct sk_buff *skb)
> >> /* looped-back to us */
> >> goto out;
> >> }
> >> + } else if (dev->type == ARPHRD_ETHER) {
> >> + if (sadr[6] == dev->dev_addr[0] &&
> >> + sadr[7] == dev->dev_addr[1] &&
> >> + sadr[8] == dev->dev_addr[2] &&
> >> + sadr[9] == dev->dev_addr[3] &&
> >> + sadr[10] == dev->dev_addr[4] &&
> >> + sadr[11] == dev->dev_addr[5]) {
> >> + /* looped-back to us */
> >> + goto out;
> >> + }
> >> }
> >>
> >> /*
> >> --
> >> 1.7.4.1
> >>
> >Nack, This seems like it will just completely break DAD. What if theres another
> >system out there with the same mac address. A response from that system would
> >get dropped by this filter, instead of causing The local system to stop using
> >the address. What you really want to do is modify
> >bond_should_deliver_exact_match to detect this frame on the inactive slave or
> >some such, and drop the frame there.
>
> Also NACK; and adding a bit of information. The balance-xor
> mode is nominally expecting to interact with a switch whose ports are
> set for etherchannel ("static link aggregation"), in which case the
> switch will not loop the packet back around.
>
> If your switch can do etherchannel, then enable it and the
> problem should go away. If your switch cannot do this, then you may
> have other issues, because all of the multicast or broadcast packets
> going out any bonding slave will loop around to another slave. You
> could also use 802.3ad / LACP if you switch supports that.
>
> For balance-xor (or balance-rr, for that matter) mode to a
> non-etherchannel switch, it's going to be difficult, if not impossible,
> to modify bond_should_deliver_exact_match, because there are no inactive
> slaves. In this mode, bonding is expecting the switch to balance
> incoming traffic across the ports, and not deliver looped back packets
> or duplicates. There are no restrictions on what type of traffic
> (mcast, bcast, ucast) may arrive on any given port.
>
> I can't think of a way to make the non-etherchannel case work
> for balance-xor (or balance-rr) without breaking the DAD functionality
> in the case of an actual duplicate. I'm not aware of a way to
> distinguish a looped back DAD probe from an actual duplicate address
> probe elsewhere on the network.
>
Hi Neil & Jay,
Thanks a lot for the comments.
The use case is to add IPv6 address on the bonding interface first,
and then set up port channel on switch. We'll hit this issue and the
new address will stay tentative and unusable after port channel is set
up on switch. This patch is for this valid use case.
Except failover mode, all slaves are active on receiving packets, so
we are receiving such looped back DAD and the bonding driver cannot
ignore them. I cannot think of a way to distinguish if a DAD is looped
back or from someone else having the same mac address. They look the
same to the host. If there is another machine having the same mac
address, this code path gets executed if both are doing DAD at the
same time for the same IPv6 address. Maybe we should find out what the
specification defines for this case?
Thanks.
Yinglin
^ permalink raw reply
* Re: [PATCH] SELinux: Fix RCU deref check warning in sel_netport_insert()
From: Paul Moore @ 2011-10-06 22:51 UTC (permalink / raw)
To: David Howells; +Cc: selinux, netdev
In-Reply-To: <1624.1317821523@redhat.com>
On Wednesday, October 05, 2011 02:32:03 PM David Howells wrote:
> Paul Moore <paul@paul-moore.com> wrote:
> > We should probably do the same for the security/selinux/netif.c as it
> > uses the same logic; David is this something you want to tackle?
>
> netif.c doesn't use any rcu_dereference*() function directly, though it does
> use list_for_each_entry_rcu(). However, I'm not sure that's a problem.
> What is it you're referring to?
My apologies, the netport.c and netif.c code is very, very similar and
whenever I see a patch just for one of the two it causes a reaction that you
saw above. While netif.c has a similar function, sel_netif_insert(), it is
slightly different and doesn't need a rcu_dereference() ad the netport.c code
does.
Sorry for the confusion.
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: [PATCH] SELinux: Fix RCU deref check warning in sel_netport_insert() [ver #3]
From: Paul Moore @ 2011-10-06 22:52 UTC (permalink / raw)
To: David Howells; +Cc: selinux, netdev
In-Reply-To: <20111005111919.30551.77529.stgit@warthog.procyon.org.uk>
On Wednesday, October 05, 2011 12:19:19 PM David Howells wrote:
> Fix the following bug in sel_netport_insert() where rcu_dereference() should
> be rcu_dereference_protected() as sel_netport_lock is held.
>
> ===================================================
> [ INFO: suspicious rcu_dereference_check() usage. ]
> ---------------------------------------------------
> security/selinux/netport.c:127 invoked rcu_dereference_check() without
> protection!
>
> other info that might help us debug this:
>
>
> rcu_scheduler_active = 1, debug_locks = 0
> 1 lock held by ossec-rootcheck/3323:
> #0: (sel_netport_lock){+.....}, at: [<ffffffff8117d775>]
> sel_netport_sid+0xbb/0x226
>
> stack backtrace:
> Pid: 3323, comm: ossec-rootcheck Not tainted 3.1.0-rc8-fsdevel+ #1095
> Call Trace:
> [<ffffffff8105cfb7>] lockdep_rcu_dereference+0xa7/0xb0
> [<ffffffff8117d871>] sel_netport_sid+0x1b7/0x226
> [<ffffffff8117d6ba>] ? sel_netport_avc_callback+0xbc/0xbc
> [<ffffffff8117556c>] selinux_socket_bind+0x115/0x230
> [<ffffffff810a5388>] ? might_fault+0x4e/0x9e
> [<ffffffff810a53d1>] ? might_fault+0x97/0x9e
> [<ffffffff81171cf4>] security_socket_bind+0x11/0x13
> [<ffffffff812ba967>] sys_bind+0x56/0x95
> [<ffffffff81380dac>] ? sysret_check+0x27/0x62
> [<ffffffff8105b767>] ? trace_hardirqs_on_caller+0x11e/0x155
> [<ffffffff81076fcd>] ? audit_syscall_entry+0x17b/0x1ae
> [<ffffffff811b5eae>] ? trace_hardirqs_on_thunk+0x3a/0x3f
> [<ffffffff81380d7b>] system_call_fastpath+0x16/0x1b
>
> Signed-off-by: David Howells <dhowells@redhat.com>
> ---
>
> security/selinux/netport.c | 4 +++-
> 1 files changed, 3 insertions(+), 1 deletions(-)
Acked-by: Paul Moore <paul@paul-moore.com>
> diff --git a/security/selinux/netport.c b/security/selinux/netport.c
> index 0b62bd1..7b9eb1f 100644
> --- a/security/selinux/netport.c
> +++ b/security/selinux/netport.c
> @@ -123,7 +123,9 @@ static void sel_netport_insert(struct sel_netport *port)
> if (sel_netport_hash[idx].size == SEL_NETPORT_HASH_BKT_LIMIT) {
> struct sel_netport *tail;
> tail = list_entry(
> - rcu_dereference(sel_netport_hash[idx].list.prev),
> + rcu_dereference_protected(
> + sel_netport_hash[idx].list.prev,
> + lockdep_is_held(&sel_netport_lock)),
> struct sel_netport, list);
> list_del_rcu(&tail->list);
> kfree_rcu(tail, rcu);
>
> --
> To unsubscribe from this list: send the line "unsubscribe netdev" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
paul moore
www.paul-moore.com
^ permalink raw reply
* Re: [PATCH] bridge: fix hang on removal of bridge via netlink
From: Sridhar Samudrala @ 2011-10-06 23:02 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: David Miller, netdev
In-Reply-To: <20111006141941.437be127@nehalam.linuxnetplumber.net>
On Thu, 2011-10-06 at 14:19 -0700, Stephen Hemminger wrote:
> Need to cleanup bridge device timers and ports when being bridge
> device is being removed via netlink.
>
> This fixes the problem of observed when doing:
> ip link add br0 type bridge
> ip link set dev eth1 master br0
> ip link set br0 up
> ip link del br0
>
> which would cause br0 to hang in unregister_netdev because
> of leftover reference count.
>
> Reported-by: Sridhar Samudrala <sri@us.ibm.com>
> Signed-off-by: Stephen Hemminger <shemminger@vyatta.com>
Acked-by: Sridhar Samudrala <sri@us.ibm.com>
>
> ---
> Patch is against net-next but should go to net and stable trees
> since it is an observable hang on 3.0 and later kernels.
>
> --- a/net/bridge/br_if.c 2011-10-03 11:08:36.304168386 -0700
> +++ b/net/bridge/br_if.c 2011-10-06 11:27:47.682488755 -0700
> @@ -160,9 +160,10 @@ static void del_nbp(struct net_bridge_po
> call_rcu(&p->rcu, destroy_nbp_rcu);
> }
>
> -/* called with RTNL */
> -static void del_br(struct net_bridge *br, struct list_head *head)
> +/* Delete bridge device */
> +void br_dev_delete(struct net_device *dev, struct list_head *head)
> {
> + struct net_bridge *br = netdev_priv(dev);
> struct net_bridge_port *p, *n;
>
> list_for_each_entry_safe(p, n, &br->port_list, list) {
> @@ -267,7 +268,7 @@ int br_del_bridge(struct net *net, const
> }
>
> else
> - del_br(netdev_priv(dev), NULL);
> + br_dev_delete(dev, NULL);
>
> rtnl_unlock();
> return ret;
> @@ -446,7 +447,7 @@ void __net_exit br_net_exit(struct net *
> rtnl_lock();
> for_each_netdev(net, dev)
> if (dev->priv_flags & IFF_EBRIDGE)
> - del_br(netdev_priv(dev), &list);
> + br_dev_delete(dev, &list);
>
> unregister_netdevice_many(&list);
> rtnl_unlock();
> --- a/net/bridge/br_netlink.c 2011-09-16 13:12:58.061369744 -0700
> +++ b/net/bridge/br_netlink.c 2011-10-06 11:20:21.808911679 -0700
> @@ -210,6 +210,7 @@ static struct rtnl_link_ops br_link_ops
> .priv_size = sizeof(struct net_bridge),
> .setup = br_dev_setup,
> .validate = br_validate,
> + .dellink = br_dev_delete,
> };
>
> int __init br_netlink_init(void)
> --- a/net/bridge/br_private.h 2011-10-06 08:42:27.353044954 -0700
> +++ b/net/bridge/br_private.h 2011-10-06 11:25:17.845118817 -0700
> @@ -301,6 +301,7 @@ static inline int br_is_root_bridge(cons
>
> /* br_device.c */
> extern void br_dev_setup(struct net_device *dev);
> +extern void br_dev_delete(struct net_device *dev, struct list_head *list);
> extern netdev_tx_t br_dev_xmit(struct sk_buff *skb,
> struct net_device *dev);
> #ifdef CONFIG_NET_POLL_CONTROLLER
^ permalink raw reply
* [PATCH] iproute2: Fix usage and man page for 'ip link'
From: Sridhar Samudrala @ 2011-10-06 23:10 UTC (permalink / raw)
To: Stephen Hemminger; +Cc: netdev
Add bridge as a supported type with 'ip link' in usage and all the missing
types in 'ip' man page. Also fixed some typos.
Signed-off-by: Sridhar Samudrala <sri@us.ibm.com>
diff --git a/ip/iplink.c b/ip/iplink.c
index e5325a6..35e6dc6 100644
--- a/ip/iplink.c
+++ b/ip/iplink.c
@@ -43,7 +43,7 @@ static int iplink_have_newlink(void);
void iplink_usage(void)
{
if (iplink_have_newlink()) {
- fprintf(stderr, "Usage: ip link add link DEV [ name ] NAME\n");
+ fprintf(stderr, "Usage: ip link add [link DEV] [ name ] NAME\n");
fprintf(stderr, " [ txqueuelen PACKETS ]\n");
fprintf(stderr, " [ address LLADDR ]\n");
fprintf(stderr, " [ broadcast LLADDR ]\n");
@@ -78,7 +78,7 @@ void iplink_usage(void)
if (iplink_have_newlink()) {
fprintf(stderr, "\n");
- fprintf(stderr, "TYPE := { vlan | veth | vcan | dummy | ifb | macvlan | can }\n");
+ fprintf(stderr, "TYPE := { vlan | veth | vcan | dummy | ifb | macvlan | can | bridge }\n");
}
exit(-1);
}
diff --git a/man/man8/ip.8 b/man/man8/ip.8
index 27993a4..49d94f5 100644
--- a/man/man8/ip.8
+++ b/man/man8/ip.8
@@ -47,7 +47,7 @@ ip \- show / manipulate routing, devices, policy routing and tunnels
.ti -8
.IR TYPE " := [ "
-.BR vlan " | " maclan " | " can " ]"
+.BR vlan " | " veth " | " vcan " | " dummy " | " ifb " | " macvlan " | " can " | " bridge ]"
.ti -8
.BI "ip link delete " DEVICE
@@ -989,13 +989,28 @@ Link types:
.in +8
.B vlan
-- 802.1q tagged virrtual LAN interface
+- 802.1q tagged virtual LAN interface
+.sp
+.B veth
+- Virtual ethernet interface
+.sp
+.B vcan
+- Virtual Local CAN interface
+.sp
+.B dummy
+- Dummy network interface
+.sp
+.B ifb
+- Intermediate Functional Block device
.sp
.B macvlan
- virtual interface base on link layer address (MAC)
.sp
.B can
- Controller Area Network interface
+.sp
+.B bridge
+- Ethernet Bridge device
.in -8
.SS ip link delete - delete virtual link
^ permalink raw reply related
* Re: Asserting ECN from userspace?
From: Andi Kleen @ 2011-10-06 23:52 UTC (permalink / raw)
To: David Täht; +Cc: netdev, bloat-devel
In-Reply-To: <4E8BF6B2.6030101@gmail.com>
David Täht <dave.taht@gmail.com> writes:
>
> And twiddling them, on a per stream basis, for a single packet, would
> seem to require something more robust than setsockopt/getsockopt
> (although that would work for udp streams)
With netfilter nf_queue you can construct a rule that passes packets
through user space and reinjects them.
I would suggest to just use that to modify the ECN bits.
I'm sure with reasonable google skills you can find some examples
how to do this on the web.
-Andi
--
ak@linux.intel.com -- Speaking for myself only
^ permalink raw reply
* Re: [PATCH] IPv6: DAD from bonding iface is treated as dup address from others
From: Yinglin Sun @ 2011-10-07 0:03 UTC (permalink / raw)
To: Jay Vosburgh
Cc: Neil Horman, David S. Miller, Alexey Kuznetsov, James Morris,
Hideaki YOSHIFUJI, Patrick McHardy, netdev
In-Reply-To: <CAN17JHUo00BGbsQ0wDjhcY-wWRuGr-in-i_JBzE5__jgO5be=g@mail.gmail.com>
On Thu, Oct 6, 2011 at 3:17 PM, Yinglin Sun <Yinglin.Sun@emc.com> wrote:
>
> On Thu, Oct 6, 2011 at 12:05 PM, Jay Vosburgh <fubar@us.ibm.com> wrote:
> >
> > Neil Horman <nhorman@tuxdriver.com> wrote:
> >
> > >On Wed, Oct 05, 2011 at 08:59:10PM -0700, Yinglin Sun wrote:
> > >> Steps to reproduce this issue:
> > >> 1. create bond0 over eth0 and eth1, set the mode to balance-xor
> > >> 2. add an IPv6 address to bond0
> > >> 3. DAD packet is sent out from one slave and then is looped back from
> > >> the other slave. Therefore, it is treated as a duplicate address and
> > >> stays tentative afterwards:
> > >> kern.info:
> > >> Oct 5 11:50:18 testvm1 kernel: [ 129.224353] bond0: IPv6 duplicate address 1234::1 detected!
> > >>
> > >> Signed-off-by: Yinglin Sun <Yinglin.Sun@emc.com>
> > >> ---
> > >> net/ipv6/ndisc.c | 15 +++++++++++++--
> > >> 1 files changed, 13 insertions(+), 2 deletions(-)
> > >>
> > >> diff --git a/net/ipv6/ndisc.c b/net/ipv6/ndisc.c
> > >> index 9da6e02..c82f4c7 100644
> > >> --- a/net/ipv6/ndisc.c
> > >> +++ b/net/ipv6/ndisc.c
> > >> @@ -809,9 +809,10 @@ static void ndisc_recv_ns(struct sk_buff *skb)
> > >>
> > >> if (ifp->flags & (IFA_F_TENTATIVE|IFA_F_OPTIMISTIC)) {
> > >> if (dad) {
> > >> + const unsigned char *sadr;
> > >> + sadr = skb_mac_header(skb);
> > >> +
> > >> if (dev->type == ARPHRD_IEEE802_TR) {
> > >> - const unsigned char *sadr;
> > >> - sadr = skb_mac_header(skb);
> > >> if (((sadr[8] ^ dev->dev_addr[0]) & 0x7f) == 0 &&
> > >> sadr[9] == dev->dev_addr[1] &&
> > >> sadr[10] == dev->dev_addr[2] &&
> > >> @@ -821,6 +822,16 @@ static void ndisc_recv_ns(struct sk_buff *skb)
> > >> /* looped-back to us */
> > >> goto out;
> > >> }
> > >> + } else if (dev->type == ARPHRD_ETHER) {
> > >> + if (sadr[6] == dev->dev_addr[0] &&
> > >> + sadr[7] == dev->dev_addr[1] &&
> > >> + sadr[8] == dev->dev_addr[2] &&
> > >> + sadr[9] == dev->dev_addr[3] &&
> > >> + sadr[10] == dev->dev_addr[4] &&
> > >> + sadr[11] == dev->dev_addr[5]) {
> > >> + /* looped-back to us */
> > >> + goto out;
> > >> + }
> > >> }
> > >>
> > >> /*
> > >> --
> > >> 1.7.4.1
> > >>
> > >Nack, This seems like it will just completely break DAD. What if theres another
> > >system out there with the same mac address. A response from that system would
> > >get dropped by this filter, instead of causing The local system to stop using
> > >the address. What you really want to do is modify
> > >bond_should_deliver_exact_match to detect this frame on the inactive slave or
> > >some such, and drop the frame there.
> >
> > Also NACK; and adding a bit of information. The balance-xor
> > mode is nominally expecting to interact with a switch whose ports are
> > set for etherchannel ("static link aggregation"), in which case the
> > switch will not loop the packet back around.
> >
> > If your switch can do etherchannel, then enable it and the
> > problem should go away. If your switch cannot do this, then you may
> > have other issues, because all of the multicast or broadcast packets
> > going out any bonding slave will loop around to another slave. You
> > could also use 802.3ad / LACP if you switch supports that.
> >
> > For balance-xor (or balance-rr, for that matter) mode to a
> > non-etherchannel switch, it's going to be difficult, if not impossible,
> > to modify bond_should_deliver_exact_match, because there are no inactive
> > slaves. In this mode, bonding is expecting the switch to balance
> > incoming traffic across the ports, and not deliver looped back packets
> > or duplicates. There are no restrictions on what type of traffic
> > (mcast, bcast, ucast) may arrive on any given port.
> >
> > I can't think of a way to make the non-etherchannel case work
> > for balance-xor (or balance-rr) without breaking the DAD functionality
> > in the case of an actual duplicate. I'm not aware of a way to
> > distinguish a looped back DAD probe from an actual duplicate address
> > probe elsewhere on the network.
> >
>
> Hi Neil & Jay,
>
> Thanks a lot for the comments.
>
> The use case is to add IPv6 address on the bonding interface first,
> and then set up port channel on switch. We'll hit this issue and the
> new address will stay tentative and unusable after port channel is set
> up on switch. This patch is for this valid use case.
>
> Except failover mode, all slaves are active on receiving packets, so
> we are receiving such looped back DAD and the bonding driver cannot
> ignore them. I cannot think of a way to distinguish if a DAD is looped
> back or from someone else having the same mac address. They look the
> same to the host. If there is another machine having the same mac
> address, this code path gets executed if both are doing DAD at the
> same time for the same IPv6 address. Maybe we should find out what the
> specification defines for this case?
>
RFC4862 has a discussion about this issue:
http://tools.ietf.org/html/rfc4862#appendix-A
The better solution could be to record the number of DAD sent out. If
we received more DAD packets than we sent out, there is someone else
on the network who has the same mac address and sent DAD for the same
IPv6 address. However, this solution doesn't work with bonding
interface, since all other active slaves but the one sending out DAD
will receive packet looped back. It doesn't seem there is a simple
solution for this issue.
Yinglin
^ permalink raw reply
* Re: [PATCH] IPv6: DAD from bonding iface is treated as dup address from others
From: Jay Vosburgh @ 2011-10-07 0:59 UTC (permalink / raw)
To: Yinglin Sun
Cc: Neil Horman, David S. Miller, Alexey Kuznetsov, James Morris,
Hideaki YOSHIFUJI, Patrick McHardy, netdev
In-Reply-To: <CAN17JHWgYyrz0WFmXiyR8Ep1W-zGjeQEKCADmX6zMT=ZfQ8n7Q@mail.gmail.com>
Yinglin Sun <Yinglin.Sun@emc.com> wrote:
>On Thu, Oct 6, 2011 at 3:17 PM, Yinglin Sun <Yinglin.Sun@emc.com> wrote:
>>
>> On Thu, Oct 6, 2011 at 12:05 PM, Jay Vosburgh <fubar@us.ibm.com> wrote:
>> >
>> > Neil Horman <nhorman@tuxdriver.com> wrote:
>> >
>> > >On Wed, Oct 05, 2011 at 08:59:10PM -0700, Yinglin Sun wrote:
>> > >> Steps to reproduce this issue:
>> > >> 1. create bond0 over eth0 and eth1, set the mode to balance-xor
>> > >> 2. add an IPv6 address to bond0
>> > >> 3. DAD packet is sent out from one slave and then is looped back from
>> > >> the other slave. Therefore, it is treated as a duplicate address and
>> > >> stays tentative afterwards:
>> > >> kern.info:
>> > >> Oct 5 11:50:18 testvm1 kernel: [ 129.224353] bond0: IPv6 duplicate address 1234::1 detected!
[...]
>> > >Nack, This seems like it will just completely break DAD. What if theres another
>> > >system out there with the same mac address. A response from that system would
>> > >get dropped by this filter, instead of causing The local system to stop using
>> > >the address. What you really want to do is modify
>> > >bond_should_deliver_exact_match to detect this frame on the inactive slave or
>> > >some such, and drop the frame there.
>> >
>> > Also NACK; and adding a bit of information. The balance-xor
>> > mode is nominally expecting to interact with a switch whose ports are
>> > set for etherchannel ("static link aggregation"), in which case the
>> > switch will not loop the packet back around.
>> >
>> > If your switch can do etherchannel, then enable it and the
>> > problem should go away. If your switch cannot do this, then you may
>> > have other issues, because all of the multicast or broadcast packets
>> > going out any bonding slave will loop around to another slave. You
>> > could also use 802.3ad / LACP if you switch supports that.
>> >
>> > For balance-xor (or balance-rr, for that matter) mode to a
>> > non-etherchannel switch, it's going to be difficult, if not impossible,
>> > to modify bond_should_deliver_exact_match, because there are no inactive
>> > slaves. In this mode, bonding is expecting the switch to balance
>> > incoming traffic across the ports, and not deliver looped back packets
>> > or duplicates. There are no restrictions on what type of traffic
>> > (mcast, bcast, ucast) may arrive on any given port.
>> >
>> > I can't think of a way to make the non-etherchannel case work
>> > for balance-xor (or balance-rr) without breaking the DAD functionality
>> > in the case of an actual duplicate. I'm not aware of a way to
>> > distinguish a looped back DAD probe from an actual duplicate address
>> > probe elsewhere on the network.
>> >
>>
>> Hi Neil & Jay,
>>
>> Thanks a lot for the comments.
>>
>> The use case is to add IPv6 address on the bonding interface first,
>> and then set up port channel on switch. We'll hit this issue and the
>> new address will stay tentative and unusable after port channel is set
>> up on switch. This patch is for this valid use case.
>>
>> Except failover mode, all slaves are active on receiving packets, so
>> we are receiving such looped back DAD and the bonding driver cannot
>> ignore them. I cannot think of a way to distinguish if a DAD is looped
>> back or from someone else having the same mac address. They look the
>> same to the host. If there is another machine having the same mac
>> address, this code path gets executed if both are doing DAD at the
>> same time for the same IPv6 address. Maybe we should find out what the
>> specification defines for this case?
>>
>
>RFC4862 has a discussion about this issue:
>http://tools.ietf.org/html/rfc4862#appendix-A
>The better solution could be to record the number of DAD sent out. If
>we received more DAD packets than we sent out, there is someone else
>on the network who has the same mac address and sent DAD for the same
>IPv6 address. However, this solution doesn't work with bonding
>interface, since all other active slaves but the one sending out DAD
>will receive packet looped back. It doesn't seem there is a simple
>solution for this issue.
Why are you setting up the port channel after configuring the
bond?
As a possible workaround, if you have control over the setup
process (perhaps it's some sort of manual process), adding one slave to
the bond, leaving the other soon-to-be slaves down, then setting up the
switch, and finally adding the remaining slaves should work around the
issue, since if the bond has only one slave it won't see any looped
packets.
Or you could bring the bond up as active-backup, then change the
mode to balance-xor once the switch is configured.
Ultimately, though, the problem stems from the settings mismatch
between the switch and the bonding system; balance-xor is meant to
interoperate with etherchannel, and when the switch is not configured
properly, correct behavior is difficult to guarantee.
-J
---
-Jay Vosburgh, IBM Linux Technology Center, fubar@us.ibm.com
^ permalink raw reply
* Re: [PATCH] IPv6: DAD from bonding iface is treated as dup address from others
From: Yinglin Sun @ 2011-10-07 1:24 UTC (permalink / raw)
To: Jay Vosburgh
Cc: Neil Horman, David S. Miller, Alexey Kuznetsov, James Morris,
Hideaki YOSHIFUJI, Patrick McHardy, netdev
In-Reply-To: <7122.1317949193@death>
On Thu, Oct 6, 2011 at 5:59 PM, Jay Vosburgh <fubar@us.ibm.com> wrote:
> Yinglin Sun <Yinglin.Sun@emc.com> wrote:
>
>>On Thu, Oct 6, 2011 at 3:17 PM, Yinglin Sun <Yinglin.Sun@emc.com> wrote:
>>>
>>> On Thu, Oct 6, 2011 at 12:05 PM, Jay Vosburgh <fubar@us.ibm.com> wrote:
>>> >
>>> > Neil Horman <nhorman@tuxdriver.com> wrote:
>>> >
>>> > >On Wed, Oct 05, 2011 at 08:59:10PM -0700, Yinglin Sun wrote:
>>> > >> Steps to reproduce this issue:
>>> > >> 1. create bond0 over eth0 and eth1, set the mode to balance-xor
>>> > >> 2. add an IPv6 address to bond0
>>> > >> 3. DAD packet is sent out from one slave and then is looped back from
>>> > >> the other slave. Therefore, it is treated as a duplicate address and
>>> > >> stays tentative afterwards:
>>> > >> kern.info:
>>> > >> Oct 5 11:50:18 testvm1 kernel: [ 129.224353] bond0: IPv6 duplicate address 1234::1 detected!
>
> [...]
>
>>> > >Nack, This seems like it will just completely break DAD. What if theres another
>>> > >system out there with the same mac address. A response from that system would
>>> > >get dropped by this filter, instead of causing The local system to stop using
>>> > >the address. What you really want to do is modify
>>> > >bond_should_deliver_exact_match to detect this frame on the inactive slave or
>>> > >some such, and drop the frame there.
>>> >
>>> > Also NACK; and adding a bit of information. The balance-xor
>>> > mode is nominally expecting to interact with a switch whose ports are
>>> > set for etherchannel ("static link aggregation"), in which case the
>>> > switch will not loop the packet back around.
>>> >
>>> > If your switch can do etherchannel, then enable it and the
>>> > problem should go away. If your switch cannot do this, then you may
>>> > have other issues, because all of the multicast or broadcast packets
>>> > going out any bonding slave will loop around to another slave. You
>>> > could also use 802.3ad / LACP if you switch supports that.
>>> >
>>> > For balance-xor (or balance-rr, for that matter) mode to a
>>> > non-etherchannel switch, it's going to be difficult, if not impossible,
>>> > to modify bond_should_deliver_exact_match, because there are no inactive
>>> > slaves. In this mode, bonding is expecting the switch to balance
>>> > incoming traffic across the ports, and not deliver looped back packets
>>> > or duplicates. There are no restrictions on what type of traffic
>>> > (mcast, bcast, ucast) may arrive on any given port.
>>> >
>>> > I can't think of a way to make the non-etherchannel case work
>>> > for balance-xor (or balance-rr) without breaking the DAD functionality
>>> > in the case of an actual duplicate. I'm not aware of a way to
>>> > distinguish a looped back DAD probe from an actual duplicate address
>>> > probe elsewhere on the network.
>>> >
>>>
>>> Hi Neil & Jay,
>>>
>>> Thanks a lot for the comments.
>>>
>>> The use case is to add IPv6 address on the bonding interface first,
>>> and then set up port channel on switch. We'll hit this issue and the
>>> new address will stay tentative and unusable after port channel is set
>>> up on switch. This patch is for this valid use case.
>>>
>>> Except failover mode, all slaves are active on receiving packets, so
>>> we are receiving such looped back DAD and the bonding driver cannot
>>> ignore them. I cannot think of a way to distinguish if a DAD is looped
>>> back or from someone else having the same mac address. They look the
>>> same to the host. If there is another machine having the same mac
>>> address, this code path gets executed if both are doing DAD at the
>>> same time for the same IPv6 address. Maybe we should find out what the
>>> specification defines for this case?
>>>
>>
>>RFC4862 has a discussion about this issue:
>>http://tools.ietf.org/html/rfc4862#appendix-A
>>The better solution could be to record the number of DAD sent out. If
>>we received more DAD packets than we sent out, there is someone else
>>on the network who has the same mac address and sent DAD for the same
>>IPv6 address. However, this solution doesn't work with bonding
>>interface, since all other active slaves but the one sending out DAD
>>will receive packet looped back. It doesn't seem there is a simple
>>solution for this issue.
>
> Why are you setting up the port channel after configuring the
> bond?
>
> As a possible workaround, if you have control over the setup
> process (perhaps it's some sort of manual process), adding one slave to
> the bond, leaving the other soon-to-be slaves down, then setting up the
> switch, and finally adding the remaining slaves should work around the
> issue, since if the bond has only one slave it won't see any looped
> packets.
>
> Or you could bring the bond up as active-backup, then change the
> mode to balance-xor once the switch is configured.
>
> Ultimately, though, the problem stems from the settings mismatch
> between the switch and the bonding system; balance-xor is meant to
> interoperate with etherchannel, and when the switch is not configured
> properly, correct behavior is difficult to guarantee.
>
Jay,
Thanks a lot for the suggestion.
It's mainly about usability. We would like to provide customers with
consistent IPv6 configuration procedures as IPv4. Such workarounds
could be confusing and generate customer calls.
Yinglin
^ permalink raw reply
* Re: big picture UDP/IP performance question re 2.6.18 -> 2.6.32
From: starlight @ 2011-10-07 2:24 UTC (permalink / raw)
To: linux-kernel, netdev, Peter Zijlstra, Christoph Lameter,
Eric Dumazet, Will
UPDATE
Hi all, I'm back with a significant update.
First, discovered that the problem with UDP was a tightening of the reverse-path filter logic in newer kernels. To make the test work had to configure 'net.ipv4.conf.default.rp_filter = 0' in 'sysctl.conf'. Was able to rerun all the tests in the nominal UDP sockets mode instead of packet socket mode.
Second I was intrigued by an assertion put forward that older kernels are always better than newer kernels. So with some effort I managed to coerce 2.6.9(rhel4) into running on the Opteron 6174 CPU, though it only recognized eight of the twelve cores.
Here are the CPU results. User and sys are expressed in jiffies.
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 02:18:40 625666 206423 (24.8%) 18k
2.6.9-101.EL(rhel4) 02:31:12 689024 218198 (24.0%) 18k*
2.6.32-71.29.1.el6 02:50:14 602191 419276 (41.0%) 64k
2.6.39.4(vanilla) 02:42:35 629817 345674 (35.4%) 85k
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 - - - -
2.6.9-101.EL(rhel4) +9.0% +10.1% +5.7% 0%*
2.6.32-71.29.1.el6 +22.7% -3.8% +103% +255%
2.6.39.4(vanilla) +17.2% +0.7% +67.4% +372%
* test speed was run at 2/3rds to equalize
the per-core frame rate since four cores
were disabled
Here are some latency data samples. Report interval is 10 seconds, all latency columns are in milliseconds.
2.6.18-194.8.1.el5
sample min max mean sigma
198283 0.011 3.139 0.239 0.169
206597 0.012 3.085 0.237 0.178
195939 0.012 4.220 0.266 0.211
206378 0.013 4.006 0.274 0.218
211994 0.012 3.771 0.248 0.184
222325 0.011 3.106 0.234 0.156
210871 0.011 2.693 0.254 0.177
2.6.9-101.EL(rhel4)
sample min max mean sigma
139786 0.013 3.637 0.335 0.238
149788 0.013 3.957 0.384 0.258
144065 0.014 7.637 0.376 0.283
142088 0.012 3.996 0.398 0.281
141026 0.014 4.174 0.383 0.253
143387 0.014 4.457 0.366 0.236
147699 0.013 4.615 0.359 0.238
2.6.32-71.29.1.el6
sample min max mean sigma
206452 0.015 4.516 0.268 0.197
195740 0.016 4.227 0.277 0.206
206644 0.012 3.412 0.276 0.194
212008 0.012 2.569 0.269 0.182
222119 0.011 2.523 0.266 0.178
211377 0.012 2.779 0.277 0.178
214113 0.013 2.680 0.277 0.184
2.6.39.4(vanilla)
sample min max mean sigma
198530 0.012 2.736 0.274 0.147
200148 0.012 1.971 0.272 0.145
209972 0.010 2.975 0.270 0.150
219775 0.012 2.595 0.263 0.151
215601 0.015 2.554 0.276 0.153
211549 0.010 3.075 0.282 0.158
219332 0.012 2.658 0.271 0.144
2.6.9(rhel4) was a bit worse than 2.6.18(rhel5). However the fact that it ran on eight cores instead of twelve may have subtly affected the outcome. Also this kernel was built with the gcc 3.4.6 RH distro compiler which may generate code less well optimized for the 6174 CPU. Therefore I view it as pretty close to a tie, but 2.6.18(rhel5) is the obviously better choice for the superior hardware support it provides.
Despite consuming a great deal more CPU, the newer kernels made a good showing on latency in this mode where only one thread wakeup occurs per arriving frame. Both have slighter tighter latency distributions and slightly improved worst-case latencies. 2.6.39.4 is somewhat better than 2.6.32(rhel6) on all fronts. Nonetheless I find the much higher CPU consumption to be a major negative since this reduces the overall capacity available on identical hardware.
-----
Now for the tests performed earlier and to which all my comments prior to this post apply. Ran these only because I thought UDP was broken, but now I find I'm pleased with the added perspective they provide. In this mode our application reads all the data from four interfaces via packet sockets (one per interface), and then requeues the packets to a pool of worker threads. Potentially two application worker thread wakeup events occur for each frame, though wakeups only occur when work queues become empty. The worker thread pool consists of nine threads with packets routed by similar type to maximize cache locality during processing.
2.6.18-194.8.1.el5
sample min max mean sigma
217629 0.015 1.841 0.150 0.080
217523 0.015 1.213 0.155 0.076
209183 0.014 0.624 0.129 0.066
220726 0.014 1.255 0.160 0.087
220400 0.015 3.374 0.197 0.172
238151 0.014 1.275 0.182 0.090
249239 0.016 1.399 0.197 0.093
2.6.9-101.EL(rhel4)
sample min max mean sigma
138561 0.019 3.163 0.333 0.236
144033 0.014 3.691 0.337 0.240
147437 0.016 3.802 0.320 0.226
147297 0.016 4.063 0.351 0.292
156178 0.018 3.560 0.322 0.244
166156 0.019 3.983 0.326 0.246
161930 0.017 3.441 0.311 0.197
2.6.32-71.29.1.el6
sample min max mean sigma
210340 0.017 8.734 0.344 0.638
220199 0.020 7.319 0.341 0.530
216868 0.019 6.376 0.332 0.544
211556 0.019 7.494 0.318 0.493
219462 0.014 7.472 0.344 0.545
225027 0.022 8.103 0.382 0.639
245629 0.017 8.713 0.382 0.594
2.6.39.4(vanilla)
sample min max mean sigma
253791 0.020 6.127 0.505 0.544
256739 0.020 6.960 0.535 0.577
258719 0.019 8.116 0.500 0.541
244735 0.018 6.781 0.542 0.634
250195 0.021 8.205 0.531 0.568
In this mode latency is quite a lot better with the 2.6.18(rhel5) kernel and seriously worse in the newer kernels. Perhaps what is happening in 2.6.18(rhel) is that frames are being received on one set of cores and processed on a different set, with relatively few actual thread sleep/wake events. The cache on each core is hot for the task it is performing, and due to the specialization of each worker thread less code cache pressure is present. In 2.6.39.4 I can only say that the context switch rate was in the typical 200k/sec range where older kernels ran at less than half that. Sorry now that I did not record CS rates more carefully.
Here's CPU consumption:
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 02:07:16 615516 148152 (19.4%) 28k
2.6.9-101.EL(rhel4) 02:30:22 696344 205953 (22.8%) 20k
2.6.32-71.29.1.el6 02:15:50 585276 229767 (28.1%) 163k
2.6.39.4(vanilla) 02:27:44 658074 228420 (25.7%) 165k
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 - - - -
2.6.9-101.EL(rhel4) +18.1% +13.1% +39.0% -29%
2.6.32-71.29.1.el6 +6.7% -4.9% +55.0% +482%
2.6.39.4(vanilla) +16.0% +6.0% +54.1% +489%
The 2.6.18(rhel5) kernel performed significantly better here than in the many-threaded UDP-mode which I attribute to the close matching of the thread and core count and the reasons stated in the previous paragraph. CPU consumption of 2.6.9(rhel4) with thirteen active threads scheduled on eight cores was about the same here as it was with the large UDP-mode thread count.
Relative to the many-threaded UDP-mode, matching of thread and core counts seems to help .32(rhel6) a great deal, but to a lesser degree for .39.
As with UDP-mode, system overhead is substantialy higher with newer kernels.
-----
Note that some tests were run with packets sockets and a large thread pool, but not all scenarios were covered and I find it less interesting than the small thread pool so it is omitted here.
Have perf reports for all 2.6.39.4 runs and a perf report for the the packet socket run on 2.6.32(rhel6). If anyone is interested let me know and I'll post them.
-----
Overall our application runs best with 2.6.18(rhel5) in all regards excepting a slight improvement in latency distribution where, despite higher CPU consumption, the .32 and .39 kernels are better at 50% CPU load. Naturally at higher data rates the newer kernel latency will go to pieces sooner and the absolute maximum performance attainable is substantially lower.
Until truly spectacular core counts arrive 2.6.18 remains the kernel of choice.
We would find it attractive if an option to compile newer kernels with a circa 2.6.18 O(1) scheduler was made available in the future. Not a problem if doing so would disable any number of dependent kernel features. Target here is HPC where in our view less is more and virtuzlization, resource allocation fairness (not to mention the SCHED_OTHER priority policy) etc. have little or no utility. A shorter scheduler code path-length and lower cache pressure are more important than enhanced functionality. The original O(1) scheduler appears to handle chains of related-processing thread hand-offs dramatically better.
^ permalink raw reply
* Re: big picture UDP/IP performance question re 2.6.18 -> 2.6.32
From: starlight @ 2011-10-07 2:33 UTC (permalink / raw)
To: linux-kernel, netdev
[repost with wrapped lines]
UPDATE
Hi all, I'm back with a significant update.
First, discovered that the problem with UDP was a
tightening of the reverse-path filter logic in
newer kernels. To make the test work had to
configure 'net.ipv4.conf.default.rp_filter = 0' in
'sysctl.conf'. Was able to rerun all the tests in
the nominal UDP sockets mode instead of packet
socket mode.
Second I was intrigued by an assertion put forward
that older kernels are always better than newer
kernels. So with some effort I managed to coerce
2.6.9(rhel4) into running on the Opteron 6174 CPU,
though it only recognized eight of the twelve
cores.
Here are the CPU results. User and sys are expressed in jiffies.
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 02:18:40 625666 206423 (24.8%) 18k
2.6.9-101.EL(rhel4) 02:31:12 689024 218198 (24.0%) 18k*
2.6.32-71.29.1.el6 02:50:14 602191 419276 (41.0%) 64k
2.6.39.4(vanilla) 02:42:35 629817 345674 (35.4%) 85k
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 - - - -
2.6.9-101.EL(rhel4) +9.0% +10.1% +5.7% 0%*
2.6.32-71.29.1.el6 +22.7% -3.8% +103% +255%
2.6.39.4(vanilla) +17.2% +0.7% +67.4% +372%
* test speed was run at 2/3rds to equalize
the per-core frame rate since four cores
were disabled
Here are some latency data samples. Report
interval is 10 seconds, all latency columns are in
milliseconds.
2.6.18-194.8.1.el5
sample min max mean sigma
198283 0.011 3.139 0.239 0.169
206597 0.012 3.085 0.237 0.178
195939 0.012 4.220 0.266 0.211
206378 0.013 4.006 0.274 0.218
211994 0.012 3.771 0.248 0.184
222325 0.011 3.106 0.234 0.156
210871 0.011 2.693 0.254 0.177
2.6.9-101.EL(rhel4)
sample min max mean sigma
139786 0.013 3.637 0.335 0.238
149788 0.013 3.957 0.384 0.258
144065 0.014 7.637 0.376 0.283
142088 0.012 3.996 0.398 0.281
141026 0.014 4.174 0.383 0.253
143387 0.014 4.457 0.366 0.236
147699 0.013 4.615 0.359 0.238
2.6.32-71.29.1.el6
sample min max mean sigma
206452 0.015 4.516 0.268 0.197
195740 0.016 4.227 0.277 0.206
206644 0.012 3.412 0.276 0.194
212008 0.012 2.569 0.269 0.182
222119 0.011 2.523 0.266 0.178
211377 0.012 2.779 0.277 0.178
214113 0.013 2.680 0.277 0.184
2.6.39.4(vanilla)
sample min max mean sigma
198530 0.012 2.736 0.274 0.147
200148 0.012 1.971 0.272 0.145
209972 0.010 2.975 0.270 0.150
219775 0.012 2.595 0.263 0.151
215601 0.015 2.554 0.276 0.153
211549 0.010 3.075 0.282 0.158
219332 0.012 2.658 0.271 0.144
2.6.9(rhel4) was a bit worse than 2.6.18(rhel5).
However the fact that it ran on eight cores
instead of twelve may have subtly affected the
outcome. Also this kernel was built with the gcc
3.4.6 RH distro compiler which may generate code
less well optimized for the 6174 CPU. Therefore I
view it as pretty close to a tie, but
2.6.18(rhel5) is the obviously better choice for
the superior hardware support it provides.
Despite consuming a great deal more CPU, the newer
kernels made a good showing on latency in this
mode where only one thread wakeup occurs per
arriving frame. Both have slighter tighter
latency distributions and slightly improved
worst-case latencies. 2.6.39.4 is somewhat better
than 2.6.32(rhel6) on all fronts. Nonetheless I
find the much higher CPU consumption to be a major
negative since this reduces the overall capacity
available on identical hardware.
-----
Now for the tests performed earlier and to which
all my comments prior to this post apply. Ran
these only because I thought UDP was broken, but
now I find I'm pleased with the added perspective
they provide. In this mode our application reads
all the data from four interfaces via packet
sockets (one per interface), and then requeues the
packets to a pool of worker threads. Potentially
two application worker thread wakeup events occur
for each frame, though wakeups only occur when
work queues become empty. The worker thread pool
consists of nine threads with packets routed by
similar type to maximize cache locality during
processing.
2.6.18-194.8.1.el5
sample min max mean sigma
217629 0.015 1.841 0.150 0.080
217523 0.015 1.213 0.155 0.076
209183 0.014 0.624 0.129 0.066
220726 0.014 1.255 0.160 0.087
220400 0.015 3.374 0.197 0.172
238151 0.014 1.275 0.182 0.090
249239 0.016 1.399 0.197 0.093
2.6.9-101.EL(rhel4)
sample min max mean sigma
138561 0.019 3.163 0.333 0.236
144033 0.014 3.691 0.337 0.240
147437 0.016 3.802 0.320 0.226
147297 0.016 4.063 0.351 0.292
156178 0.018 3.560 0.322 0.244
166156 0.019 3.983 0.326 0.246
161930 0.017 3.441 0.311 0.197
2.6.32-71.29.1.el6
sample min max mean sigma
210340 0.017 8.734 0.344 0.638
220199 0.020 7.319 0.341 0.530
216868 0.019 6.376 0.332 0.544
211556 0.019 7.494 0.318 0.493
219462 0.014 7.472 0.344 0.545
225027 0.022 8.103 0.382 0.639
245629 0.017 8.713 0.382 0.594
2.6.39.4(vanilla)
sample min max mean sigma
253791 0.020 6.127 0.505 0.544
256739 0.020 6.960 0.535 0.577
258719 0.019 8.116 0.500 0.541
244735 0.018 6.781 0.542 0.634
250195 0.021 8.205 0.531 0.568
In this mode latency is quite a lot better with
the 2.6.18(rhel5) kernel and seriously worse in
the newer kernels. Perhaps what is happening in
2.6.18(rhel) is that frames are being received on
one set of cores and processed on a different set,
with relatively few actual thread sleep/wake
events. The cache on each core is hot for the
task it is performing, and due to the
specialization of each worker thread less code
cache pressure is present. In 2.6.39.4 I can only
say that the context switch rate was in the
typical 200k/sec range where older kernels ran at
less than half that. Sorry now that I did not
record CS rates more carefully.
Here's CPU consumption:
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 02:07:16 615516 148152 (19.4%) 28k
2.6.9-101.EL(rhel4) 02:30:22 696344 205953 (22.8%) 20k
2.6.32-71.29.1.el6 02:15:50 585276 229767 (28.1%) 163k
2.6.39.4(vanilla) 02:27:44 658074 228420 (25.7%) 165k
kernel version cpu total user sys IRQ/s
2.6.18-194.8.1.el5 - - - -
2.6.9-101.EL(rhel4) +18.1% +13.1% +39.0% -29%
2.6.32-71.29.1.el6 +6.7% -4.9% +55.0% +482%
2.6.39.4(vanilla) +16.0% +6.0% +54.1% +489%
The 2.6.18(rhel5) kernel performed significantly
better here than in the many-threaded UDP-mode
which I attribute to the close matching of the
thread and core count and the reasons stated in
the previous paragraph. CPU consumption of
2.6.9(rhel4) with thirteen active threads
scheduled on eight cores was about the same here
as it was with the large UDP-mode thread count.
Relative to the many-threaded UDP-mode, matching
of thread and core counts seems to help .32(rhel6)
a great deal, but to a lesser degree for .39.
As with UDP-mode, system overhead is substantialy
higher with newer kernels.
-----
Note that some tests were run with packets sockets
and a large thread pool, but not all scenarios
were covered and I find it less interesting than
the small thread pool so it is omitted here.
Have perf reports for all 2.6.39.4 runs and a perf
report for the the packet socket run on
2.6.32(rhel6). If anyone is interested let me
know and I'll post them.
-----
Overall our application runs best with
2.6.18(rhel5) in all regards excepting a slight
improvement in latency distribution where, despite
higher CPU consumption, the .32 and .39 kernels
are better at 50% CPU load. Naturally at higher
data rates the newer kernel latency will go to
pieces sooner and the absolute maximum performance
attainable is substantially lower.
Until truly spectacular core counts arrive 2.6.18
remains the kernel of choice.
We would find it attractive if an option to
compile newer kernels with a circa 2.6.18 O(1)
scheduler was made available in the future. Not a
problem if doing so would disable any number of
dependent kernel features. Target here is HPC
where in our view less is more and virtuzlization,
resource allocation fairness (not to mention the
SCHED_OTHER priority policy) etc. have little or
no utility. A shorter scheduler code path-length
and lower cache pressure are more important than
enhanced functionality. The original O(1)
scheduler appears to handle chains of
related-processing thread hand-offs dramatically
better.
^ permalink raw reply
* Re: big picture UDP/IP performance question re 2.6.18 -> 2.6.32
From: starlight @ 2011-10-07 3:27 UTC (permalink / raw)
To: linux-kernel, netdev, Peter Zijlstra, Christoph Lameter,
Eric Dumazet, Will
After writing the last post, the large
difference in IRQ rate between the older
and newer kernels caught my eye.
I wonder if the hugely lower rate in the older
kernels reflects a more agile shifting
into and out of NAPI mode by the network
bottom-half.
In this test the sending system
pulses data out on millisecond boundaries
due to the behavior of nsleep(), which
is used to establish the playback pace.
If the older kernels are switching to NAPI
for much of surge and the switching out
once the pulse falls off, it might
conceivably result in much better latency
and overall performance.
All tests were run with Intel 82571
network interfaces and the 'e1000e'
device driver. Some used the driver
packaged with the kernel, some used
Intel driver compiled from the source
found on sourceforge.net. Never could
detected any difference between the two.
Since data in the production environment
also tends to arrive in bursts, I don't find
the pulsing playback behavior a detriment.
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox