* Re: [GIT/PATCH v3] xen network backend driver
From: Ian Campbell @ 2011-03-01 10:06 UTC (permalink / raw)
To: Ben Hutchings
Cc: netdev@vger.kernel.org, xen-devel, Jeremy Fitzhardinge,
Herbert Xu, Konrad Rzeszutek Wilk, Francois Romieu
In-Reply-To: <1298919198.2569.14.camel@bwh-desktop>
On Mon, 2011-02-28 at 18:53 +0000, Ben Hutchings wrote:
> On Mon, 2011-02-28 at 17:27 +0000, Ian Campbell wrote:
> This should be defined as unsigned long (ideally it would be u64, but
> that can't be updated atomically on 32-bit systems).
[...]
> Don't update last_rx; it's only needed on slave devices of a bond, and
> the bonding driver takes care of it now.
I made these two changes.
> [...]
> > +static int xenvif_change_mtu(struct net_device *dev, int mtu)
> > +{
> > + struct xenvif *vif = netdev_priv(dev);
> > + int max = vif->can_sg ? 65535 - ETH_HLEN : ETH_DATA_LEN;
> > + if (mtu > max)
> > + return -EINVAL;
> > + dev->mtu = mtu;
> > + return 0;
> > +}
> [...]
>
> Since any VLAN tag must be inserted inline, shouldn't the MTU limit be
> 65535 - VLAN_ETH_HLEN?
In that case shouldn't the other case also be ETH_FRAME_LEN -
VLAN_ETH_HLEN?
I'm not sure what is customary wrt MTU vs VLAN (or other) overheads
under Linux, do we take the hit of the overhead for every device
regardless of VLAN being configured or not or do we expect that people
will configure the MTU as necessary when they configure a VLAN?
Netback itself will cope fine with either MTU, it's the external
connectivity which will actually matter. e.g. the usual configuration
would be (where vifX.Y represents potentially multiple netback devices):
eth0 <-> eth0.VLAN <-> br0.VLAN <=> vifX.Y
So ultimately it will be the eth0 hardware/driver which matters.
There is a comment in net/8021q/vlan.c which says
/* need 4 bytes for extra VLAN header info,
* hope the underlying device can handle it.
*/
and propagates the underlying device's MTU unmodified so it seems that
the norm is to leave the MTU at maximum assuming no VLAN overhead and
defer any required tweaking to the admin?
Alternatively you might have the VLAN on the eth0 device inside the
guest (e.g. netback<->netfront acts like a trunk link) in which case
basically the same argument applies?
I don't really mind either way so I'm happy to follow whatever the
convention is.
Ian.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 10:19 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298912869.2941.687.camel@edumazet-laptop>
On Mon, Feb 28, 2011 at 06:07:49PM +0100, Eric Dumazet wrote:
> > The numbers differ a lot depending on the architecture we test on.
> >
> > F.e. on a 12 core AMD with 2 NUMA nodes:
> >
> > 2.6.32 named -n 1: 37.0kqps
> > named: 3.8kqps (yes, no joke, the socket receive buffer is
> > always full and the kernel drops pkts)
>
> Yes, this old kernel miss commit c377411f2494a93 added in 2.6.35
> (net: sk_add_backlog() take rmem_alloc into account)
I retested with net-2.6 w/o Herbert's patch:
named -n 1: 36.9kqps
named: 16.2kqps
> > 2.6.38-rc5+ with Herbert's patches:
> > named -n 1: 36.9kqps
> > named: 222.0kqps
^ permalink raw reply
* Re: 2.6.37 regression: adding main interface to a bridge breaks vlan interface RX
From: Francois Romieu @ 2011-03-01 10:16 UTC (permalink / raw)
To: Jesse Gross; +Cc: chriss, netdev
In-Reply-To: <AANLkTimmbG3_GjYhoO7F46hOcRM3Fq_mNifk2PpLstPa@mail.gmail.com>
Jesse Gross <jesse@nicira.com> :
[...]
> Putting a vlan device on eth1 actually affects the behavior of the
> driver, which is why that works. I suspect that if you put the vlan
> device on both the physical interface and the bridge you would see
> packets with tags on the bridge.
>
> Regardless, the solution is to remove the dependency on vlan devices
> by converting over to the new vlan model. Francois, any chance that
> you might have some time to look at this?
The completely untested patch below is available for the (really) desperate
souls.
I should do a -next testing session this afternoon for this patch +
Oliver's Neukum advertisement control + Hayes 8105e stuff (in reverse order).
diff --git a/drivers/net/Kconfig b/drivers/net/Kconfig
index 0382332..8e7da78 100644
--- a/drivers/net/Kconfig
+++ b/drivers/net/Kconfig
@@ -2235,15 +2235,6 @@ config R8169
To compile this driver as a module, choose M here: the module
will be called r8169. This is recommended.
-config R8169_VLAN
- bool "VLAN support"
- depends on R8169 && VLAN_8021Q
- ---help---
- Say Y here for the r8169 driver to support the functions required
- by the kernel 802.1Q code.
-
- If in doubt, say Y.
-
config SB1250_MAC
tristate "SB1250 Gigabit Ethernet support"
depends on SIBYTE_SB1xxx_SOC
diff --git a/drivers/net/r8169.c b/drivers/net/r8169.c
index bde7d61..5c4f03d 100644
--- a/drivers/net/r8169.c
+++ b/drivers/net/r8169.c
@@ -526,9 +526,6 @@ struct rtl8169_private {
u16 napi_event;
u16 intr_mask;
int phy_1000_ctrl_reg;
-#ifdef CONFIG_R8169_VLAN
- struct vlan_group *vlgrp;
-#endif
struct mdio_ops {
void (*write)(void __iomem *, int, int);
@@ -1254,8 +1251,6 @@ static int rtl8169_set_rx_csum(struct net_device *dev, u32 data)
return 0;
}
-#ifdef CONFIG_R8169_VLAN
-
static inline u32 rtl8169_tx_vlan_tag(struct rtl8169_private *tp,
struct sk_buff *skb)
{
@@ -1263,64 +1258,37 @@ static inline u32 rtl8169_tx_vlan_tag(struct rtl8169_private *tp,
TxVlanTag | swab16(vlan_tx_tag_get(skb)) : 0x00;
}
-static void rtl8169_vlan_rx_register(struct net_device *dev,
- struct vlan_group *grp)
+#define NETIF_F_HW_VLAN_TX_RX (NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX)
+
+static void rtl8169_vlan_mode(struct net_device *dev)
{
struct rtl8169_private *tp = netdev_priv(dev);
void __iomem *ioaddr = tp->mmio_addr;
unsigned long flags;
spin_lock_irqsave(&tp->lock, flags);
- tp->vlgrp = grp;
- /*
- * Do not disable RxVlan on 8110SCd.
- */
- if (tp->vlgrp || (tp->mac_version == RTL_GIGA_MAC_VER_05))
+ if (dev->features & NETIF_F_HW_VLAN_RX)
tp->cp_cmd |= RxVlan;
else
tp->cp_cmd &= ~RxVlan;
RTL_W16(CPlusCmd, tp->cp_cmd);
+ /* PCI commit */
RTL_R16(CPlusCmd);
spin_unlock_irqrestore(&tp->lock, flags);
+
+ dev->vlan_features = dev->features &~ NETIF_F_HW_VLAN_TX_RX;
}
-static int rtl8169_rx_vlan_skb(struct rtl8169_private *tp, struct RxDesc *desc,
- struct sk_buff *skb, int polling)
+static void rtl8169_rx_vlan_tag(struct RxDesc *desc, struct sk_buff *skb)
{
u32 opts2 = le32_to_cpu(desc->opts2);
- struct vlan_group *vlgrp = tp->vlgrp;
- int ret;
- if (vlgrp && (opts2 & RxVlanTag)) {
- u16 vtag = swab16(opts2 & 0xffff);
+ if (opts2 & RxVlanTag)
+ __vlan_hwaccel_put_tag(skb, swab16(opts2 & 0xffff));
- if (likely(polling))
- vlan_gro_receive(&tp->napi, vlgrp, vtag, skb);
- else
- __vlan_hwaccel_rx(skb, vlgrp, vtag, polling);
- ret = 0;
- } else
- ret = -1;
desc->opts2 = 0;
- return ret;
-}
-
-#else /* !CONFIG_R8169_VLAN */
-
-static inline u32 rtl8169_tx_vlan_tag(struct rtl8169_private *tp,
- struct sk_buff *skb)
-{
- return 0;
-}
-
-static int rtl8169_rx_vlan_skb(struct rtl8169_private *tp, struct RxDesc *desc,
- struct sk_buff *skb, int polling)
-{
- return -1;
}
-#endif
-
static int rtl8169_gset_tbi(struct net_device *dev, struct ethtool_cmd *cmd)
{
struct rtl8169_private *tp = netdev_priv(dev);
@@ -1491,6 +1459,28 @@ static void rtl8169_get_strings(struct net_device *dev, u32 stringset, u8 *data)
}
}
+static int rtl8169_set_flags(struct net_device *dev, u32 data)
+{
+ struct rtl8169_private *tp = netdev_priv(dev);
+ unsigned long old_feat = dev->features;
+ int rc;
+
+ if ((tp->mac_version == RTL_GIGA_MAC_VER_05) &&
+ !(data & ETH_FLAG_RXVLAN)) {
+ netif_info(tp, drv, dev, "8110SCd requires hardware Rx VLAN\n");
+ return -EINVAL;
+ }
+
+ rc = ethtool_op_set_flags(dev, data, ETH_FLAG_TXVLAN | ETH_FLAG_RXVLAN);
+ if (rc)
+ return rc;
+
+ if ((old_feat ^ dev->features) & NETIF_F_HW_VLAN_RX)
+ rtl8169_vlan_mode(dev);
+
+ return 0;
+}
+
static const struct ethtool_ops rtl8169_ethtool_ops = {
.get_drvinfo = rtl8169_get_drvinfo,
.get_regs_len = rtl8169_get_regs_len,
@@ -1510,6 +1500,8 @@ static const struct ethtool_ops rtl8169_ethtool_ops = {
.get_strings = rtl8169_get_strings,
.get_sset_count = rtl8169_get_sset_count,
.get_ethtool_stats = rtl8169_get_ethtool_stats,
+ .set_flags = rtl8169_set_flags,
+ .get_flags = ethtool_op_get_flags,
};
static void rtl8169_get_mac_version(struct rtl8169_private *tp,
@@ -2792,9 +2784,6 @@ static const struct net_device_ops rtl8169_netdev_ops = {
.ndo_set_mac_address = rtl_set_mac_address,
.ndo_do_ioctl = rtl8169_ioctl,
.ndo_set_multicast_list = rtl_set_rx_mode,
-#ifdef CONFIG_R8169_VLAN
- .ndo_vlan_rx_register = rtl8169_vlan_rx_register,
-#endif
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = rtl8169_netpoll,
#endif
@@ -3086,6 +3075,13 @@ rtl8169_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Identify chip attached to board */
rtl8169_get_mac_version(tp, ioaddr);
+ /*
+ * Pretend we are using VLANs; This bypasses a nasty bug where
+ * Interrupts stop flowing on high load on 8110SCd controllers.
+ */
+ if (tp->mac_version == RTL_GIGA_MAC_VER_05)
+ tp->cp_cmd |= RxVlan;
+
rtl_init_mdio_ops(tp);
rtl_init_pll_power_ops(tp);
@@ -3154,10 +3150,7 @@ rtl8169_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
netif_napi_add(dev, &tp->napi, rtl8169_poll, R8169_NAPI_WEIGHT);
-#ifdef CONFIG_R8169_VLAN
- dev->features |= NETIF_F_HW_VLAN_TX | NETIF_F_HW_VLAN_RX;
-#endif
- dev->features |= NETIF_F_GRO;
+ dev->features |= NETIF_F_HW_VLAN_TX_RX | NETIF_F_GRO;
tp->intr_mask = 0xffff;
tp->hw_start = cfg->hw_start;
@@ -3273,12 +3266,7 @@ static int rtl8169_open(struct net_device *dev)
rtl8169_init_phy(dev, tp);
- /*
- * Pretend we are using VLANs; This bypasses a nasty bug where
- * Interrupts stop flowing on high load on 8110SCd controllers.
- */
- if (tp->mac_version == RTL_GIGA_MAC_VER_05)
- RTL_W16(CPlusCmd, RTL_R16(CPlusCmd) | RxVlan);
+ rtl8169_vlan_mode(dev);
rtl_pll_power_up(tp);
@@ -4589,12 +4577,12 @@ static int rtl8169_rx_interrupt(struct net_device *dev,
skb_put(skb, pkt_size);
skb->protocol = eth_type_trans(skb, dev);
- if (rtl8169_rx_vlan_skb(tp, desc, skb, polling) < 0) {
- if (likely(polling))
- napi_gro_receive(&tp->napi, skb);
- else
- netif_rx(skb);
- }
+ rtl8169_rx_vlan_tag(desc, skb);
+
+ if (likely(polling))
+ napi_gro_receive(&tp->napi, skb);
+ else
+ netif_rx(skb);
dev->stats.rx_bytes += pkt_size;
dev->stats.rx_packets++;
^ permalink raw reply related
* Re: ICMP reply uses wrong source address as destinatio
From: Eric Dumazet @ 2011-03-01 10:27 UTC (permalink / raw)
To: Anders Nilsson Plymoth; +Cc: netdev
In-Reply-To: <AANLkTinOCp3e6tiWgesPtKYwB7Mj=Ch6Ccm1xeSFLcpD@mail.gmail.com>
Le mardi 01 mars 2011 à 09:49 +0100, Anders Nilsson Plymoth a écrit :
> Dear linux kernel enthusiasts,
>
> I came upon an issue where ICMP reply packets were issued towards the
> IP address of the receiving interface, rather than the source IP
> address.
> Looking at the kernel code, I saw that this is caused by the following
> line in net/ipv4/icmp.c function icmp_reply:
>
> daddr = ipc.addr = rt->rt_src;
>
> For most cases the original line of code is ok, but in some situations
> doesn't arrive to the kernel from the network device, but through some
> other mechanism such as a userspace application. In these cases the
> receiving device in the skb appears to be the loopback interface, not
> a physical device. icmp_reply will thus issue the reply to the
> loopback IP address, rather than the source IP address as it should.
>
> While googling to see if this issue have been submitted, I found this
> two posts that address the same problem:
> h**p://www.mail-archive.com/linux-kernel@vger.kernel.org/msg209746.html
> h**p://www.mail-archive.com/linux-kernel@vger.kernel.org/msg208272.html
> Some of the questions there are easy to answer; such as this doesn't
> affect DNAT, and if source address is not set then you can' reply
> anyway.
>
> As to the statement:
> "... which IP address should be used as the source
>
> 1. the destination address of the packet that generated the message
>
> or.
>
> 2. the IP address that the machine would use by default if the machine
> were to generate a new connection to the destination."
>
> These may be relevant questions, but the ICMP RFC clearly states the
> answer is 1. 2. may seem relevant to multi-homing, but its not the
> role of the ICMP reply to resolve multi-homing issues.
>
> The following code will correct the issue.
>
> {
> struct iphdr *ip = ip_hdr(skb);
> daddr = ipc.addr = ip->saddr;
> }
>
> The only functions that use icmp_reply are icmp_echo and
> icmp_timestamp, and this change do not modify their behavior. After
> extensive testing, in regular setups and DNATed situations, I can
> verify this change works as intended.
I suspect you could respin Lepton Wu patch (from msg209746.html) on top
of net-next-2.6 and add your "Acked-by or Tested-by"
Keep in mind changelog should be very very detailed because patch was
delayed because of this.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 10:33 UTC (permalink / raw)
To: Thomas Graf
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301101955.GI9763@canuck.infradead.org>
Le mardi 01 mars 2011 à 05:19 -0500, Thomas Graf a écrit :
> On Mon, Feb 28, 2011 at 06:07:49PM +0100, Eric Dumazet wrote:
> > > The numbers differ a lot depending on the architecture we test on.
> > >
> > > F.e. on a 12 core AMD with 2 NUMA nodes:
> > >
> > > 2.6.32 named -n 1: 37.0kqps
> > > named: 3.8kqps (yes, no joke, the socket receive buffer is
> > > always full and the kernel drops pkts)
> >
> > Yes, this old kernel miss commit c377411f2494a93 added in 2.6.35
> > (net: sk_add_backlog() take rmem_alloc into account)
>
> I retested with net-2.6 w/o Herbert's patch:
>
> named -n 1: 36.9kqps
> named: 16.2kqps
Thats better ;)
You could do "cat /proc/net/udp" to check if drops occur on port 53
socket (last column)
But maybe your queryperf is limited to few queries in flight (default is
20 per queryperf instance)
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 11:07 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298975602.3284.13.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 11:33:22AM +0100, Eric Dumazet wrote:
> > I retested with net-2.6 w/o Herbert's patch:
> >
> > named -n 1: 36.9kqps
> > named: 16.2kqps
>
> Thats better ;)
>
> You could do "cat /proc/net/udp" to check if drops occur on port 53
> socket (last column)
>
> But maybe your queryperf is limited to few queries in flight (default is
> 20 per queryperf instance)
I tried -q 10, 20, 30, 50, 100. Starting with 20 I see drops, at q=50
queryperf reports 99% drops.
I also tested again on the Intel machine that did ~650kqps using SO_REUSEPORT.
net-2.6: 106.3kqps, 101.2kqps
net-2.6 lockless udp: 251.7kqps, 250.4kqps
I see drops in both test cases occur so I believe the rate supplied by the
clients is sufficient.
The difference is obvious when looking at top and mpstat:
UDP lockless (250kqps):
Cpu0 : 46.4%us, 28.8%sy, 0.0%ni, 24.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 2.0%us, 1.3%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 93.6%si, 0.0%st
Cpu2 : 45.9%us, 28.2%sy, 0.0%ni, 25.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu3 : 50.0%us, 21.6%sy, 0.0%ni, 28.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu4 : 45.4%us, 27.8%sy, 0.0%ni, 26.5%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
Cpu5 : 50.7%us, 23.2%sy, 0.0%ni, 26.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu6 : 45.2%us, 28.9%sy, 0.0%ni, 25.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu7 : 50.5%us, 22.0%sy, 0.0%ni, 27.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu8 : 45.3%us, 29.3%sy, 0.0%ni, 25.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu9 : 50.8%us, 20.8%sy, 0.0%ni, 28.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu10 : 46.1%us, 27.8%sy, 0.0%ni, 26.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu11 : 27.2%us, 11.3%sy, 0.0%ni, 3.3%id, 0.0%wa, 0.0%hi, 58.1%si, 0.0%st
05:50:44 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
05:50:44 AM all 23.86 0.00 13.02 0.22 0.00 6.98 0.00 0.00 55.92
05:50:44 AM 0 26.16 0.00 17.20 0.73 0.00 0.30 0.00 0.00 55.61
05:50:44 AM 1 2.36 0.00 2.11 0.70 0.00 51.97 0.00 0.00 42.87
05:50:44 AM 2 25.90 0.00 16.38 0.32 0.00 0.03 0.00 0.00 57.36
05:50:44 AM 3 28.26 0.00 12.73 0.27 0.00 0.02 0.00 0.00 58.73
05:50:44 AM 4 25.63 0.00 16.04 0.13 0.00 0.03 0.00 0.00 58.17
05:50:44 AM 5 28.19 0.00 12.54 0.17 0.00 0.01 0.00 0.00 59.09
05:50:44 AM 6 25.28 0.00 15.21 0.02 0.00 1.95 0.00 0.00 57.54
05:50:44 AM 7 28.34 0.00 12.40 0.10 0.00 0.01 0.00 0.00 59.14
05:50:44 AM 8 25.70 0.00 15.91 0.01 0.00 0.02 0.00 0.00 58.37
05:50:44 AM 9 28.31 0.00 12.56 0.11 0.00 0.01 0.00 0.00 59.01
05:50:44 AM 10 25.85 0.00 15.65 0.01 0.00 0.02 0.00 0.00 58.47
05:50:44 AM 11 16.11 0.00 7.44 0.10 0.00 29.87 0.00 0.00 46.49
SO_REUSEPORT test (doing 640kqps):
Cpu0 : 57.3%us, 26.5%sy, 0.0%ni, 3.3%id, 0.0%wa, 0.0%hi, 12.9%si, 0.0%st
Cpu1 : 25.7%us, 10.0%sy, 0.0%ni, 0.3%id, 0.0%wa, 0.0%hi, 64.0%si, 0.0%st
Cpu2 : 56.3%us, 28.8%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 11.9%si, 0.0%st
Cpu3 : 29.1%us, 10.9%sy, 0.0%ni, 1.3%id, 0.0%wa, 0.0%hi, 58.6%si, 0.0%st
Cpu4 : 57.3%us, 28.5%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.0%hi, 11.9%si, 0.0%st
Cpu5 : 64.8%us, 22.6%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 9.6%si, 0.0%st
Cpu6 : 59.0%us, 26.7%sy, 0.0%ni, 2.7%id, 0.0%wa, 0.0%hi, 11.7%si, 0.0%st
Cpu7 : 64.1%us, 22.3%sy, 0.0%ni, 3.7%id, 0.0%wa, 0.0%hi, 10.0%si, 0.0%st
Cpu8 : 57.6%us, 27.5%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 11.9%si, 0.0%st
Cpu9 : 65.2%us, 22.2%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.0%hi, 10.3%si, 0.0%st
Cpu10 : 56.9%us, 28.3%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 11.8%si, 0.0%st
Cpu11 : 40.2%us, 14.6%sy, 0.0%ni, 2.3%id, 0.0%wa, 0.0%hi, 42.9%si, 0.0%st
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 11:13 UTC (permalink / raw)
To: Thomas Graf
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301110708.GJ9763@canuck.infradead.org>
Le mardi 01 mars 2011 à 06:07 -0500, Thomas Graf a écrit :
> On Tue, Mar 01, 2011 at 11:33:22AM +0100, Eric Dumazet wrote:
> > > I retested with net-2.6 w/o Herbert's patch:
> > >
> > > named -n 1: 36.9kqps
> > > named: 16.2kqps
> >
> > Thats better ;)
> >
> > You could do "cat /proc/net/udp" to check if drops occur on port 53
> > socket (last column)
> >
> > But maybe your queryperf is limited to few queries in flight (default is
> > 20 per queryperf instance)
>
> I tried -q 10, 20, 30, 50, 100. Starting with 20 I see drops, at q=50
> queryperf reports 99% drops.
>
> I also tested again on the Intel machine that did ~650kqps using SO_REUSEPORT.
>
> net-2.6: 106.3kqps, 101.2kqps
> net-2.6 lockless udp: 251.7kqps, 250.4kqps
>
> I see drops in both test cases occur so I believe the rate supplied by the
> clients is sufficient.
>
> The difference is obvious when looking at top and mpstat:
>
> UDP lockless (250kqps):
>
> Cpu0 : 46.4%us, 28.8%sy, 0.0%ni, 24.8%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu1 : 2.0%us, 1.3%sy, 0.0%ni, 3.0%id, 0.0%wa, 0.0%hi, 93.6%si, 0.0%st
> Cpu2 : 45.9%us, 28.2%sy, 0.0%ni, 25.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu3 : 50.0%us, 21.6%sy, 0.0%ni, 28.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu4 : 45.4%us, 27.8%sy, 0.0%ni, 26.5%id, 0.0%wa, 0.0%hi, 0.3%si, 0.0%st
> Cpu5 : 50.7%us, 23.2%sy, 0.0%ni, 26.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu6 : 45.2%us, 28.9%sy, 0.0%ni, 25.9%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu7 : 50.5%us, 22.0%sy, 0.0%ni, 27.5%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu8 : 45.3%us, 29.3%sy, 0.0%ni, 25.4%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu9 : 50.8%us, 20.8%sy, 0.0%ni, 28.3%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu10 : 46.1%us, 27.8%sy, 0.0%ni, 26.1%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st
> Cpu11 : 27.2%us, 11.3%sy, 0.0%ni, 3.3%id, 0.0%wa, 0.0%hi, 58.1%si, 0.0%st
Its a bit strange two cpus spend time in softirq, unless you have two
queryperf sources, and a multiqueue NIC, or maybe you use two NICS ?
Mind use "perf top -C 1" and "perf top -C 11" to check what these cpus
do ?
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 11:27 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298977984.3284.15.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 12:13:04PM +0100, Eric Dumazet wrote:
> Its a bit strange two cpus spend time in softirq, unless you have two
> queryperf sources, and a multiqueue NIC, or maybe you use two NICS ?
one NIC, 2 clients (12 instances per client)
[root@hp-bl460cg7-01 ~]# cat /sys/class/net/eth0/queues/rx-0/rps_cpus
00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
[root@hp-bl460cg7-01 ~]# netstat -s | grep err
1781377 packet receive errors
> Mind use "perf top -C 1" and "perf top -C 11" to check what these cpus
> do ?
--------------------------------------------------------------------------------------------------------------------
PerfTop: 16198 irqs/sec kernel:99.1% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
--------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ ___________________________ ___________________________________________________________
51675.00 33.2% _raw_spin_unlock_irqrestore [kernel.kallsyms]
12426.00 8.0% clflush_cache_range [kernel.kallsyms]
5511.00 3.5% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
4567.00 2.9% __udp4_lib_lookup [kernel.kallsyms]
3981.00 2.6% __kmalloc_node_track_caller [kernel.kallsyms]
3975.00 2.6% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
3725.00 2.4% sk_run_filter [kernel.kallsyms]
3606.00 2.3% get_page_from_freelist [kernel.kallsyms]
3178.00 2.0% __domain_mapping [kernel.kallsyms]
3122.00 2.0% kmem_cache_alloc_node [kernel.kallsyms]
2839.00 1.8% sock_queue_rcv_skb [kernel.kallsyms]
2246.00 1.4% __netif_receive_skb [kernel.kallsyms]
2245.00 1.4% nf_iterate [kernel.kallsyms]
2081.00 1.3% __udp4_lib_rcv [kernel.kallsyms]
2042.00 1.3% ipt_do_table [kernel.kallsyms]
1901.00 1.2% _raw_spin_lock [kernel.kallsyms]
1856.00 1.2% __alloc_skb [kernel.kallsyms]
1645.00 1.1% read_tsc [kernel.kallsyms]
1562.00 1.0% nf_ct_tuple_equal [kernel.kallsyms]
1562.00 1.0% ip_rcv [kernel.kallsyms]
1495.00 1.0% __nf_conntrack_find_get [kernel.kallsyms]
1477.00 0.9% sock_def_readable [kernel.kallsyms]
1363.00 0.9% find_first_bit [kernel.kallsyms]
1360.00 0.9% domain_get_iommu [kernel.kallsyms]
1255.00 0.8% udp_queue_rcv_skb [kernel.kallsyms]
1174.00 0.8% xfrm4_policy_check.clone.0 [kernel.kallsyms]
1138.00 0.7% hash_conntrack_raw [kernel.kallsyms]
1000.00 0.6% intel_unmap_page [kernel.kallsyms]
959.00 0.6% load_pointer [kernel.kallsyms]
957.00 0.6% sock_flag [kernel.kallsyms]
938.00 0.6% nf_conntrack_in [kernel.kallsyms]
891.00 0.6% _local_bh_enable_ip [kernel.kallsyms]
884.00 0.6% eth_type_trans [kernel.kallsyms]
832.00 0.5% be_post_rx_frags /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
829.00 0.5% __alloc_pages_nodemask [kernel.kallsyms]
813.00 0.5% kmem_cache_alloc [kernel.kallsyms]
802.00 0.5% netif_receive_skb [kernel.kallsyms]
802.00 0.5% ip_route_input_common [kernel.kallsyms]
723.00 0.5% nf_ct_get_tuple [kernel.kallsyms]
720.00 0.5% __intel_map_single [kernel.kallsyms]
720.00 0.5% udp_error [kernel.kallsyms]
--------------------------------------------------------------------------------------------------------------------
PerfTop: 16360 irqs/sec kernel:72.6% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
--------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ ___________________________________________________________
16993.00 32.4% _raw_spin_unlock_irqrestore [kernel.kallsyms]
5833.00 11.1% clflush_cache_range [kernel.kallsyms]
3315.00 6.3% be_tx_compl_process /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
1818.00 3.5% kmem_cache_free [kernel.kallsyms]
1415.00 2.7% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
1090.00 2.1% be_poll_tx_mcc /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
811.00 1.5% skb_release_head_state [kernel.kallsyms]
772.00 1.5% skb_release_data [kernel.kallsyms]
712.00 1.4% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
703.00 1.3% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
695.00 1.3% dma_pte_clear_range [kernel.kallsyms]
618.00 1.2% kfree_skb [kernel.kallsyms]
597.00 1.1% kfree [kernel.kallsyms]
553.00 1.1% intel_unmap_page [kernel.kallsyms]
531.00 1.0% __do_softirq [kernel.kallsyms]
504.00 1.0% isc_stats_increment /usr/lib64/libisc.so.62.0.1
397.00 0.8% virt_to_head_page [kernel.kallsyms]
306.00 0.6% _raw_spin_lock [kernel.kallsyms]
270.00 0.5% domain_get_iommu [kernel.kallsyms]
256.00 0.5% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
233.00 0.4% find_first_bit [kernel.kallsyms]
222.00 0.4% dns_name_equal /usr/lib64/libdns.so.69.0.1
218.00 0.4% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
207.00 0.4% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
196.00 0.4% dns_acl_match /usr/lib64/libdns.so.69.0.1
194.00 0.4% dma_pte_free_pagetable [kernel.kallsyms]
192.00 0.4% dns_name_getlabelsequence /usr/lib64/libdns.so.69.0.1
^ permalink raw reply
* [PATCH] Fix possible null pointer dereference in ariadne.c
From: huzaifas @ 2011-03-01 11:33 UTC (permalink / raw)
To: netdev; +Cc: geert, tj, davem, eric.dumazet, Huzaifa Sidhpurwala
From: Huzaifa Sidhpurwala <huzaifas@redhat.com>
Make sure dev exists before referencing it
Signed-off-by: Huzaifa Sidhpurwala <huzaifas@redhat.com>
---
drivers/net/ariadne.c | 3 ++-
1 files changed, 2 insertions(+), 1 deletions(-)
diff --git a/drivers/net/ariadne.c b/drivers/net/ariadne.c
index 39214e5..fc3c762 100644
--- a/drivers/net/ariadne.c
+++ b/drivers/net/ariadne.c
@@ -420,7 +420,7 @@ static inline void ariadne_reset(struct net_device *dev)
static irqreturn_t ariadne_interrupt(int irq, void *data)
{
struct net_device *dev = (struct net_device *)data;
- volatile struct Am79C960 *lance = (struct Am79C960*)dev->base_addr;
+ volatile struct Am79C960 *lance;
struct ariadne_private *priv;
int csr0, boguscnt;
int handled = 0;
@@ -430,6 +430,7 @@ static irqreturn_t ariadne_interrupt(int irq, void *data)
return IRQ_NONE;
}
+ *lance = (struct Am79C960*)dev->base_addr;
lance->RAP = CSR0; /* PCnet-ISA Controller Status */
if (!(lance->RDP & INTR)) /* Check if any interrupt has been */
--
1.7.3.4
^ permalink raw reply related
* [PATCH] vhost: Cleanup vhost.c and net.c
From: Krishna Kumar @ 2011-03-01 11:36 UTC (permalink / raw)
To: mst; +Cc: netdev, Krishna Kumar
Minor cleanup of vhost.c and net.c to match coding style.
Signed-off-by: Krishna Kumar <krkumar2@in.ibm.com>
---
drivers/vhost/net.c | 19 +++++++++-----
drivers/vhost/vhost.c | 53 +++++++++++++++++++++++++++-------------
2 files changed, 49 insertions(+), 23 deletions(-)
diff -ruNp org/drivers/vhost/net.c new/drivers/vhost/net.c
--- org/drivers/vhost/net.c 2011-03-01 13:33:25.000000000 +0530
+++ new/drivers/vhost/net.c 2011-03-01 13:36:44.000000000 +0530
@@ -60,6 +60,7 @@ static int move_iovec_hdr(struct iovec *
{
int seg = 0;
size_t size;
+
while (len && seg < iov_count) {
size = min(from->iov_len, len);
to->iov_base = from->iov_base;
@@ -79,6 +80,7 @@ static void copy_iovec_hdr(const struct
{
int seg = 0;
size_t size;
+
while (len && seg < iovcount) {
size = min(from->iov_len, len);
to->iov_base = from->iov_base;
@@ -296,17 +298,16 @@ static void handle_rx_big(struct vhost_n
.msg_iov = vq->iov,
.msg_flags = MSG_DONTWAIT,
};
-
struct virtio_net_hdr hdr = {
.flags = 0,
.gso_type = VIRTIO_NET_HDR_GSO_NONE
};
-
size_t len, total_len = 0;
int err;
size_t hdr_size;
/* TODO: check that we are running from vhost_worker? */
struct socket *sock = rcu_dereference_check(vq->private_data, 1);
+
if (!sock || skb_queue_empty(&sock->sk->sk_receive_queue))
return;
@@ -405,18 +406,17 @@ static void handle_rx_mergeable(struct v
.msg_iov = vq->iov,
.msg_flags = MSG_DONTWAIT,
};
-
struct virtio_net_hdr_mrg_rxbuf hdr = {
.hdr.flags = 0,
.hdr.gso_type = VIRTIO_NET_HDR_GSO_NONE
};
-
size_t total_len = 0;
int err, headcount;
size_t vhost_hlen, sock_hlen;
size_t vhost_len, sock_len;
/* TODO: check that we are running from vhost_worker? */
struct socket *sock = rcu_dereference_check(vq->private_data, 1);
+
if (!sock || skb_queue_empty(&sock->sk->sk_receive_queue))
return;
@@ -654,6 +654,7 @@ static struct socket *get_raw_socket(int
} uaddr;
int uaddr_len = sizeof uaddr, r;
struct socket *sock = sockfd_lookup(fd, &r);
+
if (!sock)
return ERR_PTR(-ENOTSOCK);
@@ -682,6 +683,7 @@ static struct socket *get_tap_socket(int
{
struct file *file = fget(fd);
struct socket *sock;
+
if (!file)
return ERR_PTR(-EBADF);
sock = tun_get_socket(file);
@@ -696,6 +698,7 @@ static struct socket *get_tap_socket(int
static struct socket *get_socket(int fd)
{
struct socket *sock;
+
/* special case to disable backend */
if (fd == -1)
return NULL;
@@ -741,9 +744,9 @@ static long vhost_net_set_backend(struct
oldsock = rcu_dereference_protected(vq->private_data,
lockdep_is_held(&vq->mutex));
if (sock != oldsock) {
- vhost_net_disable_vq(n, vq);
- rcu_assign_pointer(vq->private_data, sock);
- vhost_net_enable_vq(n, vq);
+ vhost_net_disable_vq(n, vq);
+ rcu_assign_pointer(vq->private_data, sock);
+ vhost_net_enable_vq(n, vq);
}
mutex_unlock(&vq->mutex);
@@ -768,6 +771,7 @@ static long vhost_net_reset_owner(struct
struct socket *tx_sock = NULL;
struct socket *rx_sock = NULL;
long err;
+
mutex_lock(&n->dev.mutex);
err = vhost_dev_check_owner(&n->dev);
if (err)
@@ -829,6 +833,7 @@ static long vhost_net_ioctl(struct file
struct vhost_vring_file backend;
u64 features;
int r;
+
switch (ioctl) {
case VHOST_NET_SET_BACKEND:
if (copy_from_user(&backend, argp, sizeof backend))
diff -ruNp org/drivers/vhost/vhost.c new/drivers/vhost/vhost.c
--- org/drivers/vhost/vhost.c 2011-01-19 20:01:29.000000000 +0530
+++ new/drivers/vhost/vhost.c 2011-03-01 13:36:58.000000000 +0530
@@ -41,8 +41,8 @@ static void vhost_poll_func(struct file
poll_table *pt)
{
struct vhost_poll *poll;
- poll = container_of(pt, struct vhost_poll, table);
+ poll = container_of(pt, struct vhost_poll, table);
poll->wqh = wqh;
add_wait_queue(wqh, &poll->wait);
}
@@ -85,6 +85,7 @@ void vhost_poll_init(struct vhost_poll *
void vhost_poll_start(struct vhost_poll *poll, struct file *file)
{
unsigned long mask;
+
mask = file->f_op->poll(file, &poll->table);
if (mask)
vhost_poll_wakeup(&poll->wait, 0, 0, (void *)mask);
@@ -101,6 +102,7 @@ static bool vhost_work_seq_done(struct v
unsigned seq)
{
int left;
+
spin_lock_irq(&dev->work_lock);
left = seq - work->done_seq;
spin_unlock_irq(&dev->work_lock);
@@ -222,6 +224,7 @@ static int vhost_worker(void *data)
static long vhost_dev_alloc_iovecs(struct vhost_dev *dev)
{
int i;
+
for (i = 0; i < dev->nvqs; ++i) {
dev->vqs[i].indirect = kmalloc(sizeof *dev->vqs[i].indirect *
UIO_MAXIOV, GFP_KERNEL);
@@ -235,6 +238,7 @@ static long vhost_dev_alloc_iovecs(struc
goto err_nomem;
}
return 0;
+
err_nomem:
for (; i >= 0; --i) {
kfree(dev->vqs[i].indirect);
@@ -247,6 +251,7 @@ err_nomem:
static void vhost_dev_free_iovecs(struct vhost_dev *dev)
{
int i;
+
for (i = 0; i < dev->nvqs; ++i) {
kfree(dev->vqs[i].indirect);
dev->vqs[i].indirect = NULL;
@@ -296,26 +301,28 @@ long vhost_dev_check_owner(struct vhost_
}
struct vhost_attach_cgroups_struct {
- struct vhost_work work;
- struct task_struct *owner;
- int ret;
+ struct vhost_work work;
+ struct task_struct *owner;
+ int ret;
};
static void vhost_attach_cgroups_work(struct vhost_work *work)
{
- struct vhost_attach_cgroups_struct *s;
- s = container_of(work, struct vhost_attach_cgroups_struct, work);
- s->ret = cgroup_attach_task_all(s->owner, current);
+ struct vhost_attach_cgroups_struct *s;
+
+ s = container_of(work, struct vhost_attach_cgroups_struct, work);
+ s->ret = cgroup_attach_task_all(s->owner, current);
}
static int vhost_attach_cgroups(struct vhost_dev *dev)
{
- struct vhost_attach_cgroups_struct attach;
- attach.owner = current;
- vhost_work_init(&attach.work, vhost_attach_cgroups_work);
- vhost_work_queue(dev, &attach.work);
- vhost_work_flush(dev, &attach.work);
- return attach.ret;
+ struct vhost_attach_cgroups_struct attach;
+
+ attach.owner = current;
+ vhost_work_init(&attach.work, vhost_attach_cgroups_work);
+ vhost_work_queue(dev, &attach.work);
+ vhost_work_flush(dev, &attach.work);
+ return attach.ret;
}
/* Caller should have device mutex */
@@ -323,11 +330,13 @@ static long vhost_dev_set_owner(struct v
{
struct task_struct *worker;
int err;
+
/* Is there an owner already? */
if (dev->mm) {
err = -EBUSY;
goto err_mm;
}
+
/* No owner, become one */
dev->mm = get_task_mm(current);
worker = kthread_create(vhost_worker, dev, "vhost-%d", current->pid);
@@ -380,6 +389,7 @@ long vhost_dev_reset_owner(struct vhost_
void vhost_dev_cleanup(struct vhost_dev *dev)
{
int i;
+
for (i = 0; i < dev->nvqs; ++i) {
if (dev->vqs[i].kick && dev->vqs[i].handle_kick) {
vhost_poll_stop(&dev->vqs[i].poll);
@@ -421,6 +431,7 @@ void vhost_dev_cleanup(struct vhost_dev
static int log_access_ok(void __user *log_base, u64 addr, unsigned long sz)
{
u64 a = addr / VHOST_PAGE_SIZE / 8;
+
/* Make sure 64 bit math will not overflow. */
if (a > ULONG_MAX - (unsigned long)log_base ||
a + (unsigned long)log_base > ULONG_MAX)
@@ -461,6 +472,7 @@ static int memory_access_ok(struct vhost
int log_all)
{
int i;
+
for (i = 0; i < d->nvqs; ++i) {
int ok;
mutex_lock(&d->vqs[i].mutex);
@@ -527,6 +539,7 @@ static long vhost_set_memory(struct vhos
{
struct vhost_memory mem, *newmem, *oldmem;
unsigned long size = offsetof(struct vhost_memory, regions);
+
if (copy_from_user(&mem, m, size))
return -EFAULT;
if (mem.padding)
@@ -544,7 +557,8 @@ static long vhost_set_memory(struct vhos
return -EFAULT;
}
- if (!memory_access_ok(d, newmem, vhost_has_feature(d, VHOST_F_LOG_ALL))) {
+ if (!memory_access_ok(d, newmem,
+ vhost_has_feature(d, VHOST_F_LOG_ALL))) {
kfree(newmem);
return -EFAULT;
}
@@ -560,6 +574,7 @@ static int init_used(struct vhost_virtqu
struct vring_used __user *used)
{
int r = put_user(vq->used_flags, &used->flags);
+
if (r)
return r;
return get_user(vq->last_used_idx, &used->idx);
@@ -849,6 +864,7 @@ static const struct vhost_memory_region
{
struct vhost_memory_region *reg;
int i;
+
/* linear search is not brilliant, but we really have on the order of 6
* regions in practice */
for (i = 0; i < mem->nregions; ++i) {
@@ -871,6 +887,7 @@ static int set_bit_to_user(int nr, void
void *base;
int bit = nr + (log % PAGE_SIZE) * 8;
int r;
+
r = get_user_pages_fast(log, 1, 1, &page);
if (r < 0)
return r;
@@ -888,6 +905,7 @@ static int log_write(void __user *log_ba
{
u64 write_page = write_address / VHOST_PAGE_SIZE;
int r;
+
if (!write_length)
return 0;
write_length += write_address % VHOST_PAGE_SIZE;
@@ -1037,8 +1055,8 @@ static int get_indirect(struct vhost_dev
i, count);
return -EINVAL;
}
- if (unlikely(memcpy_fromiovec((unsigned char *)&desc, vq->indirect,
- sizeof desc))) {
+ if (unlikely(memcpy_fromiovec((unsigned char *)&desc,
+ vq->indirect, sizeof desc))) {
vq_err(vq, "Failed indirect descriptor: idx %d, %zx\n",
i, (size_t)indirect->addr + i * sizeof desc);
return -EINVAL;
@@ -1317,6 +1335,7 @@ int vhost_add_used_n(struct vhost_virtqu
void vhost_signal(struct vhost_dev *dev, struct vhost_virtqueue *vq)
{
__u16 flags;
+
/* Flush out used index updates. This is paired
* with the barrier that the Guest executes when enabling
* interrupts. */
@@ -1361,6 +1380,7 @@ bool vhost_enable_notify(struct vhost_vi
{
u16 avail_idx;
int r;
+
if (!(vq->used_flags & VRING_USED_F_NO_NOTIFY))
return false;
vq->used_flags &= ~VRING_USED_F_NO_NOTIFY;
@@ -1387,6 +1407,7 @@ bool vhost_enable_notify(struct vhost_vi
void vhost_disable_notify(struct vhost_virtqueue *vq)
{
int r;
+
if (vq->used_flags & VRING_USED_F_NO_NOTIFY)
return;
vq->used_flags |= VRING_USED_F_NO_NOTIFY;
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Eric Dumazet @ 2011-03-01 11:45 UTC (permalink / raw)
To: Thomas Graf
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301112759.GK9763@canuck.infradead.org>
Le mardi 01 mars 2011 à 06:27 -0500, Thomas Graf a écrit :
> On Tue, Mar 01, 2011 at 12:13:04PM +0100, Eric Dumazet wrote:
> > Its a bit strange two cpus spend time in softirq, unless you have two
> > queryperf sources, and a multiqueue NIC, or maybe you use two NICS ?
>
> one NIC, 2 clients (12 instances per client)
>
> [root@hp-bl460cg7-01 ~]# cat /sys/class/net/eth0/queues/rx-0/rps_cpus
> 00000000,00000000,00000000,00000000,00000000,00000000,00000000,00000000
>
> [root@hp-bl460cg7-01 ~]# netstat -s | grep err
> 1781377 packet receive errors
>
> > Mind use "perf top -C 1" and "perf top -C 11" to check what these cpus
> > do ?
>
Thanks that's really interesting
> --------------------------------------------------------------------------------------------------------------------
> PerfTop: 16198 irqs/sec kernel:99.1% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
> --------------------------------------------------------------------------------------------------------------------
>
> samples pcnt function DSO
> _______ _____ ___________________________ ___________________________________________________________
>
CPU 1 handles receives from your BENET NIC
(Its a bit strange, given this NIC should provide 4 rx queues). Load
could be split to two cpus in your case (two sources)
Try :
ethtool -S eth0 | grep rx_pk
rxq0: rx_pkts: ??
rxq1: rx_pkts: ??
rxq2: rx_pkts: ??
rxq3: rx_pkts: ??
rxq4: rx_pkts: ??
Its BE_HDR_LEN being 64, small UDP frames are too big to fit in skb
head.
> 51675.00 33.2% _raw_spin_unlock_irqrestore [kernel.kallsyms]
> 12426.00 8.0% clflush_cache_range [kernel.kallsyms]
> 5511.00 3.5% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 4567.00 2.9% __udp4_lib_lookup [kernel.kallsyms]
> 3981.00 2.6% __kmalloc_node_track_caller [kernel.kallsyms]
> 3975.00 2.6% get_rx_page_info /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 3725.00 2.4% sk_run_filter [kernel.kallsyms]
> 3606.00 2.3% get_page_from_freelist [kernel.kallsyms]
> 3178.00 2.0% __domain_mapping [kernel.kallsyms]
> 3122.00 2.0% kmem_cache_alloc_node [kernel.kallsyms]
> 2839.00 1.8% sock_queue_rcv_skb [kernel.kallsyms]
> 2246.00 1.4% __netif_receive_skb [kernel.kallsyms]
> 2245.00 1.4% nf_iterate [kernel.kallsyms]
> 2081.00 1.3% __udp4_lib_rcv [kernel.kallsyms]
> 2042.00 1.3% ipt_do_table [kernel.kallsyms]
> 1901.00 1.2% _raw_spin_lock [kernel.kallsyms]
> 1856.00 1.2% __alloc_skb [kernel.kallsyms]
> 1645.00 1.1% read_tsc [kernel.kallsyms]
> 1562.00 1.0% nf_ct_tuple_equal [kernel.kallsyms]
> 1562.00 1.0% ip_rcv [kernel.kallsyms]
> 1495.00 1.0% __nf_conntrack_find_get [kernel.kallsyms]
> 1477.00 0.9% sock_def_readable [kernel.kallsyms]
> 1363.00 0.9% find_first_bit [kernel.kallsyms]
> 1360.00 0.9% domain_get_iommu [kernel.kallsyms]
> 1255.00 0.8% udp_queue_rcv_skb [kernel.kallsyms]
> 1174.00 0.8% xfrm4_policy_check.clone.0 [kernel.kallsyms]
> 1138.00 0.7% hash_conntrack_raw [kernel.kallsyms]
> 1000.00 0.6% intel_unmap_page [kernel.kallsyms]
> 959.00 0.6% load_pointer [kernel.kallsyms]
> 957.00 0.6% sock_flag [kernel.kallsyms]
> 938.00 0.6% nf_conntrack_in [kernel.kallsyms]
> 891.00 0.6% _local_bh_enable_ip [kernel.kallsyms]
> 884.00 0.6% eth_type_trans [kernel.kallsyms]
> 832.00 0.5% be_post_rx_frags /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 829.00 0.5% __alloc_pages_nodemask [kernel.kallsyms]
> 813.00 0.5% kmem_cache_alloc [kernel.kallsyms]
> 802.00 0.5% netif_receive_skb [kernel.kallsyms]
> 802.00 0.5% ip_route_input_common [kernel.kallsyms]
> 723.00 0.5% nf_ct_get_tuple [kernel.kallsyms]
> 720.00 0.5% __intel_map_single [kernel.kallsyms]
> 720.00 0.5% udp_error [kernel.kallsyms]
>
> --------------------------------------------------------------------------------------------------------------------
> PerfTop: 16360 irqs/sec kernel:72.6% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
> --------------------------------------------------------------------------------------------------------------------
>
CPU 11 handles all TX completions : Its a potential bottleneck.
I might ressurect XPS patch ;)
> samples pcnt function DSO
> _______ _____ _____________________________ ___________________________________________________________
>
> 16993.00 32.4% _raw_spin_unlock_irqrestore [kernel.kallsyms]
> 5833.00 11.1% clflush_cache_range [kernel.kallsyms]
> 3315.00 6.3% be_tx_compl_process /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 1818.00 3.5% kmem_cache_free [kernel.kallsyms]
> 1415.00 2.7% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
> 1090.00 2.1% be_poll_tx_mcc /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
> 811.00 1.5% skb_release_head_state [kernel.kallsyms]
> 772.00 1.5% skb_release_data [kernel.kallsyms]
> 712.00 1.4% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
> 703.00 1.3% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
> 695.00 1.3% dma_pte_clear_range [kernel.kallsyms]
> 618.00 1.2% kfree_skb [kernel.kallsyms]
> 597.00 1.1% kfree [kernel.kallsyms]
> 553.00 1.1% intel_unmap_page [kernel.kallsyms]
> 531.00 1.0% __do_softirq [kernel.kallsyms]
> 504.00 1.0% isc_stats_increment /usr/lib64/libisc.so.62.0.1
> 397.00 0.8% virt_to_head_page [kernel.kallsyms]
> 306.00 0.6% _raw_spin_lock [kernel.kallsyms]
> 270.00 0.5% domain_get_iommu [kernel.kallsyms]
> 256.00 0.5% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
> 233.00 0.4% find_first_bit [kernel.kallsyms]
> 222.00 0.4% dns_name_equal /usr/lib64/libdns.so.69.0.1
> 218.00 0.4% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
> 207.00 0.4% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
> 196.00 0.4% dns_acl_match /usr/lib64/libdns.so.69.0.1
> 194.00 0.4% dma_pte_free_pagetable [kernel.kallsyms]
> 192.00 0.4% dns_name_getlabelsequence /usr/lib64/libdns.so.69.0.1
>
^ permalink raw reply
* skb->frag_list != NULL in start_xmit for device without NETIF_F_FRAGLIST
From: Ian Campbell @ 2011-03-01 11:46 UTC (permalink / raw)
To: netdev; +Cc: Paul Durrant
Hi,
We are seeing cases where Xen netback's start_xmit is being passed an
skb which has a ->frag_list, despite the driver not advertising the
NETIF_F_FRAGLIST feature.
Is this indicative of a problem somewhere? Are drivers expected to
handle a frag_list? grepping for frag_list in drivers/net it appears not
many drivers do anything with the frag_list.
The netback driver is bridged with a tg3 physical device and we think
the problematic skb's are originating on the wire.
The case we are actually seeing is with 2.6.32 + tg3 3.110g so obviously
the next step is to reproduce on a more modern kernel and the in-tree
driver and then to attempt to determine if the fault is in what the
physical interface's driver is passing up the stack or in the stack's
handling of those skbs. Any hints on where to look would be much
appreciated.
In particular I'm not sure where the frag_list is supposed to get
resolved in the case where dev_hard_start_xmit takes the netif_needs_gso
== true path and calls dev_gso_segment rather than taking the
__skb_linearize path (the issue appears to occur only when the netback
device has NETIF_F_GSO but not NETIF_F_TSO). AFAICT dev_gso_segment goes
to tcp_tso_segment and then to skb_segment which does appear to create
skbs with a frag_list (although it's not outside the realms of
possibility that I'm reading it wrong).
I did wonder if perhaps we should just ignore the field, since something
higher up the stack walked the list and called start_xmit repeatedly,
but if that's the case I can't see where...
Ian.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 11:53 UTC (permalink / raw)
To: Eric Dumazet
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298979909.3284.28.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
>
> CPU 11 handles all TX completions : Its a potential bottleneck.
>
> I might ressurect XPS patch ;)
Actually this has been my gripe all along with our TX multiqueue
support. We should not decide the queue based on the socket, but
on the current CPU.
We already do the right thing for forwarded packets because there
is no socket to latch onto, we just need to fix it for locally
generated traffic.
The odd packet reordering each time your scheduler decides to
migrate the process isn't a big deal IMHO. If your scheduler
is constantly moving things you've got bigger problems to worry
about.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 12:01 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298979909.3284.28.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
This is how perf top looks like with SO_REUSEPORT
----------------------------------------------------------------------------------------------------------------------------------
PerfTop: 27498 irqs/sec kernel:50.5% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 1)
----------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ __________________
16464.00 6.0% isc_rwlock_lock libisc.so.62.0.1
15462.00 5.7% intel_idle [kernel.kallsyms]
13140.00 4.8% _spin_unlock_irqrestore [kernel.kallsyms]
9283.00 3.4% __do_softirq [kernel.kallsyms]
8469.00 3.1% finish_task_switch [kernel.kallsyms]
8189.00 3.0% __udp4_lib_lookup [kernel.kallsyms]
8096.00 3.0% dns_rbt_findnode libdns.so.69.0.1
7619.00 2.8% isc_rwlock_unlock libisc.so.62.0.1
5090.00 1.9% isc_stats_increment libisc.so.62.0.1
4325.00 1.6% tick_nohz_stop_sched_tick [kernel.kallsyms]
3656.00 1.3% _spin_lock [kernel.kallsyms]
3540.00 1.3% __pthread_mutex_lock_internal libpthread-2.12.so
3168.00 1.2% _spin_lock_bh [kernel.kallsyms]
2576.00 0.9% dns_name_fullcompare libdns.so.69.0.1
2492.00 0.9% __pthread_mutex_unlock libpthread-2.12.so
2486.00 0.9% isc___mempool_get libisc.so.62.0.1
2475.00 0.9% dns_rbtnodechain_init libdns.so.69.0.1
2454.00 0.9% be_poll_rx [be2net]
2417.00 0.9% sk_run_filter [kernel.kallsyms]
2411.00 0.9% tick_nohz_restart_sched_tick [kernel.kallsyms]
2331.00 0.9% dns_name_equal libdns.so.69.0.1
2198.00 0.8% net_rx_action [kernel.kallsyms]
2135.00 0.8% fget_light [kernel.kallsyms]
2130.00 0.8% dns_zone_attach libdns.so.69.0.1
2073.00 0.8% dns_name_getlabelsequence libdns.so.69.0.1
2024.00 0.7% copy_user_generic_string [kernel.kallsyms]
2003.00 0.7% dns_acl_match libdns.so.69.0.1
1868.00 0.7% be_xmit [be2net]
----------------------------------------------------------------------------------------------------------------------------------
PerfTop: 16206 irqs/sec kernel:88.6% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 3)
----------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ __________________
15662.00 11.3% __udp4_lib_lookup [kernel.kallsyms]
10404.00 7.5% intel_idle [kernel.kallsyms]
10248.00 7.4% _spin_unlock_irqrestore [kernel.kallsyms]
4386.00 3.2% __do_softirq [kernel.kallsyms]
4324.00 3.1% be_poll_rx [be2net]
4165.00 3.0% get_rx_page_info [be2net]
4050.00 2.9% get_page_from_freelist [kernel.kallsyms]
4045.00 2.9% finish_task_switch [kernel.kallsyms]
3861.00 2.8% sk_run_filter [kernel.kallsyms]
3544.00 2.5% ip_route_input [kernel.kallsyms]
3385.00 2.4% _spin_lock [kernel.kallsyms]
2583.00 1.9% get_rps_cpu [kernel.kallsyms]
2042.00 1.5% tick_nohz_stop_sched_tick [kernel.kallsyms]
1971.00 1.4% kmem_cache_alloc_node_notrace [kernel.kallsyms]
1788.00 1.3% _read_lock [kernel.kallsyms]
1777.00 1.3% __netif_receive_skb [kernel.kallsyms]
1777.00 1.3% isc_rwlock_lock libisc.so.62.0.1
1769.00 1.3% memcpy_c [kernel.kallsyms]
1618.00 1.2% __alloc_skb [kernel.kallsyms]
1591.00 1.1% __pthread_mutex_lock_internal libpthread-2.12.so
1576.00 1.1% kmem_cache_alloc_node [kernel.kallsyms]
1450.00 1.0% sock_queue_rcv_skb [kernel.kallsyms]
1427.00 1.0% tick_nohz_restart_sched_tick [kernel.kallsyms]
1214.00 0.9% __udp4_lib_rcv [kernel.kallsyms]
1124.00 0.8% net_rx_action [kernel.kallsyms]
1113.00 0.8% getnstimeofday [kernel.kallsyms]
1072.00 0.8% selinux_socket_sock_rcv_skb [kernel.kallsyms]
1016.00 0.7% ip_rcv [kernel.kallsyms]
992.00 0.7% sock_def_readable [kernel.kallsyms]
961.00 0.7% dns_rbt_findnode libdns.so.69.0.1
899.00 0.6% fget [kernel.kallsyms]
898.00 0.6% datagram_poll [kernel.kallsyms]
809.00 0.6% isc_rwlock_unlock libisc.so.62.0.1
803.00 0.6% __alloc_pages_nodemask [kernel.kallsyms]
799.00 0.6% udp_queue_rcv_skb [kernel.kallsyms]
694.00 0.5% packet_rcv [kernel.kallsyms]
662.00 0.5% mutex_lock [kernel.kallsyms]
------------------------------------------------------------------------------------------------------------------------------------------
PerfTop: 31619 irqs/sec kernel:37.7% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 10)
-------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ __________________
6726.00 7.7% isc_rwlock_lock libisc.so.62.0.1
4597.00 5.3% _spin_unlock_irqrestore [kernel.kallsyms]
4230.00 4.9% intel_idle [kernel.kallsyms]
3319.00 3.8% dns_rbt_findnode libdns.so.69.0.1
3178.00 3.7% isc_rwlock_unlock libisc.so.62.0.1
2682.00 3.1% finish_task_switch [kernel.kallsyms]
2164.00 2.5% isc_stats_increment libisc.so.62.0.1
1435.00 1.7% tick_nohz_stop_sched_tick [kernel.kallsyms]
1407.00 1.6% _spin_lock_bh [kernel.kallsyms]
1288.00 1.5% __pthread_mutex_lock_internal libpthread-2.12.so
1264.00 1.5% copy_user_generic_string [kernel.kallsyms]
1082.00 1.2% _spin_lock [kernel.kallsyms]
1061.00 1.2% be_xmit [be2net]
1024.00 1.2% __pthread_mutex_unlock libpthread-2.12.so
1014.00 1.2% dns_rbtnodechain_init libdns.so.69.0.1
989.00 1.1% isc___mempool_get libisc.so.62.0.1
964.00 1.1% dns_name_equal libdns.so.69.0.1
957.00 1.1% dns_name_getlabelsequence libdns.so.69.0.1
944.00 1.1% dns_name_fullcompare libdns.so.69.0.1
858.00 1.0% dns_zone_attach libdns.so.69.0.1
793.00 0.9% udp_sendmsg [kernel.kallsyms]
785.00 0.9% tick_nohz_restart_sched_tick [kernel.kallsyms]
784.00 0.9% dns_acl_match libdns.so.69.0.1
776.00 0.9% fget_light [kernel.kallsyms]
723.00 0.8% dns_name_hash libdns.so.69.0.1
691.00 0.8% dns_message_rendersection libdns.so.69.0.1
675.00 0.8% dns_name_fromwire libdns.so.69.0.1
658.00 0.8% udp_recvmsg [kernel.kallsyms]
646.00 0.7% kmem_cache_free [kernel.kallsyms]
641.00 0.7% kfree [kernel.kallsyms]
535.00 0.6% isc_radix_search libisc.so.62.0.1
531.00 0.6% dev_queue_xmit [kernel.kallsyms]
-------------------------------------------------------------------------------------------------------------------------------------------
PerfTop: 31136 irqs/sec kernel:48.3% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 11)
-------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ __________________
13043.00 6.0% isc_rwlock_lock libisc.so.62.0.1
10852.00 5.0% _spin_unlock_irqrestore [kernel.kallsyms]
10538.00 4.9% be_tx_compl_process [be2net]
8275.00 3.8% kfree [kernel.kallsyms]
6467.00 3.0% kmem_cache_free [kernel.kallsyms]
6453.00 3.0% dns_rbt_findnode libdns.so.69.0.1
6423.00 3.0% intel_idle [kernel.kallsyms]
6199.00 2.9% isc_rwlock_unlock libisc.so.62.0.1
5492.00 2.5% sock_wfree [kernel.kallsyms]
5372.00 2.5% finish_task_switch [kernel.kallsyms]
5321.00 2.4% kfree_skb [kernel.kallsyms]
4030.00 1.9% isc_stats_increment libisc.so.62.0.1
3820.00 1.8% skb_release_data [kernel.kallsyms]
3518.00 1.6% be_poll_tx_mcc [be2net]
3034.00 1.4% sock_def_write_space [kernel.kallsyms]
2599.00 1.2% __do_softirq [kernel.kallsyms]
2572.00 1.2% tick_nohz_stop_sched_tick [kernel.kallsyms]
2519.00 1.2% __pthread_mutex_lock_internal libpthread-2.12.so
2497.00 1.1% _spin_lock_bh [kernel.kallsyms]
2045.00 0.9% dns_name_fullcompare libdns.so.69.0.1
1960.00 0.9% isc___mempool_get libisc.so.62.0.1
1873.00 0.9% dns_rbtnodechain_init libdns.so.69.0.1
1861.00 0.9% _spin_lock [kernel.kallsyms]
1806.00 0.8% __pthread_mutex_unlock libpthread-2.12.so
1791.00 0.8% dns_name_equal libdns.so.69.0.1
1757.00 0.8% dns_zone_attach libdns.so.69.0.1
1621.00 0.7% dns_name_getlabelsequence libdns.so.69.0.1
1576.00 0.7% fget_light [kernel.kallsyms]
1532.00 0.7% dns_acl_match libdns.so.69.0.1
1515.00 0.7% tick_nohz_restart_sched_tick [kernel.kallsyms]
1510.00 0.7% be_xmit [be2net]
^ permalink raw reply
* Re: dccp: null-pointer dereference on close
From: Johan Hovold @ 2011-03-01 12:03 UTC (permalink / raw)
To: Gerrit Renker; +Cc: dccp, netdev
In-Reply-To: <20110301055945.GA4761@gerrit.erg.abdn.ac.uk>
On Tue, Mar 01, 2011 at 06:59:45AM +0100, Gerrit Renker wrote:
> Johan,
>
> thanks a lot for the detailed description.
>
> I think I have found the cause of the dccp timewait problem: in the
> mainline tree there is a path
>
> dccp_v4_do_rcv()
> |
> | state other than OPEN
> v
> dccp_rcv_state_process()
> |
> | DCCP_PKT_RESET
> v
> dccp_rcv_reset()
> |
> v
> dccp_time_wait()
>
> In the backtrace dccp_close() had been called, hence dccp_set_state() has
> destroyed inet_csk(sk)->icsk_bind_hash, which then subsequently in the
> misplaced dccp_time_wait() caused the NULL pointer exception.
>
> I have just checked, this problem seems to not be possible in the test
> tree, since it checks first in dccp_rcv_state_process() if DCCP_CLOSED
> has been entered (if it receives a packet in this state, it sends a
> Reset with code 3, "No Connection").
>
> I am attaching the relevant patch from the test tree - would it be possible
> for you to test it with the same setup? (The relevant passage is right in
> the first hunk, where it tests for state == DCCP_CLOSED).
As expected I do not seem able to trigger the null-pointer exception
with the patch applied to 2.6.38-rc6. The patch does not apply to the
2.6.37 kernel I was using, but only moving to closed-state check does the
trick.
> Will submit this patch subsequently also.
May I suggest separating the closed-state-check fix into a patch of
its own which could be marked for stable and more easily backported as
it fixes a pretty severe bug?
Below are the bits that fixed the issue on 2.6.37. Perhaps this along
with a more detailed description of the error, including the panic
message, could serve as the basis for such a patch?
Feel free to add
Reported-and-tested-by: Johan Hovold <jhovold@gmail.com>
Thanks,
Johan
>From eda93f93102ad13bc470db63cfd7b0dc27d1e4fa Mon Sep 17 00:00:00 2001
From: Johan Hovold <jhovold@gmail.com>
Date: Tue, 1 Mar 2011 12:13:39 +0100
Subject: [PATCH] net: dccp: fix null-pointer dereference on close
---
net/dccp/input.c | 7 +++----
1 files changed, 3 insertions(+), 4 deletions(-)
diff --git a/net/dccp/input.c b/net/dccp/input.c
index e424a09..421f42c 100644
--- a/net/dccp/input.c
+++ b/net/dccp/input.c
@@ -621,6 +621,9 @@ int dccp_rcv_state_process(struct sock *sk, struct sk_buff *skb,
/* Caller (dccp_v4_do_rcv) will send Reset */
dcb->dccpd_reset_code = DCCP_RESET_CODE_NO_CONNECTION;
return 1;
+ } else if (sk->sk_state == DCCP_CLOSED) {
+ dcb->dccpd_reset_code = DCCP_RESET_CODE_NO_CONNECTION;
+ return 1;
}
if (sk->sk_state != DCCP_REQUESTING && sk->sk_state != DCCP_RESPOND) {
@@ -683,10 +686,6 @@ int dccp_rcv_state_process(struct sock *sk, struct sk_buff *skb,
}
switch (sk->sk_state) {
- case DCCP_CLOSED:
- dcb->dccpd_reset_code = DCCP_RESET_CODE_NO_CONNECTION;
- return 1;
-
case DCCP_REQUESTING:
queued = dccp_rcv_request_sent_state_process(sk, skb, dh, len);
if (queued >= 0)
--
1.7.4
^ permalink raw reply related
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 12:15 UTC (permalink / raw)
To: Eric Dumazet, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301120112.GL9763@canuck.infradead.org>
On Tue, Mar 01, 2011 at 07:01:12AM -0500, Thomas Graf wrote:
> On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
>
> This is how perf top looks like with SO_REUSEPORT
Yeah I think Eric is spot on. The remaining bottleneck is because
we hash all outbound packets from a single socket to a single TX
queue, despite the fact that they were produced on difference CPUs.
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: dccp: null-pointer dereference on close
From: Gerrit Renker @ 2011-03-01 12:16 UTC (permalink / raw)
To: Johan Hovold; +Cc: dccp, netdev
In-Reply-To: <20110301120327.GA13273@localhost>
| Below are the bits that fixed the issue on 2.6.37. Perhaps this along
| with a more detailed description of the error, including the panic
| message, could serve as the basis for such a patch?
|
Yes, thank you for the suggestion. I will incorporate your suggestions and submit the
reworked patch first thing tomorrow morning, need to go through the original also.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Thomas Graf @ 2011-03-01 12:18 UTC (permalink / raw)
To: Eric Dumazet
Cc: Herbert Xu, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <1298979909.3284.28.camel@edumazet-laptop>
On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
> ethtool -S eth0 | grep rx_pk
> rxq0: rx_pkts: ??
> rxq1: rx_pkts: ??
> rxq2: rx_pkts: ??
> rxq3: rx_pkts: ??
> rxq4: rx_pkts: ??
It could do multiqueue but it doesnt:
[root@hp-bl460cg7-01 ~]# ethtool -S eth0 | grep rx_pk
rxq0: rx_pkts: 1512
rxq1: rx_pkts: 462
rxq2: rx_pkts: 122
rxq3: rx_pkts: 24751393
rxq4: rx_pkts: 35
So, adding a third client making sure it would hit another queue:
rxq0: rx_pkts: 3041
rxq1: rx_pkts: 867
rxq2: rx_pkts: 4610476
rxq3: rx_pkts: 57418776
rxq4: rx_pkts: 40
... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
PerfTop: 18417 irqs/sec kernel:50.2% exact: 0.0% [1000Hz cpu-clock-msecs], (all, CPU: 5)
-----------------------------------------------------------------------------------------------------------------------------------------------------------------
samples pcnt function DSO
_______ _____ _____________________________ ___________________________________________________________
12712.00 18.5% _raw_spin_unlock_irqrestore [kernel.kallsyms]
3697.00 5.4% isc_rwlock_lock /usr/lib64/libisc.so.62.0.1
1948.00 2.8% dns_rbt_findnode /usr/lib64/libdns.so.69.0.1
1809.00 2.6% isc_rwlock_unlock /usr/lib64/libisc.so.62.0.1
1631.00 2.4% __do_softirq [kernel.kallsyms]
1237.00 1.8% isc_stats_increment /usr/lib64/libisc.so.62.0.1
1106.00 1.6% clflush_cache_range [kernel.kallsyms]
964.00 1.4% _raw_spin_lock [kernel.kallsyms]
714.00 1.0% be_poll_rx /lib/modules/2.6.38-rc5+/kernel/drivers/net/benet/be2net.ko
630.00 0.9% __pthread_mutex_lock_internal /lib64/libpthread-2.12.so
627.00 0.9% dns_name_fullcompare /usr/lib64/libdns.so.69.0.1
582.00 0.8% dns_rbtnodechain_init /usr/lib64/libdns.so.69.0.1
552.00 0.8% sk_run_filter [kernel.kallsyms]
527.00 0.8% dns_name_getlabelsequence /usr/lib64/libdns.so.69.0.1
525.00 0.8% __pthread_mutex_unlock /lib64/libpthread-2.12.so
492.00 0.7% dns_name_equal /usr/lib64/libdns.so.69.0.1
468.00 0.7% isc___mempool_get /usr/lib64/libisc.so.62.0.1
462.00 0.7% __udp4_lib_lookup [kernel.kallsyms]
457.00 0.7% dns_acl_match /usr/lib64/libdns.so.69.0.1
453.00 0.7% dns_zone_attach /usr/lib64/libdns.so.69.0.1
451.00 0.7% fget_light [kernel.kallsyms]
443.00 0.6% dns_message_rendersection /usr/lib64/libdns.so.69.0.1
431.00 0.6% ipt_do_table [kernel.kallsyms]
429.00 0.6% nf_iterate [kernel.kallsyms]
422.00 0.6% __kmalloc_node_track_caller [kernel.kallsyms]
408.00 0.6% __domain_mapping [kernel.kallsyms]
387.00 0.6% dns_name_hash /usr/lib64/libdns.so.69.0.1
353.00 0.5% copy_user_generic_string [kernel.kallsyms]
349.00 0.5% dns_name_fromwire /usr/lib64/libdns.so.69.0.1
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 12:19 UTC (permalink / raw)
To: Eric Dumazet, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301121829.GM9763@canuck.infradead.org>
On Tue, Mar 01, 2011 at 07:18:29AM -0500, Thomas Graf wrote:
>
>
> ... makes it use CPU 5 for rxq2 and the qps goes up from 250kqps to 270kqps
I think the increase here comes from the larger number of packets
in flight more than anything.
The bottleneck is still the TX queue (both software and hardware).
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* Re: [GIT/PATCH v3] xen network backend driver
From: Ben Hutchings @ 2011-03-01 12:28 UTC (permalink / raw)
To: Ian Campbell
Cc: netdev@vger.kernel.org, xen-devel, Jeremy Fitzhardinge,
Herbert Xu, Konrad Rzeszutek Wilk, Francois Romieu
In-Reply-To: <1298974009.5034.2029.camel@zakaz.uk.xensource.com>
On Tue, 2011-03-01 at 10:06 +0000, Ian Campbell wrote:
> On Mon, 2011-02-28 at 18:53 +0000, Ben Hutchings wrote:
> > On Mon, 2011-02-28 at 17:27 +0000, Ian Campbell wrote:
>
> > This should be defined as unsigned long (ideally it would be u64, but
> > that can't be updated atomically on 32-bit systems).
> [...]
> > Don't update last_rx; it's only needed on slave devices of a bond, and
> > the bonding driver takes care of it now.
>
> I made these two changes.
>
> > [...]
> > > +static int xenvif_change_mtu(struct net_device *dev, int mtu)
> > > +{
> > > + struct xenvif *vif = netdev_priv(dev);
> > > + int max = vif->can_sg ? 65535 - ETH_HLEN : ETH_DATA_LEN;
> > > + if (mtu > max)
> > > + return -EINVAL;
> > > + dev->mtu = mtu;
> > > + return 0;
> > > +}
> > [...]
> >
> > Since any VLAN tag must be inserted inline, shouldn't the MTU limit be
> > 65535 - VLAN_ETH_HLEN?
>
> In that case shouldn't the other case also be ETH_FRAME_LEN -
> VLAN_ETH_HLEN?
[...]
IEEE 802.3, in its infinite wisdom, says that the maximum frame length
is 1514, except that when an 802.1q tag is present it is 1518. So the
MTU for standard Ethernet remains 1500, regardless of the use of VLANs.
Ben.
--
Ben Hutchings, Senior Software Engineer, Solarflare Communications
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.
^ permalink raw reply
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 12:32 UTC (permalink / raw)
To: Eric Dumazet
Cc: Thomas Graf, David Miller, rick.jones2, therbert, wsommerfeld,
daniel.baluta, netdev
In-Reply-To: <20110301115305.GA6984@gondor.apana.org.au>
On Tue, Mar 01, 2011 at 07:53:05PM +0800, Herbert Xu wrote:
> On Tue, Mar 01, 2011 at 12:45:09PM +0100, Eric Dumazet wrote:
> >
> > CPU 11 handles all TX completions : Its a potential bottleneck.
> >
> > I might ressurect XPS patch ;)
>
> Actually this has been my gripe all along with our TX multiqueue
> support. We should not decide the queue based on the socket, but
> on the current CPU.
>
> We already do the right thing for forwarded packets because there
> is no socket to latch onto, we just need to fix it for locally
> generated traffic.
>
> The odd packet reordering each time your scheduler decides to
> migrate the process isn't a big deal IMHO. If your scheduler
> is constantly moving things you've got bigger problems to worry
> about.
If anybody wants to play here is a patch to do exactly that:
net: Determine TX queue purely by current CPU
Distributing packets generated on one CPU to multiple queues
makes no sense. Nor does putting packets from multiple CPUs
into a single queue.
While this may introduce packet reordering should the scheduler
decide to migrate a thread, it isn't a big deal because migration
is meant to be a rare event, and nothing will die as long as the
ordering doesn't occur all the time.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
diff --git a/net/core/dev.c b/net/core/dev.c
index 8ae6631..87bd20a 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -2164,22 +2164,12 @@ static u32 hashrnd __read_mostly;
u16 __skb_tx_hash(const struct net_device *dev, const struct sk_buff *skb,
unsigned int num_tx_queues)
{
- u32 hash;
+ u32 hash = raw_smp_processor_id();
- if (skb_rx_queue_recorded(skb)) {
- hash = skb_get_rx_queue(skb);
- while (unlikely(hash >= num_tx_queues))
- hash -= num_tx_queues;
- return hash;
- }
+ while (unlikely(hash >= num_tx_queues))
+ hash -= num_tx_queues;
- if (skb->sk && skb->sk->sk_hash)
- hash = skb->sk->sk_hash;
- else
- hash = (__force u16) skb->protocol ^ skb->rxhash;
- hash = jhash_1word(hash, hashrnd);
-
- return (u16) (((u64) hash * num_tx_queues) >> 32);
+ return hash;
}
EXPORT_SYMBOL(__skb_tx_hash);
Cheers,
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply related
* Re: SO_REUSEPORT - can it be done in kernel?
From: Herbert Xu @ 2011-03-01 12:35 UTC (permalink / raw)
To: David Miller, rick.jones2, therbert, wsommerfeld, daniel.baluta,
netdev
In-Reply-To: <20110228113659.GA20726@gondor.apana.org.au>
On Mon, Feb 28, 2011 at 07:36:59PM +0800, Herbert Xu wrote:
> Here are the patches I used. Please don't them yet as I intend
> to clean them up quite a bit.
OK here is the version ready for merging (please retest them
though as I have changed things substantially).
The main change is that the legacy UDP code path is now gone
so we use the same UDP header generation whether corking is on
or off.
I will add IPv6 support in a later patch set.
Thanks!
--
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
^ permalink raw reply
* [PATCH 5/5] udp: Add lockless transmit path
From: Herbert Xu @ 2011-03-01 12:36 UTC (permalink / raw)
To: David Miller, rick.jones2, therbert, wsommerfeld, daniel.baluta,
netdev, Thomas Graf
In-Reply-To: <20110301123519.GA7453@gondor.apana.org.au>
udp: Add lockless transmit path
The UDP transmit path has been running under the socket lock
for a long time because of the corking feature. This means that
transmitting to the same socket in multiple threads does not
scale at all.
However, as most users don't actually use corking, the locking
can be removed in the common case.
This patch creates a lockless fast path where corking is not used.
Please note that this does create a slight inaccuracy in the
enforcement of socket send buffer limits. In particular, we
may exceed the socket limit by up to (number of CPUs) * (packet
size) because of the way the limit is computed.
As the primary purpose of socket buffers is to indicate congestion,
this should not be a great problem for now.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
net/ipv4/udp.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 9a6d326..bb9f707 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -802,6 +802,7 @@ int udp_sendmsg(struct kiocb *iocb, struct sock *sk, struct msghdr *msg,
int err, is_udplite = IS_UDPLITE(sk);
int corkreq = up->corkflag || msg->msg_flags&MSG_MORE;
int (*getfrag)(void *, char *, int, int, int, struct sk_buff *);
+ struct sk_buff *skb;
if (len > 0xFFFF)
return -EMSGSIZE;
@@ -816,6 +817,8 @@ int udp_sendmsg(struct kiocb *iocb, struct sock *sk, struct msghdr *msg,
ipc.opt = NULL;
ipc.tx_flags = 0;
+ getfrag = is_udplite ? udplite_getfrag : ip_generic_getfrag;
+
if (up->pending) {
/*
* There are pending frames.
@@ -940,6 +943,17 @@ back_from_confirm:
if (!ipc.addr)
daddr = ipc.addr = rt->rt_dst;
+ /* Lockless fast path for the non-corking case. */
+ if (!corkreq) {
+ skb = ip_make_skb(sk, getfrag, msg->msg_iov, ulen,
+ sizeof(struct udphdr), &ipc, &rt,
+ msg->msg_flags);
+ err = PTR_ERR(skb);
+ if (skb && !IS_ERR(skb))
+ err = udp_send_skb(skb, daddr, dport);
+ goto out;
+ }
+
lock_sock(sk);
if (unlikely(up->pending)) {
/* The socket is already corked while preparing it. */
@@ -961,7 +975,6 @@ back_from_confirm:
do_append_data:
up->len += ulen;
- getfrag = is_udplite ? udplite_getfrag : ip_generic_getfrag;
err = ip_append_data(sk, getfrag, msg->msg_iov, ulen,
sizeof(struct udphdr), &ipc, &rt,
corkreq ? msg->msg_flags|MSG_MORE : msg->msg_flags);
^ permalink raw reply related
* [PATCH 4/5] udp: Switch to ip_finish_skb
From: Herbert Xu @ 2011-03-01 12:36 UTC (permalink / raw)
To: David Miller, rick.jones2, therbert, wsommerfeld, daniel.baluta,
netdev, Thomas Graf
In-Reply-To: <20110301123519.GA7453@gondor.apana.org.au>
udp: Switch to ip_finish_skb
This patch converts UDP to use the new ip_finish_skb API. This
would then allows us to more easily use ip_make_skb which allows
UDP to run without a socket lock.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
include/net/udp.h | 11 ++++++
include/net/udplite.h | 12 +++++++
net/ipv4/udp.c | 83 ++++++++++++++++++++++++++++++--------------------
3 files changed, 73 insertions(+), 33 deletions(-)
diff --git a/include/net/udp.h b/include/net/udp.h
index bb967dd..b8563ba 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -144,6 +144,17 @@ static inline __wsum udp_csum_outgoing(struct sock *sk, struct sk_buff *skb)
return csum;
}
+static inline __wsum udp_csum(struct sk_buff *skb)
+{
+ __wsum csum = csum_partial(skb_transport_header(skb),
+ sizeof(struct udphdr), skb->csum);
+
+ for (skb = skb_shinfo(skb)->frag_list; skb; skb = skb->next) {
+ csum = csum_add(csum, skb->csum);
+ }
+ return csum;
+}
+
/* hash routines shared between UDPv4/6 and UDP-Litev4/6 */
static inline void udp_lib_hash(struct sock *sk)
{
diff --git a/include/net/udplite.h b/include/net/udplite.h
index afdffe6..673a024 100644
--- a/include/net/udplite.h
+++ b/include/net/udplite.h
@@ -115,6 +115,18 @@ static inline __wsum udplite_csum_outgoing(struct sock *sk, struct sk_buff *skb)
return csum;
}
+static inline __wsum udplite_csum(struct sk_buff *skb)
+{
+ struct sock *sk = skb->sk;
+ int cscov = udplite_sender_cscov(udp_sk(sk), udp_hdr(skb));
+ const int off = skb_transport_offset(skb);
+ const int len = skb->len - off;
+
+ skb->ip_summed = CHECKSUM_NONE; /* no HW support for checksumming */
+
+ return skb_checksum(skb, off, min(cscov, len), 0);
+}
+
extern void udplite4_register(void);
extern int udplite_get_port(struct sock *sk, unsigned short snum,
int (*scmp)(const struct sock *, const struct sock *));
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index 8157b17..9a6d326 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -663,75 +663,72 @@ void udp_flush_pending_frames(struct sock *sk)
EXPORT_SYMBOL(udp_flush_pending_frames);
/**
- * udp4_hwcsum_outgoing - handle outgoing HW checksumming
- * @sk: socket we are sending on
+ * udp4_hwcsum - handle outgoing HW checksumming
* @skb: sk_buff containing the filled-in UDP header
* (checksum field must be zeroed out)
+ * @src: source IP address
+ * @dst: destination IP address
*/
-static void udp4_hwcsum_outgoing(struct sock *sk, struct sk_buff *skb,
- __be32 src, __be32 dst, int len)
+static void udp4_hwcsum(struct sk_buff *skb, __be32 src, __be32 dst)
{
- unsigned int offset;
struct udphdr *uh = udp_hdr(skb);
+ struct sk_buff *frags = skb_shinfo(skb)->frag_list;
+ int offset = skb_transport_offset(skb);
+ int len = skb->len - offset;
+ int hlen = len;
__wsum csum = 0;
- if (skb_queue_len(&sk->sk_write_queue) == 1) {
+ if (!frags) {
/*
* Only one fragment on the socket.
*/
skb->csum_start = skb_transport_header(skb) - skb->head;
skb->csum_offset = offsetof(struct udphdr, check);
- uh->check = ~csum_tcpudp_magic(src, dst, len, IPPROTO_UDP, 0);
+ uh->check = ~csum_tcpudp_magic(src, dst, len,
+ IPPROTO_UDP, 0);
} else {
/*
* HW-checksum won't work as there are two or more
* fragments on the socket so that all csums of sk_buffs
* should be together
*/
- offset = skb_transport_offset(skb);
- skb->csum = skb_checksum(skb, offset, skb->len - offset, 0);
+ do {
+ csum = csum_add(csum, frags->csum);
+ hlen -= frags->len;
+ } while ((frags = frags->next));
+ csum = skb_checksum(skb, offset, hlen, csum);
skb->ip_summed = CHECKSUM_NONE;
- skb_queue_walk(&sk->sk_write_queue, skb) {
- csum = csum_add(csum, skb->csum);
- }
-
uh->check = csum_tcpudp_magic(src, dst, len, IPPROTO_UDP, csum);
if (uh->check == 0)
uh->check = CSUM_MANGLED_0;
}
}
-/*
- * Push out all pending data as one UDP datagram. Socket is locked.
- */
-static int udp_push_pending_frames(struct sock *sk)
+static int udp_send_skb(struct sk_buff *skb, __be32 daddr, __be32 dport)
{
- struct udp_sock *up = udp_sk(sk);
+ struct sock *sk = skb->sk;
struct inet_sock *inet = inet_sk(sk);
- struct flowi *fl = &inet->cork.fl;
- struct sk_buff *skb;
struct udphdr *uh;
+ struct rtable *rt = (struct rtable *)skb_dst(skb);
int err = 0;
int is_udplite = IS_UDPLITE(sk);
+ int offset = skb_transport_offset(skb);
+ int len = skb->len - offset;
__wsum csum = 0;
- /* Grab the skbuff where UDP header space exists. */
- if ((skb = skb_peek(&sk->sk_write_queue)) == NULL)
- goto out;
-
/*
* Create a UDP header
*/
uh = udp_hdr(skb);
- uh->source = fl->fl_ip_sport;
- uh->dest = fl->fl_ip_dport;
- uh->len = htons(up->len);
+ uh->source = inet->inet_sport;
+ uh->dest = dport;
+ uh->len = htons(len);
uh->check = 0;
if (is_udplite) /* UDP-Lite */
- csum = udplite_csum_outgoing(sk, skb);
+ csum = udplite_csum(skb);
else if (sk->sk_no_check == UDP_CSUM_NOXMIT) { /* UDP csum disabled */
@@ -740,20 +737,20 @@ static int udp_push_pending_frames(struct sock *sk)
} else if (skb->ip_summed == CHECKSUM_PARTIAL) { /* UDP hardware csum */
- udp4_hwcsum_outgoing(sk, skb, fl->fl4_src, fl->fl4_dst, up->len);
+ udp4_hwcsum(skb, rt->rt_src, daddr);
goto send;
- } else /* `normal' UDP */
- csum = udp_csum_outgoing(sk, skb);
+ } else
+ csum = udp_csum(skb);
/* add protocol-dependent pseudo-header */
- uh->check = csum_tcpudp_magic(fl->fl4_src, fl->fl4_dst, up->len,
+ uh->check = csum_tcpudp_magic(rt->rt_src, daddr, len,
sk->sk_protocol, csum);
if (uh->check == 0)
uh->check = CSUM_MANGLED_0;
send:
- err = ip_push_pending_frames(sk);
+ err = ip_send_skb(skb);
if (err) {
if (err == -ENOBUFS && !inet->recverr) {
UDP_INC_STATS_USER(sock_net(sk),
@@ -763,6 +760,26 @@ send:
} else
UDP_INC_STATS_USER(sock_net(sk),
UDP_MIB_OUTDATAGRAMS, is_udplite);
+ return err;
+}
+
+/*
+ * Push out all pending data as one UDP datagram. Socket is locked.
+ */
+static int udp_push_pending_frames(struct sock *sk)
+{
+ struct udp_sock *up = udp_sk(sk);
+ struct inet_sock *inet = inet_sk(sk);
+ struct flowi *fl = &inet->cork.fl;
+ struct sk_buff *skb;
+ int err = 0;
+
+ skb = ip_finish_skb(sk);
+ if (!skb)
+ goto out;
+
+ err = udp_send_skb(skb, fl->fl4_dst, fl->fl_ip_dport);
+
out:
up->len = 0;
up->pending = 0;
^ permalink raw reply related
* [PATCH 1/5] inet: Remove unused sk_sndmsg_* from UFO
From: Herbert Xu @ 2011-03-01 12:36 UTC (permalink / raw)
To: David Miller, rick.jones2, therbert, wsommerfeld, daniel.baluta,
netdev, Thomas Graf
In-Reply-To: <20110301123519.GA7453@gondor.apana.org.au>
inet: Remove unused sk_sndmsg_* from UFO
UFO doesn't really use the sk_sndmsg_* parameters so touching
them is pointless. It can't use them anyway since the whole
point of UFO is to use the original pages without copying.
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
---
net/core/skbuff.c | 3 ---
net/ipv4/ip_output.c | 1 -
net/ipv6/ip6_output.c | 1 -
3 files changed, 5 deletions(-)
diff --git a/net/core/skbuff.c b/net/core/skbuff.c
index d883dcc..97011a7 100644
--- a/net/core/skbuff.c
+++ b/net/core/skbuff.c
@@ -2434,8 +2434,6 @@ int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
return -ENOMEM;
/* initialize the next frag */
- sk->sk_sndmsg_page = page;
- sk->sk_sndmsg_off = 0;
skb_fill_page_desc(skb, frg_cnt, page, 0, 0);
skb->truesize += PAGE_SIZE;
atomic_add(PAGE_SIZE, &sk->sk_wmem_alloc);
@@ -2455,7 +2453,6 @@ int skb_append_datato_frags(struct sock *sk, struct sk_buff *skb,
return -EFAULT;
/* copy was successful so update the size parameters */
- sk->sk_sndmsg_off += copy;
frag->size += copy;
skb->len += copy;
skb->data_len += copy;
diff --git a/net/ipv4/ip_output.c b/net/ipv4/ip_output.c
index 04c7b3b..d3a4540 100644
--- a/net/ipv4/ip_output.c
+++ b/net/ipv4/ip_output.c
@@ -767,7 +767,6 @@ static inline int ip_ufo_append_data(struct sock *sk,
skb->ip_summed = CHECKSUM_PARTIAL;
skb->csum = 0;
- sk->sk_sndmsg_off = 0;
/* specify the length of each IP datagram fragment */
skb_shinfo(skb)->gso_size = mtu - fragheaderlen;
diff --git a/net/ipv6/ip6_output.c b/net/ipv6/ip6_output.c
index 5f8d242..9965182 100644
--- a/net/ipv6/ip6_output.c
+++ b/net/ipv6/ip6_output.c
@@ -1061,7 +1061,6 @@ static inline int ip6_ufo_append_data(struct sock *sk,
skb->ip_summed = CHECKSUM_PARTIAL;
skb->csum = 0;
- sk->sk_sndmsg_off = 0;
}
err = skb_append_datato_frags(sk,skb, getfrag, from,
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox