* Re: ipw2100: firmware problem
From: Denis Vlasenko @ 2005-06-10 6:56 UTC (permalink / raw)
To: abonilla, 'Pavel Machek', 'Jeff Garzik',
'Netdev list', 'kernel list',
'James P. Ketrenos'
In-Reply-To: <002a01c56cff$fb64ba70$600cc60a@amer.sykes.com>
On Thursday 09 June 2005 17:31, Alejandro Bonilla wrote:
>
> > What is so nice about this? That Linux novice user with his new lappie
> > will join a neighbor's network every time he powers up the lappie,
> > even without knowing that?
> >
> > That will be analogous to me plugging ethernet cable into the
> > switch and
> > wanting it to work, without any IP addr config, even without
> > DHCP client.
> > Just power up the box (or modprobe an eth module) and it
> > works! Cool, eh?
> >
>
> You want things one way, I like them in another way. Whoever makes this
> decision should just know that we would like to have an option to make it
> load with or without the ASSOC on.
But you already _have_ the option to associate. Just issue
appropriate iwconfig command (or embed one in startup script).
> James already said to use the options ipw2100 disable=1 if you don't want it
> to associate everytime on boot.
Do we have to add such option to each and every wireless driver now?
That would be wrong since iwconfig already exists.
> At the end, who decides this?
User. As I said, with no automatic assoc at module load user still
may easily attain that with iwconfig.
Adding kernel level wireless autoconfiguration duplicates the effort.
Since I am not going to give up a requirement to be able to stay radio
silent at boot (me too wants freedom, not only you), you need to add
disable=1 module parameter to each driver, which adds to the mess.
ALSA does the Right Thing. Sound is completely muted out at module load.
It's a user freedom to set desired volume level after that.
--
vda
^ permalink raw reply
* Re: ipw2100: firmware problem
From: Jim Crilly @ 2005-06-10 3:51 UTC (permalink / raw)
To: Denis Vlasenko
Cc: David S. Miller, jketreno, pavel, jgarzik, netdev, linux-kernel,
ipw2100-admin
In-Reply-To: <200506090930.22274.vda@ilport.com.ua>
On 06/09/05 09:30:22AM +0300, Denis Vlasenko wrote:
> On Thursday 09 June 2005 09:20, David S. Miller wrote:
> > From: Denis Vlasenko <vda@ilport.com.ua>
> > Date: Thu, 9 Jun 2005 09:17:23 +0300
> >
> > > Sadly, realities are such that we have to live somehow
> > > with closed-source firmware.
> >
> > You have a choice, buy products from friendly vendors.
>
> I am trying! So far, I have Prism2.5, Prism54
> and acx111 cards, and all of them require closed binary fw.
Ralink cards don't require any binary firmware. I got one of the Hawking
Tech pcmcia cards off the shelf at a local CompUSA store and the card
was a good $20 cheaper than a Linksys card. So far I haven't had any
problems with the card or the driver.
A list of Ralink hardware and drivers can be found at
http://ralink.rapla.net/
> vda
>
Jim.
^ permalink raw reply
* Re: ipw2100: firmware problem
From: Alejandro Bonilla @ 2005-06-10 2:46 UTC (permalink / raw)
To: Jeff Garzik
Cc: James Ketrenos, David S. Miller, pavel, vda, netdev, linux-kernel,
ipw2100-admin
In-Reply-To: <42A8F758.2060008@pobox.com>
Jeff Garzik wrote:
> James Ketrenos wrote:
>
>> I don't know if all the distributions have moved away from this
>> model. If they have and the devices are brought up regardless of
>> link, then
>> going back to delaying radio initialization until the open() is called
>> is workable.
>
>
>
> When the interface is not up, we ideally want the device to be as
> passive as possible.
>
> Most net drivers shut down as much as possible at dev->close() time,
> and it would really be good if wireless drivers followed suit.
>
> Jeff
>
>
>
OK. I understand the point and I totally agree with this. We really want
the adapter to just do what the user or profiles ask the adapter to do.
Yes, in an ideal world.
Let's talk about easyness. These adapters are in laptops. You don't want
to type a lot of stop everytime you move from access points, reboots and
so on. In a server enviroment with the ethernet adapters, we really just
want them to do what they do and we have scripts for it. So, again, with
mobile is different. An association on boot is fair and really OK. You
are not really doing dhcp requests on boot and trying to get the
internet from people for free. You just want you adapter running faster,
get connected and get over whatever you have to do to start working or
whatsoever.
Let's really think what would be the nicest way that the card should
behave, after all if the adapter just associates, you are not really
stealing any Internet. :)
Association on boot is how it has worked all the time, and in the 18
months of the project, nobody has complained about it... So... I wonder,
users are happy with it? (I know it might not be the correct way)
Just a thought.
.Alejandro
^ permalink raw reply
* Re: BCM5704 performance questions.
From: Ben Greear @ 2005-06-10 2:28 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, mchan
In-Reply-To: <20050609.182943.29573731.davem@davemloft.net>
David S. Miller wrote:
> From: Ben Greear <greearb@candelatech.com>
> Date: Thu, 09 Jun 2005 18:20:33 -0700
>
>
>>I ran a test using pktgen to (try to) send 82kpps, 1514 byte packets between
>>two ports on the tg3 NIC. It can do about 780Mbps in one direction,
>>and 880Mbps in the other direction. Lots of harmless hard-start xmit errors reported
>>(tg3 may not stop it's tx queue correctly, or maybe pktgen is screwed up since
>>e1000 reports similar errors).
>
>
> There is a known race on SMP with drivers using NETIF_F_LLTX that is
> still not fixed. It will cause the error message to be reported from
> the driver's ->hard_start_xmit() routine when you hit this race.
Ok, it's not a big deal, since I can just retry the packet when
the tx queue wakes up again. The message was coming from pktgen, btw.
> You're definitely on an SMP system if you are triggering that message.
Yes, dual xeon with HT as well.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: ipw2100: firmware problem
From: Jeff Garzik @ 2005-06-10 2:13 UTC (permalink / raw)
To: James Ketrenos
Cc: David S. Miller, pavel, vda, abonilla, netdev, linux-kernel,
ipw2100-admin
In-Reply-To: <42A8AE2A.4080104@linux.intel.com>
James Ketrenos wrote:
> I don't know if all the distributions have moved away from this model.
> If they have and the devices are brought up regardless of link, then
> going back to delaying radio initialization until the open() is called
> is workable.
When the interface is not up, we ideally want the device to be as
passive as possible.
Most net drivers shut down as much as possible at dev->close() time, and
it would really be good if wireless drivers followed suit.
Jeff
^ permalink raw reply
* Re: BCM5704 performance questions.
From: David S. Miller @ 2005-06-10 1:29 UTC (permalink / raw)
To: greearb; +Cc: netdev, mchan
In-Reply-To: <42A8EAE1.5030201@candelatech.com>
From: Ben Greear <greearb@candelatech.com>
Date: Thu, 09 Jun 2005 18:20:33 -0700
> I ran a test using pktgen to (try to) send 82kpps, 1514 byte packets between
> two ports on the tg3 NIC. It can do about 780Mbps in one direction,
> and 880Mbps in the other direction. Lots of harmless hard-start xmit errors reported
> (tg3 may not stop it's tx queue correctly, or maybe pktgen is screwed up since
> e1000 reports similar errors).
There is a known race on SMP with drivers using NETIF_F_LLTX that is
still not fixed. It will cause the error message to be reported from
the driver's ->hard_start_xmit() routine when you hit this race.
The e1000 driver hits the same race, it just doesn't print any
message.
You're definitely on an SMP system if you are triggering that message.
^ permalink raw reply
* Re: BCM5704 performance questions.
From: Ben Greear @ 2005-06-10 1:24 UTC (permalink / raw)
To: Michael Chan; +Cc: 'netdev@oss.sgi.com'
In-Reply-To: <1118361376.5838.20.camel@rh4>
Michael Chan wrote:
> On Thu, 2005-06-09 at 17:38 -0700, Ben Greear wrote:
>
>>* Is the BCM5704 chipset/driver really that much slower?
>>
> Unfortunately, the 5704 requires the "ONE_DMA" workaround which will
> limit throughput in a PCIX 100/133 bus. If you comment out the line that
> sets the DMA_RWCTRL_ONE_DMA flag in tg3.c, you should see improved
> performance. However, you may run into some DMA issues on certain
> systems.
Is there any way I can tell which systems are affected? It won't be
an option for me to purposefully ship possibly busted drivers/hardware,
but if I can be certain that my systems are immune, I will try this
modification.
>>* Is there some information on tuning the tg3 somewhere?
>> (I didn't see a Documentation/networking/tg3.txt file, for instance)
>>
>>* Is there a way to verify the bus speed that the NIC is running at?
>> (ethtool -d ethX gives lots of meaningless (to me) hex)
>>
>
>
> tg3 probing string for each device will tell you the bus type, width,
> and speed.
Thanks, I found it.
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: BCM5704 performance questions.
From: Ben Greear @ 2005-06-10 1:20 UTC (permalink / raw)
To: David S. Miller; +Cc: netdev, mchan
In-Reply-To: <20050609.175417.108740435.davem@davemloft.net>
David S. Miller wrote:
> From: Ben Greear <greearb@candelatech.com>
> Date: Thu, 09 Jun 2005 17:38:22 -0700
>
>
>>I am trying to bridge as much traffic as possible across two interfaces,
>>using a proprietary kernel module.
>
>
> Ben, I'm going to just mention that I'm not going to
> look into your bug report. You consistently come here
> without a test case or setup that other developers can
> use to reproduce or investigate your problem. You always
> report things with your proprietary setup of this or that.
>
> You have our code, we don't have your's.
Fair enough.
I ran a test using pktgen to (try to) send 82kpps, 1514 byte packets between
two ports on the tg3 NIC. It can do about 780Mbps in one direction,
and 880Mbps in the other direction. Lots of harmless hard-start xmit errors reported
(tg3 may not stop it's tx queue correctly, or maybe pktgen is screwed up since
e1000 reports similar errors).
Intel e1000 can do about 960Mbps in both directions.
My pktgen is modified. You can find my full patch against 2.6.11 here if you so wish:
http://www.candelatech.com/oss/candela_2.6.11.patch
If you need the exact arguments I used to configure pktgen, I can get those
for you as well.
I found the Mhz printout (thanks Michael!)
The tg3 NIC is in the 133Mhz slot. That probably means the intel NIC is only
running at 100Mhz.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: BCM5704 performance questions.
From: David S. Miller @ 2005-06-10 0:54 UTC (permalink / raw)
To: greearb; +Cc: netdev, mchan
In-Reply-To: <42A8E0FE.3020708@candelatech.com>
From: Ben Greear <greearb@candelatech.com>
Date: Thu, 09 Jun 2005 17:38:22 -0700
> I am trying to bridge as much traffic as possible across two interfaces,
> using a proprietary kernel module.
Ben, I'm going to just mention that I'm not going to
look into your bug report. You consistently come here
without a test case or setup that other developers can
use to reproduce or investigate your problem. You always
report things with your proprietary setup of this or that.
You have our code, we don't have your's.
^ permalink raw reply
* BCM5704 performance questions.
From: Ben Greear @ 2005-06-10 0:38 UTC (permalink / raw)
To: 'netdev@oss.sgi.com'; +Cc: mchan
Hello!
I have a 4-port NIC by silicom-usa.com that uses the BCM5704 (rev 10) chipset.
It's running in a PCI-X bus (100 or 133Mhz). CPUs are dual xeon 2.8Ghz,
1MB cache, 1GB RAM, etc). Kernel is 2.6.11 + my hacks (no hacks to tg3, minor
hacks to e1000 and other parts of the networking stacks).
I am trying to bridge as much traffic as possible across two interfaces,
using a proprietary kernel module.
The network traffic is 1514 byte packets, generated by a modified version of
pktgen running on another machine with similar hardware (Intel NICs).
With the BCM NIC I can get about 600Mbps in one direction and about 800Mbps
in the other..with a great deal of dropped packets. With the Intel 4-port
NIC (same machine, different PCI slot, and also from Silicom-usa.com)
I can get 900+Mbps in both directions with virtually no drops.
So:
* Is the BCM5704 chipset/driver really that much slower?
* Is there some information on tuning the tg3 somewhere?
(I didn't see a Documentation/networking/tg3.txt file, for instance)
* Is there a way to verify the bus speed that the NIC is running at?
(ethtool -d ethX gives lots of meaningless (to me) hex)
Please let me know if more information would be useful.
Thanks,
Ben
--
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
^ permalink raw reply
* Re: BCM5704 performance questions.
From: Michael Chan @ 2005-06-10 0:37 UTC (permalink / raw)
To: Ben Greear; +Cc: 'netdev@oss.sgi.com'
In-Reply-To: <42A8EBDA.6010306@candelatech.com>
On Thu, 2005-06-09 at 18:24 -0700, Ben Greear wrote:
> Michael Chan wrote:
> >
> > Unfortunately, the 5704 requires the "ONE_DMA" workaround which will
> > limit throughput in a PCIX 100/133 bus. If you comment out the line that
> > sets the DMA_RWCTRL_ONE_DMA flag in tg3.c, you should see improved
> > performance. However, you may run into some DMA issues on certain
> > systems.
>
> Is there any way I can tell which systems are affected? It won't be
> an option for me to purposefully ship possibly busted drivers/hardware,
> but if I can be certain that my systems are immune, I will try this
> modification.
>
I mentioned this so that you could verify that the slow performance was
indeed caused by ONE_DMA. Even if your system is affected, it's a very
subtle problem that won't show up right away and should allow you to get
some performance numbers.
Unfortunately, if indeed it is ONE_DMA, there is no easy way for us to
tell which system is affected. And the recommendation is to turn it on
for all 5704 in PCIX 100/133.
^ permalink raw reply
* Re: BCM5704 performance questions.
From: Michael Chan @ 2005-06-09 23:56 UTC (permalink / raw)
To: Ben Greear; +Cc: 'netdev@oss.sgi.com'
In-Reply-To: <42A8E0FE.3020708@candelatech.com>
On Thu, 2005-06-09 at 17:38 -0700, Ben Greear wrote:
>
> * Is the BCM5704 chipset/driver really that much slower?
>
Unfortunately, the 5704 requires the "ONE_DMA" workaround which will
limit throughput in a PCIX 100/133 bus. If you comment out the line that
sets the DMA_RWCTRL_ONE_DMA flag in tg3.c, you should see improved
performance. However, you may run into some DMA issues on certain
systems.
> * Is there some information on tuning the tg3 somewhere?
> (I didn't see a Documentation/networking/tg3.txt file, for instance)
>
> * Is there a way to verify the bus speed that the NIC is running at?
> (ethtool -d ethX gives lots of meaningless (to me) hex)
>
tg3 probing string for each device will tell you the bus type, width,
and speed.
^ permalink raw reply
* Re: [PATCH] tg3: Fix link failure in 5701
From: Grant Grundler @ 2005-06-09 23:28 UTC (permalink / raw)
To: Michael Chan; +Cc: davem, iod00d, peterc, netdev
In-Reply-To: <1118086942.5008.14.camel@rh4>
On Mon, Jun 06, 2005 at 12:42:22PM -0700, Michael Chan wrote:
> On some 5701 devices with older bootcode, the LED configuration bits in
> SRAM may be invalid with value zero. The fix is to check for invalid
> bits (0) and default to PHY 1 mode. Incorrect LED mode will lead to
> error in programming the PHY.
Michael, David,
I confirmed this patch in fact fixes the problem on currently
shipping rx7620 and rx8620 "Core LAN" cards. I expected it would
but now have nice warm fuzzies that it really 100% does.
One minor issue: I unloaded the unpatched tg3 v3.29 driver and
then did not get a link when loaded the patched tg3 v3.30 driver.
Unplugging and replugging the cable made this work.
v3.30 continued to work fine after a reboot.
lspci for the offending rx8620 NIC is:
[root@n2 net]# lspci -vs 00:01.0
00:01.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5701 Gigabit Ethe
rnet (rev 15)
Subsystem: Hewlett-Packard Company HP IOX Core Lan 1000Base-T [A7109AX]
Flags: 66Mhz, medium devsel, IRQ 50
Memory at 00000f0100000000 (64-bit, non-prefetchable) [size=64K]
Capabilities: [40] PCI-X non-bridge device.
Capabilities: [48] Power Management version 2
Capabilities: [50] Vital Product Data
Capabilities: [58] Message Signalled Interrupts: 64bit+ Queue=0/3 Enable
(aka Subsystem 103c:12c1.)
rx7620 Core LAN has Subsystem Device ID 0x1300 and is also expected
to have this problem. (I've just now submitted an entry to pciids.sf.net)
I'm told another "Core LAN" NIC from an older platform (rx5670)
_may_ also have bad "boot code". But I don't have any to test with
and don't know what the SubSys DevID is.
BTW, I am still pushing for a recipe to update the bootcode.
This is just painfully slow. Not surprising given the number
of organizations involved.
hth,
grant
>
> Thanks to Grant Grundler for debugging the problem.
>
> >From Grant:
> | In May, 2004, tg3 v3.4 changed how MAC_LED_CTRL (0x40c) was getting
> | programmed and how to determine what to program into LED_CTRL. The new
> | code trusted NIC_SRAM_DATA_CFG (0x00000b58) to indicate what to write
> | to LED_CTRL and MII EXT_CTRL registers. On "IOX Core Lan", SRAM was
> | saying MODE_MAC (0x0) and that doesn't work.
>
> Signed-off-by: Michael Chan <mchan@broadcom.com>
>
> diff -Nru led1/drivers/net/tg3.c led2/drivers/net/tg3.c
> --- led1/drivers/net/tg3.c 2005-06-06 10:19:56.692541944 -0700
> +++ led2/drivers/net/tg3.c 2005-06-06 10:34:49.251852304 -0700
> @@ -8555,6 +8555,16 @@
>
> case NIC_SRAM_DATA_CFG_LED_MODE_MAC:
> tp->led_ctrl = LED_CTRL_MODE_MAC;
> +
> + /* Default to PHY_1_MODE if 0 (MAC_MODE) is
> + * read on some older 5700/5701 bootcode.
> + */
> + if (GET_ASIC_REV(tp->pci_chip_rev_id) ==
> + ASIC_REV_5700 ||
> + GET_ASIC_REV(tp->pci_chip_rev_id) ==
> + ASIC_REV_5701)
> + tp->led_ctrl = LED_CTRL_MODE_PHY_1;
> +
> break;
>
> case SHASTA_EXT_LED_SHARED:
>
>
>
^ permalink raw reply
* Re: ipw2100: firmware problem
From: Jeff Garzik @ 2005-06-09 23:13 UTC (permalink / raw)
To: Olivier Galibert
Cc: Andi Kleen, James Ketrenos, Netdev list, kernel list,
James P. Ketrenos
In-Reply-To: <20050609211242.GA30319@dspnet.fr.eu.org>
Olivier Galibert wrote:
> On Thu, Jun 09, 2005 at 03:56:15PM +0200, Andi Kleen wrote:
>
>>I guess at some point we will need a file system in there, but - oops -
>>we already have one, dont we? :)
>
>
> Well, you could put .config in it too.
>
> Frankly, a filesystem that:
> - can be somehow linked with vmlinux and not separate like an initrd
>
> - editable post vmlinux-linking
>
> - gives files that can be accessed from request_firmware, acpi and
> friends even rather early in the boot process (i.e. well before any
> userland is allowed to exist)
>
> - accessible post-boot through mounting of a special fs and/or /proc or something
>
> would be quite useful.
This exists. It's called initramfs. Read the kernel code :)
Jeff
^ permalink raw reply
* TCP stalls, 2.6.12pre6
From: Krzysztof Halasa @ 2005-06-09 22:58 UTC (permalink / raw)
To: netdev
Hi,
I got this three times recently, with 2.6.12 pre6 and possibly pre5 (4?)
kernels on both sides:
intrepid is X11-server with ssh connection to defiant (X11-forwarding,
EPIC100 PCI NIC). defiant is an older notebook machine and it was running
XEmacs and Firefox with ssh/X11 (cardbus DEC 21143).
Both on the same Ethernet subnet. Both standard MTU etc.
intrepid# netstat -to
tcp 0 0 intrepid:3457 defiant:ssh ESTABLISHED keepalive (5615.71/0/0)
defiant# netstat -to (not 100% sure about this data)
Send-Q
tcp 0 32800 defiant:ssh intrepid:3457 ESTABLISHED keepalive (20.5/0/0)
Partial dump:
intrepid# tcpdump -r qwe -vve port not 4840
00:08:32.603003 len 1514: IP (tos 0x10, ttl 64, id 17467, offset 0,
flags [DF], proto 6, length: 1500) defiant.ssh > intrepid.3457: .
[bad tcp cksum 2fa (->8fbe)!] 2999401769:2999403217(1448)
ack 1597770543 win 16022 <nop,nop,timestamp 131339306 29183805>
00:08:55.750326 len 158: IP (tos 0x10, ttl 64, id 13187, offset 0,
flags [DF], proto 6, length: 144) intrepid.3457 > defiant.ssh: P
[tcp sum ok] 1:81(80) ack 0 win 18856 <nop,nop,timestamp 29224727
131133680,nop,nop,sack sack 1 {1448:16816} >
00:08:55.750603 len 66: IP (tos 0x10, ttl 64, id 17469, offset 0,
flags [DF], proto 6, length: 52) defiant.ssh > intrepid.3457: .
[tcp sum ok] 16816:16816(0) ack 81 win 16022 <nop,nop,timestamp
131362456 29224727>
Send-Q is stuck at the fixed value. Other TCP connections (including
similar ones between the two hosts) are not affected.
Ideas?
--
Krzysztof Halasa
^ permalink raw reply
* Re: [PATCH] tg3: Fix link failure in 5701
From: Michael Chan @ 2005-06-09 22:55 UTC (permalink / raw)
To: Grant Grundler; +Cc: davem, peterc, netdev, netdev
In-Reply-To: <20050609232830.GC12434@esmail.cup.hp.com>
On Thu, 2005-06-09 at 16:28 -0700, Grant Grundler wrote:
> Michael, David,
> I confirmed this patch in fact fixes the problem on currently
> shipping rx7620 and rx8620 "Core LAN" cards. I expected it would
> but now have nice warm fuzzies that it really 100% does.
Thanks for testing, Grant.
>
> One minor issue: I unloaded the unpatched tg3 v3.29 driver and
> then did not get a link when loaded the patched tg3 v3.30 driver.
> Unplugging and replugging the cable made this work.
> v3.30 continued to work fine after a reboot.
>
This is odd. May be the link partner is in a bad state when previously
connected to unpatched tg3, and requires unplug to get out of that
state.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: David S. Miller @ 2005-06-09 22:22 UTC (permalink / raw)
To: shemminger
Cc: jesse.brandeburg, hadi, john.ronciak, mitch.a.williams, mchan,
buytenh, jdmason, netdev, Robert.Olsson, ganesh.venkatesan
In-Reply-To: <20050609150546.61b0fee7@unknown-215.office.pdx.osdl.net>
From: Stephen Hemminger <shemminger@osdl.org>
Date: Thu, 9 Jun 2005 15:05:46 -0700
> > I have profile data, here is an example of 5tx/5rx threads, where the
> > throughput was 1236Mb/s total, 936tx, 300rx, on 2.6.12-rc5 with old TSO
> > (the original problem case) we are at 100% cpu and generating 3289 ints/s,
> > with no hardware drops reported prolly due to my replenish patch
> > CPU: P4 / Xeon with 2 hyper-threads, speed 2791.36 MHz (estimated)
> > Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
> > samples % image name symbol name
> > 533687 8.1472 vmlinux pskb_expand_head
> > 428726 6.5449 vmlinux __copy_user_zeroing_intel
> > 349934 5.3421 vmlinux _read_lock_irqsave
>
> We should kill all reader/writer locks in the fastpath. reader locks are
> more expensive than spinlocks unless they are going to be held for a fairly
> large window.
True, but I see no reason why it should have any influence here.
Let's not get distracted by this in our analysis of the problem.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: jamal @ 2005-06-09 22:21 UTC (permalink / raw)
To: Jesse Brandeburg
Cc: Stephen Hemminger, David S. Miller, Ronciak, John,
Williams, Mitch A, mchan, buytenh, jdmason, netdev, Robert.Olsson,
Venkatesan, Ganesh
In-Reply-To: <Pine.LNX.4.62.0506091510560.16917@ladlxr>
On Thu, 2005-09-06 at 15:12 -0700, Jesse Brandeburg wrote:
> On Thu, 9 Jun 2005, Stephen Hemminger wrote:
> > > 313667 4.7884 vmlinux csum_partial
> > > 218870 3.3413 vmlinux _spin_lock
> > > 214302 3.2715 vmlinux __copy_user_intel
> > > 193662 2.9564 vmlinux skb_release_data
> > > 177755 2.7136 vmlinux ipt_do_table
> >
> > You are probably benchmarking iptables/netfilter! How many rules do you
> > have?
>
> I saw that... somehow iptables got compiled into kernel statically. no
> rules are active or installed iptables -L -n shows nothing in any chain.
Contracking is a lot worse of a problem. Just turn off netfilter all
together.
cheers,
jamal
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: David S. Miller @ 2005-06-09 22:21 UTC (permalink / raw)
To: jesse.brandeburg
Cc: shemminger, hadi, john.ronciak, mitch.a.williams, mchan, buytenh,
jdmason, netdev, Robert.Olsson, ganesh.venkatesan
In-Reply-To: <Pine.LNX.4.62.0506091510560.16917@ladlxr>
From: Jesse Brandeburg <jesse.brandeburg@intel.com>
Date: Thu, 9 Jun 2005 15:12:09 -0700 (PDT)
> I saw that... somehow iptables got compiled into kernel statically. no
> rules are active or installed iptables -L -n shows nothing in any chain.
Netfilter can kill performance, even if no rules are loaded at all.
Please take that out of your kernel.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: jamal @ 2005-06-09 22:20 UTC (permalink / raw)
To: Jesse Brandeburg
Cc: David S. Miller, Ronciak, John, shemminger, Williams, Mitch A,
mchan, buytenh, jdmason, netdev, Robert.Olsson,
Venkatesan, Ganesh
In-Reply-To: <Pine.LNX.4.62.0506091406590.10396@ladlxr>
On Thu, 2005-09-06 at 14:37 -0700, Jesse Brandeburg wrote:
> Okay let me clear this up once and for all, here is our test setup:
>
> * 10 1u rack machines (dual P3 - 1250MHz), with both windows and linux
> installed (running windows now)
> * Extreme 1gig switch
> * Dual 2.8 GHz P4 server, RHEL3 base, running 2.6.12-rc5 or supertso patch
>
> * the test entails transferring 1MB files of zeros from memory to memory,
> using TCP, with each client doing primary either send or recv, not both.
Linux as sender?
> > Even if they did have some smart ass thing in the middle that reorders,
> > it is still suprising that such a fast CPU cant handle a mere one Gig of
> > what seems to be MTU=1500 bytes sized packets.
>
> It can handle a single thread (or even 6) just fine, its after that we get
> in trouble somewhere.
>
Certainly interesting details?
> > I suppose a netstat -s would help for visualization in addition to those
> > dumps.
>
> Okay I have that data, do you want it for the old tso, supertso, or no tso
> at all?
>
hrmph - dont know. Dave could tell you.
I would say whatever you are running thats latest and greatest and
causes you trouble?
> > Heres what i am deducing from their data, correct me if i am wrong:
> > ->The evidence is that something is expensive in their code path (duh).
>
> Actually I've found that adding more threads (10 total) sending to the
> server, while keeping the transmit thread count constant yields an
> increase our throughput all the way to 1750+ Mb/s (with supertso)
>
Interesting tidbit
> > -> Whatever that expensive thing code is, it not helped by them
> > replenishing the descriptors after all the budget is exhausted since the
> > descriptor departure rate is much slower than packet arrival.
>
> I'm running all my tests with the replenish patch mentioned earlier in
> this thread.
>
Ok. When i said " in the data path" - it could be anything from the
driver all the way to the socket.
If you have some pig along that path - it would mean you get back less
often to replenish the descriptors.
> > ---> This is why they would be seeing that the reduction of weight
> > improves performance since the replenishing happens sooner with a
> > smaller weight.
>
> seems like we're past the weight problem now, should i start a new thread?
>
I think so.
> > ------> Clearly the driver needs some fixing - if they could do what
>
> I'm not convinced it is the driver that is having issues. We might be
> having some complex interaction with the stack, but I definitely think we
> have a lot of onion layers to hack through here, all of which are probably
> relevant.
>
I agree. But the driver could have some improvement as well if you did
what the other driver does ;->
> I have profile data, here is an example of 5tx/5rx threads, where the
> throughput was 1236Mb/s total, 936tx, 300rx, on 2.6.12-rc5 with old TSO
> (the original problem case) we are at 100% cpu and generating 3289 ints/s,
> with no hardware drops reported prolly due to my replenish patch
Hrm, reading Stephen email as well ;->
Can you turn off netfilter off totaly? Most importantly remove
contracking.
cheers,
jamal
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Jesse Brandeburg @ 2005-06-09 22:12 UTC (permalink / raw)
To: Stephen Hemminger
Cc: Brandeburg, Jesse, jamal, David S. Miller, Ronciak, John,
Williams, Mitch A, mchan, buytenh, jdmason, netdev, Robert.Olsson,
Venkatesan, Ganesh
In-Reply-To: <20050609150546.61b0fee7@unknown-215.office.pdx.osdl.net>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 564 bytes --]
On Thu, 9 Jun 2005, Stephen Hemminger wrote:
> > 313667 4.7884 vmlinux csum_partial
> > 218870 3.3413 vmlinux _spin_lock
> > 214302 3.2715 vmlinux __copy_user_intel
> > 193662 2.9564 vmlinux skb_release_data
> > 177755 2.7136 vmlinux ipt_do_table
>
> You are probably benchmarking iptables/netfilter! How many rules do you
> have?
I saw that... somehow iptables got compiled into kernel statically. no
rules are active or installed iptables -L -n shows nothing in any chain.
Jesse
^ permalink raw reply
* Re: ipw2100: firmware problem
From: David S. Miller @ 2005-06-09 22:11 UTC (permalink / raw)
To: jketreno; +Cc: pavel, vda, abonilla, jgarzik, netdev, linux-kernel,
ipw2100-admin
In-Reply-To: <42A8AE2A.4080104@linux.intel.com>
From: James Ketrenos <jketreno@linux.intel.com>
Date: Thu, 09 Jun 2005 16:01:30 -0500
> The ipw2100 originally postponed doing any initialization until open was
> called. The problem at that time was that distributions were crafted to
> rely on link detection (I believe via ethtoolop's get_link) before they
> would bring the interface up.
Yes, I see, and that does work for most ethernet devices.
I noticed that Debian's 3.1 installer used this to determine
which ethernet device it should use as the default in it's
network device dialogue.
One idea, returning true for get_link when the device is down, may not
be a bad idea for the wireless case.
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Stephen Hemminger @ 2005-06-09 22:05 UTC (permalink / raw)
To: Jesse Brandeburg
Cc: jamal, David S. Miller, Brandeburg, Jesse, Ronciak, John,
Williams, Mitch A, mchan, buytenh, jdmason, netdev, Robert.Olsson,
Venkatesan, Ganesh
In-Reply-To: <Pine.LNX.4.62.0506091406590.10396@ladlxr>
> I have profile data, here is an example of 5tx/5rx threads, where the
> throughput was 1236Mb/s total, 936tx, 300rx, on 2.6.12-rc5 with old TSO
> (the original problem case) we are at 100% cpu and generating 3289 ints/s,
> with no hardware drops reported prolly due to my replenish patch
> CPU: P4 / Xeon with 2 hyper-threads, speed 2791.36 MHz (estimated)
> Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
> samples % image name symbol name
> 533687 8.1472 vmlinux pskb_expand_head
> 428726 6.5449 vmlinux __copy_user_zeroing_intel
> 349934 5.3421 vmlinux _read_lock_irqsave
We should kill all reader/writer locks in the fastpath. reader locks are
more expensive than spinlocks unless they are going to be held for a fairly
large window.
> 313667 4.7884 vmlinux csum_partial
> 218870 3.3413 vmlinux _spin_lock
> 214302 3.2715 vmlinux __copy_user_intel
> 193662 2.9564 vmlinux skb_release_data
> 177755 2.7136 vmlinux ipt_do_table
You are probably benchmarking iptables/netfilter! How many rules do you
have?
^ permalink raw reply
* Re: RFC: NAPI packet weighting patch
From: Jesse Brandeburg @ 2005-06-09 21:37 UTC (permalink / raw)
To: jamal
Cc: David S. Miller, Brandeburg, Jesse, Ronciak, John, shemminger,
Williams, Mitch A, mchan, buytenh, jdmason, netdev, Robert.Olsson,
Venkatesan, Ganesh
In-Reply-To: <1118237775.6382.34.camel@localhost.localdomain>
[-- Attachment #1: Type: TEXT/PLAIN, Size: 7801 bytes --]
On Wed, 8 Jun 2005, jamal wrote:
> > Something is up, if a single gigabit TCP stream can fully CPU
> > load your machine. 10 gigabit, yeah, definitely all current
> > generation machines are cpu limited over that link speed, but
> > 1 gigabit should be no problem.
> >
>
> Yes, sir.
> BTW, all along i thought the sender and receiver are hooked up directly
> (there was some mention of chariot a while back).
Okay let me clear this up once and for all, here is our test setup:
* 10 1u rack machines (dual P3 - 1250MHz), with both windows and linux
installed (running windows now)
* Extreme 1gig switch
* Dual 2.8 GHz P4 server, RHEL3 base, running 2.6.12-rc5 or supertso patch
* the test entails transferring 1MB files of zeros from memory to memory,
using TCP, with each client doing primary either send or recv, not both.
> Even if they did have some smart ass thing in the middle that reorders,
> it is still suprising that such a fast CPU cant handle a mere one Gig of
> what seems to be MTU=1500 bytes sized packets.
It can handle a single thread (or even 6) just fine, its after that we get
in trouble somewhere.
> I suppose a netstat -s would help for visualization in addition to those
> dumps.
Okay I have that data, do you want it for the old tso, supertso, or no tso
at all?
> Heres what i am deducing from their data, correct me if i am wrong:
> ->The evidence is that something is expensive in their code path (duh).
Actually I've found that adding more threads (10 total) sending to the
server, while keeping the transmit thread count constant yields an
increase our throughput all the way to 1750+ Mb/s (with supertso)
> -> Whatever that expensive thing code is, it not helped by them
> replenishing the descriptors after all the budget is exhausted since the
> descriptor departure rate is much slower than packet arrival.
I'm running all my tests with the replenish patch mentioned earlier in
this thread.
> ---> This is why they would be seeing that the reduction of weight
> improves performance since the replenishing happens sooner with a
> smaller weight.
seems like we're past the weight problem now, should i start a new thread?
> ------> Clearly the driver needs some fixing - if they could do what
I'm not convinced it is the driver that is having issues. We might be
having some complex interaction with the stack, but I definitely think we
have a lot of onion layers to hack through here, all of which are probably
relevant.
> Even if they SACKed for every packet, this still would not make any
> sense. So i think a profile of where the cycles are spent would also
> help. I am suspecting the driver at this point but i could be wrong.
I have profile data, here is an example of 5tx/5rx threads, where the
throughput was 1236Mb/s total, 936tx, 300rx, on 2.6.12-rc5 with old TSO
(the original problem case) we are at 100% cpu and generating 3289 ints/s,
with no hardware drops reported prolly due to my replenish patch
CPU: P4 / Xeon with 2 hyper-threads, speed 2791.36 MHz (estimated)
Counted GLOBAL_POWER_EVENTS events (time during which processor is not stopped) with a unit mask of 0x01 (mandatory) count 100000
samples % image name symbol name
533687 8.1472 vmlinux pskb_expand_head
428726 6.5449 vmlinux __copy_user_zeroing_intel
349934 5.3421 vmlinux _read_lock_irqsave
313667 4.7884 vmlinux csum_partial
218870 3.3413 vmlinux _spin_lock
214302 3.2715 vmlinux __copy_user_intel
193662 2.9564 vmlinux skb_release_data
177755 2.7136 vmlinux ipt_do_table
148445 2.2662 vmlinux _write_lock_irqsave
148080 2.2606 vmlinux _read_unlock_bh
143308 2.1877 vmlinux tcp_sendmsg
115745 1.7670 vmlinux ip_queue_xmit
111487 1.7020 vmlinux __kfree_skb
108383 1.6546 vmlinux _spin_lock_irqsave
108071 1.6498 e1000.ko e1000_xmit_frame
107850 1.6464 vmlinux tcp_clean_rtx_queue
104552 1.5961 e1000.ko e1000_clean_tx_irq
101308 1.5466 e1000.ko e1000_clean_rx_irq
94297 1.4395 vmlinux __copy_from_user_ll
85170 1.3002 vmlinux kfree
76730 1.1714 vmlinux tcp_transmit_skb
70976 1.0835 vmlinux eth_type_trans
67381 1.0286 vmlinux tcp_rcv_established
64670 0.9872 vmlinux sub_preempt_count
64451 0.9839 vmlinux dev_queue_xmit
64010 0.9772 vmlinux skb_clone
62314 0.9513 vmlinux tcp_v4_rcv
61980 0.9462 vmlinux nf_iterate
60374 0.9217 vmlinux ip_finish_output
57407 0.8764 vmlinux _write_unlock_bh
56165 0.8574 vmlinux mark_offset_tsc
54673 0.8346 endpoint (no symbols)
52662 0.8039 vmlinux __kmalloc
50112 0.7650 vmlinux sock_wfree
50001 0.7633 vmlinux _spin_trylock
47053 0.7183 vmlinux _read_lock_bh
45988 0.7021 vmlinux tcp_write_xmit
44229 0.6752 vmlinux kmem_cache_alloc
43506 0.6642 vmlinux smp_processor_id
42401 0.6473 vmlinux ip_conntrack_find_get
42095 0.6426 vmlinux alloc_skb
40619 0.6201 vmlinux tcp_in_window
38098 0.5816 vmlinux add_preempt_count
37701 0.5755 vmlinux __copy_to_user_ll
31529 0.4813 vmlinux ip_conntrack_in
31314 0.4780 vmlinux kmem_cache_free
30954 0.4725 vmlinux __ip_conntrack_find
30863 0.4712 vmlinux local_bh_enable
30774 0.4698 vmlinux tcp_packet
29426 0.4492 vmlinux _spin_unlock_irqrestore
28716 0.4384 vmlinux hash_conntrack
27073 0.4133 vmlinux ip_route_input
26540 0.4052 e1000.ko e1000_clean
25817 0.3941 vmlinux nf_hook_slow
23395 0.3571 vmlinux schedule
22981 0.3508 vmlinux tcp_v4_send_check
22139 0.3380 vmlinux __mod_timer
22126 0.3378 vmlinux timer_interrupt
21511 0.3284 vmlinux cache_alloc_refill
21161 0.3230 vmlinux netif_receive_skb
20418 0.3117 vmlinux _write_lock_bh
19443 0.2968 vmlinux skb_copy_datagram_iovec
19100 0.2916 vmlinux ip_nat_fn
18784 0.2868 vmlinux ip_local_deliver
18251 0.2786 vmlinux _read_lock
17513 0.2674 vmlinux nat_packet
17124 0.2614 e1000.ko e1000_intr
16357 0.2497 vmlinux default_idle
15358 0.2345 vmlinux qdisc_restart
14564 0.2223 vmlinux _read_unlock
14360 0.2192 vmlinux tcp_recvmsg
13853 0.2115 oprofiled odb_insert
13374 0.2042 e1000.ko e1000_alloc_rx_buffers
13321 0.2034 vmlinux apic_timer_interrupt
12668 0.1934 vmlinux pfifo_fast_enqueue
12618 0.1926 vmlinux tcp_sack
12180 0.1859 vmlinux ip_nat_local_fn
11434 0.1746 vmlinux system_call
11426 0.1744 vmlinux free_block
11377 0.1737 vmlinux try_to_wake_up
11138 0.1700 vmlinux irq_entries_start
11017 0.1682 vmlinux ipt_route_hook
10987 0.1677 vmlinux dev_queue_xmit_nit
10970 0.1675 vmlinux tcp_push_one
10508 0.1604 vmlinux tcp_error
10365 0.1582 vmlinux pfifo_fast_dequeue
10323 0.1576 vmlinux ip_rcv
10022 0.1530 vmlinux ip_output
^ permalink raw reply
* Re: RFC: PHY Abstraction Layer II
From: Andy Fleming @ 2005-06-09 21:32 UTC (permalink / raw)
To: Glen Turner; +Cc: Stephen Hemminger, Netdev, Kumar Gala
In-Reply-To: <42A360A0.1040902@aarnet.edu.au>
On Jun 5, 2005, at 15:29, Glen Turner wrote:
>
> Operationally, it would be very useful if the PHY printed
> the physical interface detail when detected (1000Base-LX,
> etc).
I was thinking that it would be easier for the ethernet driver to do
this in the adjust_link() function, since it's going to need to track
when these things change, anyway. But if the general consensus is
that it should be in the generic code, I can see about adding it there.
>
> Also, it would be nice to be able to retrieve PHY data
> independent of the interface status (eg, to retrieve
> asset serial numbers, GBIC make/models, etc).
I'm not sure what you mean, here. The driver can use phy_read/write
to get/set information anytime it wants.
Andy Fleming
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox