PROBLEM: Alpha SMP Low Outbound Bandwidth

PROBLEM: Alpha SMP Low Outbound Bandwidth

[Jay Thorne]

[1.] One line summary of the problem:
Kernel 2.4.4 ac15
Tested with several cards and pieces of software, the outbound bandwidth
on a quad cpu alpha is 2 megabytes a second or less on a 100 mbit
switched ethernet network. Other machines on same switch do 10 or more
megabytes per second. Switch is DLink 3624, 24 port, only 12 ports in
use.

[2.] Full description of the problem/report:
Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards,
on wuFTP, the outbound bandwidth tops out at 2 megabytes per second and
the inbound at 6 megabytes per second.  Also noticeable are apparent
slowdowns or console lockups/sluggishness during the transfer.

[3.] Keywords (i.e., modules, networking, kernel):
networking, alpha, tulip, via_rhine

[4.] Kernel version (from /proc/version):
Linux version 2.4.4-ac15 (root@lister) (gcc version 2.96 20000731 (SuSE
Linux 7.1/Alpha)) #1 SMP Thu May 24 18:41:13 PDT 2001    

[5.] Output of Oops.. message (if applicable) with symbolic information
     resolved (see Documentation/oops-tracing.txt)

[6.] A small shell script or example program which triggers the
     problem (if possible)

Problem machine:
ncftp /tmp > put foo
foo:                                                    34.38 MB    5.16
MB/s
ncftp /tmp > get -z foo baz
baz:                                                    34.38 MB    1.16
MB/s

other machine on same switch to same ftp server.
ncftp /home/jay > get foo
foo:                                                    34.38 MB   10.12
MB/s
ncftp /home/jay > put -z foo baz
foo:                                                    34.38 MB    9.93
MB/s

[7.] Environment
[7.1.] Software (add the output of the ver_linux script here)
 
Linux lister 2.4.4-ac15 #1 SMP Thu May 24 18:41:13 PDT 2001 alpha
unknown
 
Gnu C                  2.96
Gnu make               3.79.1
binutils               2.10.0.33
util-linux             2.10q
mount                  2.10q
modutils               2.4.2
e2fsprogs              1.19
pcmcia-cs              3.1.22
PPP                    2.4.0
isdn4k-utils           3.1pre1a
Linux C Library        so.6.1
Dynamic linker (ldd)   2.2
Procps                 2.0.7
Net-tools              1.57
Kbd                    1.02
Sh-utils               2.0
Modules Loaded         tulip via-rhine

[7.2.] Processor information (from /proc/cpuinfo):
lister:/usr/src/linux # cat /proc/cpuinfo
cpu                     : Alpha
cpu model               : EV56
cpu variation           : 7
cpu revision            : 0
cpu serial number       :
system type             : Rawhide
system variation        : Dodge
system revision         : 0
system serial number    : NI70904KB0
cycle frequency [Hz]    : 400000000
timer frequency [Hz]    : 1200.00
page size [bytes]       : 8192
phys. address bits      : 40
max. addr. space #      : 127
BogoMIPS                : 738.12
kernel unaligned acc    : 1646246
(pc=fffffc000042a3d8,va=fffffc005d9b784e)
user unaligned acc      : 0 (pc=0,va=0)
platform string         : AlphaServer 4100 5/400 4MB
cpus detected           : 4
cpus active             : 4
cpu active mask         : 000000000000000f

[7.3.] Module information (from /proc/modules):
lister:/usr/src/linux # cat /proc/modules
tulip                  59296   1
via-rhine              16464   0 (autoclean)

[7.4.] Loaded driver and hardware information (/proc/ioports,
/proc/iomem)
lister:/usr/src/linux # cat /proc/ioports
00000000-0000ffff : PCI IO bus 0
  00000000-0000001f : dma1
  00000020-0000003f : pic1
  00000040-0000005f : timer
  00000060-0000006f : keyboard
  00000070-00000080 : rtc
    00000070-0000007f : rtc
  000000a0-000000bf : pic2
  000000c0-000000df : dma2
  000002f8-000002ff : serial(auto)
  000003f8-000003ff : serial(auto)
  00008000-000080ff : VIA Technologies, Inc. Ethernet Controller
    00008000-000080ff : via-rhine
  00008400-0000847f : Digital Equipment Corporation DECchip 21140
[FasterNet]
    00008400-0000847f : tulip
200000000-20000ffff : PCI IO bus 1
  200008000-2000080ff : Symbios Logic Inc. (formerly NCR) 53c810
    200008000-20000807f : ncr53c8xx
  200009000-2000090fe : qlogicisp
lister:/usr/src/linux # cat /proc/iomem
00000000-ffffffff : PCI mem bus 0
  00000000-07ffffff : HAE0
    02200000-0223ffff : Digital Equipment Corporation DECchip 21140
[FasterNet]
    02240000-0224ffff : S3 Inc. 86c764/765 [Trio32/64/64V+]
    02250000-0225ffff : VIA Technologies, Inc. Ethernet Controller
    02260000-022600ff : VIA Technologies, Inc. Ethernet Controller
      02260000-022600ff : via-rhine
    02261000-0226107f : Digital Equipment Corporation DECchip 21140
[FasterNet]
      02261000-0226107f : tulip
200000000-2ffffffff : PCI mem bus 1
  200000000-207ffffff : HAE0
    202200000-2022000ff : Symbios Logic Inc. (formerly NCR) 53c810

[7.5.] PCI information ('lspci -vvv' as root)
lister:/usr/src/linux # lspci -vvv
00:01.0 Non-VGA unclassified device: Intel Corporation 82375EB (rev 05)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32
        Interrupt: pin ? routed to IRQ 32

00:02.0 VGA compatible controller: S3 Inc. 86c764/765 [Trio32/64/64V+]
(rev 54) (prog-if 00 [VGA])
        Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Interrupt: pin A routed to IRQ 16
        Region 0: Memory at 000000000c000000 (32-bit, non-prefetchable)
        Expansion ROM at 0000000002240000 [size=64K]

00:03.0 Ethernet controller: Digital Equipment Corporation DECchip 21140
[FasterNet] (rev 22)
        Subsystem: Digital Equipment Corporation: Unknown device 500a
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (5000ns min, 10000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 20
        Region 0: I/O ports at 8400 [size=128]
        Region 1: Memory at 0000000002261000 (32-bit, non-prefetchable)
[size=128]
        Expansion ROM at 0000000002200000 [disabled] [size=256K]
 
00:05.0 Ethernet controller: VIA Technologies, Inc. Ethernet Controller
(rev 42)
        Subsystem: D-Link System Inc: Unknown device 1400
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping+ SERR+ FastB2B-
        Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32 (750ns min, 2000ns max), cache line size 10
        Interrupt: pin A routed to IRQ 28
        Region 0: I/O ports at 8000 [size=256]
        Region 1: Memory at 0000000002260000 (32-bit, non-prefetchable)
[size=256]
        Expansion ROM at 0000000002250000 [disabled] [size=64K]
        Capabilities: [40] Power Management version 2
                Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot-,D3cold-)
                Status: D0 PME-Enable- DSel=0 DScale=0 PME-
 
01:01.0 SCSI storage controller: Symbios Logic Inc. (formerly NCR)
53c810 (rev 02)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32
        Interrupt: pin A routed to IRQ 56
        Region 0: I/O ports at 200008000 [size=256]
        Region 1: Memory at 0000000202200000 (32-bit, non-prefetchable)
[size=256]
 
01:02.0 PCI bridge: Digital Equipment Corporation DECchip 21050 (rev 02)
(prog-if 00 [Normal decode])
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 10
        Bus: primary=01, secondary=02, subordinate=02, sec-latency=0
        I/O behind bridge: 00009000-00009fff
        Memory behind bridge: 02300000-023fffff
        Prefetchable memory behind bridge: 00100000-000fffff
        BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
 
02:00.0 SCSI storage controller: Q Logic ISP1020 (rev 02)
        Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr+ Stepping- SERR+ FastB2B-
        Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
<TAbort- <MAbort- >SERR- <PERR-
        Latency: 32, cache line size 10
        Interrupt: pin A routed to IRQ 40
        Region 0: I/O ports at 200009000 [size=256]
        Region 1: Memory at 0000000202310000 (32-bit, non-prefetchable)
[size=4K]
        Expansion ROM at 0000000202300000 [disabled] [size=64K]

[7.6.] SCSI information (from /proc/scsi/scsi)
lister:/usr/src/linux # cat /proc/scsi/scsi
Attached devices:
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: DEC      Model: RZ1CB-CS (C) DEC Rev: 0844
  Type:   Direct-Access                    ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 01 Lun: 00
  Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
  Type:   Direct-Access                    ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 02 Lun: 00
  Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
  Type:   Direct-Access                    ANSI SCSI revision: 02
Host: scsi0 Channel: 00 Id: 03 Lun: 00
  Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
  Type:   Direct-Access                    ANSI SCSI revision: 02
Host: scsi1 Channel: 00 Id: 05 Lun: 00
  Vendor: DEC      Model: RRD45   (C) DEC  Rev: 0436
  Type:   CD-ROM                           ANSI SCSI revision: 02

[7.7.] Other information that might be relevant to the problem
       (please look in /proc and include all information that you
       think to be relevant):

There is a tcpdump available. 4.6 megabytes for anyone who wants it.



[X.] Other notes, patches, fixes, workarounds:

During one of the slowdowns, the console appeared to lock up. holding
down the ALT key seemed to make it wake up.




-- 
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[George France]

Hello Jay, 

I see that you are using the tulip driver.  Could you try the de4x5 driver??

Best Regards,


--George


On Friday 25 May 2001 17:50, Jay Thorne wrote:
> [1.] One line summary of the problem:
> Kernel 2.4.4 ac15
> Tested with several cards and pieces of software, the outbound bandwidth
> on a quad cpu alpha is 2 megabytes a second or less on a 100 mbit
> switched ethernet network. Other machines on same switch do 10 or more
> megabytes per second. Switch is DLink 3624, 24 port, only 12 ports in
> use.
>
> [2.] Full description of the problem/report:
> Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards,
> on wuFTP, the outbound bandwidth tops out at 2 megabytes per second and
> the inbound at 6 megabytes per second.  Also noticeable are apparent
> slowdowns or console lockups/sluggishness during the transfer.
>
> [3.] Keywords (i.e., modules, networking, kernel):
> networking, alpha, tulip, via_rhine
>
> [4.] Kernel version (from /proc/version):
> Linux version 2.4.4-ac15 (root@lister) (gcc version 2.96 20000731 (SuSE
> Linux 7.1/Alpha)) #1 SMP Thu May 24 18:41:13 PDT 2001
>
> [5.] Output of Oops.. message (if applicable) with symbolic information
>      resolved (see Documentation/oops-tracing.txt)
>
> [6.] A small shell script or example program which triggers the
>      problem (if possible)
>
> Problem machine:
> ncftp /tmp > put foo
> foo:                                                    34.38 MB    5.16
> MB/s
> ncftp /tmp > get -z foo baz
> baz:                                                    34.38 MB    1.16
> MB/s
>
> other machine on same switch to same ftp server.
> ncftp /home/jay > get foo
> foo:                                                    34.38 MB   10.12
> MB/s
> ncftp /home/jay > put -z foo baz
> foo:                                                    34.38 MB    9.93
> MB/s
>
> [7.] Environment
> [7.1.] Software (add the output of the ver_linux script here)
>
> Linux lister 2.4.4-ac15 #1 SMP Thu May 24 18:41:13 PDT 2001 alpha
> unknown
>
> Gnu C                  2.96
> Gnu make               3.79.1
> binutils               2.10.0.33
> util-linux             2.10q
> mount                  2.10q
> modutils               2.4.2
> e2fsprogs              1.19
> pcmcia-cs              3.1.22
> PPP                    2.4.0
> isdn4k-utils           3.1pre1a
> Linux C Library        so.6.1
> Dynamic linker (ldd)   2.2
> Procps                 2.0.7
> Net-tools              1.57
> Kbd                    1.02
> Sh-utils               2.0
> Modules Loaded         tulip via-rhine
>
> [7.2.] Processor information (from /proc/cpuinfo):
> lister:/usr/src/linux # cat /proc/cpuinfo
> cpu                     : Alpha
> cpu model               : EV56
> cpu variation           : 7
> cpu revision            : 0
> cpu serial number       :
> system type             : Rawhide
> system variation        : Dodge
> system revision         : 0
> system serial number    : NI70904KB0
> cycle frequency [Hz]    : 400000000
> timer frequency [Hz]    : 1200.00
> page size [bytes]       : 8192
> phys. address bits      : 40
> max. addr. space #      : 127
> BogoMIPS                : 738.12
> kernel unaligned acc    : 1646246
> (pc=fffffc000042a3d8,va=fffffc005d9b784e)
> user unaligned acc      : 0 (pc=0,va=0)
> platform string         : AlphaServer 4100 5/400 4MB
> cpus detected           : 4
> cpus active             : 4
> cpu active mask         : 000000000000000f
>
> [7.3.] Module information (from /proc/modules):
> lister:/usr/src/linux # cat /proc/modules
> tulip                  59296   1
> via-rhine              16464   0 (autoclean)
>
> [7.4.] Loaded driver and hardware information (/proc/ioports,
> /proc/iomem)
> lister:/usr/src/linux # cat /proc/ioports
> 00000000-0000ffff : PCI IO bus 0
>   00000000-0000001f : dma1
>   00000020-0000003f : pic1
>   00000040-0000005f : timer
>   00000060-0000006f : keyboard
>   00000070-00000080 : rtc
>     00000070-0000007f : rtc
>   000000a0-000000bf : pic2
>   000000c0-000000df : dma2
>   000002f8-000002ff : serial(auto)
>   000003f8-000003ff : serial(auto)
>   00008000-000080ff : VIA Technologies, Inc. Ethernet Controller
>     00008000-000080ff : via-rhine
>   00008400-0000847f : Digital Equipment Corporation DECchip 21140
> [FasterNet]
>     00008400-0000847f : tulip
> 200000000-20000ffff : PCI IO bus 1
>   200008000-2000080ff : Symbios Logic Inc. (formerly NCR) 53c810
>     200008000-20000807f : ncr53c8xx
>   200009000-2000090fe : qlogicisp
> lister:/usr/src/linux # cat /proc/iomem
> 00000000-ffffffff : PCI mem bus 0
>   00000000-07ffffff : HAE0
>     02200000-0223ffff : Digital Equipment Corporation DECchip 21140
> [FasterNet]
>     02240000-0224ffff : S3 Inc. 86c764/765 [Trio32/64/64V+]
>     02250000-0225ffff : VIA Technologies, Inc. Ethernet Controller
>     02260000-022600ff : VIA Technologies, Inc. Ethernet Controller
>       02260000-022600ff : via-rhine
>     02261000-0226107f : Digital Equipment Corporation DECchip 21140
> [FasterNet]
>       02261000-0226107f : tulip
> 200000000-2ffffffff : PCI mem bus 1
>   200000000-207ffffff : HAE0
>     202200000-2022000ff : Symbios Logic Inc. (formerly NCR) 53c810
>
> [7.5.] PCI information ('lspci -vvv' as root)
> lister:/usr/src/linux # lspci -vvv
> 00:01.0 Non-VGA unclassified device: Intel Corporation 82375EB (rev 05)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32
>         Interrupt: pin ? routed to IRQ 32
>
> 00:02.0 VGA compatible controller: S3 Inc. 86c764/765 [Trio32/64/64V+]
> (rev 54) (prog-if 00 [VGA])
>         Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Interrupt: pin A routed to IRQ 16
>         Region 0: Memory at 000000000c000000 (32-bit, non-prefetchable)
>         Expansion ROM at 0000000002240000 [size=64K]
>
> 00:03.0 Ethernet controller: Digital Equipment Corporation DECchip 21140
> [FasterNet] (rev 22)
>         Subsystem: Digital Equipment Corporation: Unknown device 500a
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV+ VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32 (5000ns min, 10000ns max), cache line size 10
>         Interrupt: pin A routed to IRQ 20
>         Region 0: I/O ports at 8400 [size=128]
>         Region 1: Memory at 0000000002261000 (32-bit, non-prefetchable)
> [size=128]
>         Expansion ROM at 0000000002200000 [disabled] [size=256K]
>
> 00:05.0 Ethernet controller: VIA Technologies, Inc. Ethernet Controller
> (rev 42)
>         Subsystem: D-Link System Inc: Unknown device 1400
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping+ SERR+ FastB2B-
>         Status: Cap+ 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32 (750ns min, 2000ns max), cache line size 10
>         Interrupt: pin A routed to IRQ 28
>         Region 0: I/O ports at 8000 [size=256]
>         Region 1: Memory at 0000000002260000 (32-bit, non-prefetchable)
> [size=256]
>         Expansion ROM at 0000000002250000 [disabled] [size=64K]
>         Capabilities: [40] Power Management version 2
>                 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
> PME(D0+,D1-,D2-,D3hot-,D3cold-)
>                 Status: D0 PME-Enable- DSel=0 DScale=0 PME-
>
> 01:01.0 SCSI storage controller: Symbios Logic Inc. (formerly NCR)
> 53c810 (rev 02)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32
>         Interrupt: pin A routed to IRQ 56
>         Region 0: I/O ports at 200008000 [size=256]
>         Region 1: Memory at 0000000202200000 (32-bit, non-prefetchable)
> [size=256]
>
> 01:02.0 PCI bridge: Digital Equipment Corporation DECchip 21050 (rev 02)
> (prog-if 00 [Normal decode])
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B+ ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32, cache line size 10
>         Bus: primary=01, secondary=02, subordinate=02, sec-latency=0
>         I/O behind bridge: 00009000-00009fff
>         Memory behind bridge: 02300000-023fffff
>         Prefetchable memory behind bridge: 00100000-000fffff
>         BridgeCtl: Parity- SERR- NoISA+ VGA- MAbort- >Reset- FastB2B-
>
> 02:00.0 SCSI storage controller: Q Logic ISP1020 (rev 02)
>         Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr+ Stepping- SERR+ FastB2B-
>         Status: Cap- 66Mhz- UDF- FastB2B- ParErr- DEVSEL=medium >TAbort-
> <TAbort- <MAbort- >SERR- <PERR-
>         Latency: 32, cache line size 10
>         Interrupt: pin A routed to IRQ 40
>         Region 0: I/O ports at 200009000 [size=256]
>         Region 1: Memory at 0000000202310000 (32-bit, non-prefetchable)
> [size=4K]
>         Expansion ROM at 0000000202300000 [disabled] [size=64K]
>
> [7.6.] SCSI information (from /proc/scsi/scsi)
> lister:/usr/src/linux # cat /proc/scsi/scsi
> Attached devices:
> Host: scsi0 Channel: 00 Id: 00 Lun: 00
>   Vendor: DEC      Model: RZ1CB-CS (C) DEC Rev: 0844
>   Type:   Direct-Access                    ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 01 Lun: 00
>   Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
>   Type:   Direct-Access                    ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 02 Lun: 00
>   Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
>   Type:   Direct-Access                    ANSI SCSI revision: 02
> Host: scsi0 Channel: 00 Id: 03 Lun: 00
>   Vendor: DEC      Model: RZ1CB-CA (C) DEC Rev: LYJ0
>   Type:   Direct-Access                    ANSI SCSI revision: 02
> Host: scsi1 Channel: 00 Id: 05 Lun: 00
>   Vendor: DEC      Model: RRD45   (C) DEC  Rev: 0436
>   Type:   CD-ROM                           ANSI SCSI revision: 02
>
> [7.7.] Other information that might be relevant to the problem
>        (please look in /proc and include all information that you
>        think to be relevant):
>
> There is a tcpdump available. 4.6 megabytes for anyone who wants it.
>
>
>
> [X.] Other notes, patches, fixes, workarounds:
>
> During one of the slowdowns, the console appeared to lock up. holding
> down the ALT key seemed to make it wake up.

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[Jay Thorne]

On 25 May 2001 18:52:33 -0400, George France wrote:
> Hello Jay, 
> 
> I see that you are using the tulip driver.  Could you try the de4x5 driver??
> 
Its worse: reports 3.1 MBs and 1.6 MBs

-- 
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[George France]

On Friday 25 May 2001 19:05, Jay Thorne wrote:
> On 25 May 2001 18:52:33 -0400, George France wrote:
> > Hello Jay,
> >
> > I see that you are using the tulip driver.  Could you try the de4x5
> > driver??
>
> Its worse: reports 3.1 MBs and 1.6 MBs

wuftp is not exactly a performance benchmark, have you tried 'netperf'?

--George

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[Jay Thorne]

On 25 May 2001 19:31:21 -0400, George France wrote:
> On Friday 25 May 2001 19:05, Jay Thorne wrote:
> > On 25 May 2001 18:52:33 -0400, George France wrote:
> > > Hello Jay,
> > >
> > > I see that you are using the tulip driver.  Could you try the de4x5
> > > driver??
> >
> > Its worse: reports 3.1 MBs and 1.6 MBs
> 
> wuftp is not exactly a performance benchmark, have you tried 'netperf'?
> 
> --George

While I agree with you completely that wuftpd is not exactly a
performance leader, this is the simplest way to recreate a problem I was
having with a much more complex setup involving apache and SMP and a
whole bunch of things. 

I posted 2 weeks ago and got no response, I assume because everyone
thought it was my software. After reducing the problem to eliminate the
possibility that my code is the real problem, I'm left with a quite
repeatable state. I have two nearly identical machines, one with 466 mhz
cpus the other with 400mhz, and they both do the same thing. The
via-rhine performs similarly to the de4x5.

Netperf is a pretty good idea. Should not be a cpu bottleneck. Thats a
good thing. So pretty much the same results as wu-ftpd: Note that I used
the 466 mhz quad with a via-rhine, since the 400 locked up and was still
fscking when I started this test.

             Recv   Send    Send                          
             Socket Socket  Message  Elapsed              
             Size   Size    Size     Time     Throughput  
             bytes  bytes   bytes    secs.    10^6bits/sec  

To alpha     87380  16384  16384    10.02      39.25   
x86 local    87380  16384  16384    9.99      559.46
alpha local  87380  16384  16384    10.01     547.27   
alp to x86   87380  16384  16384    10.01      25.77   
another x86  87380  16384  16384    9.99      553.67   
to same x86  87380  16384  16384    10.00      82.79   
and back     87380  16384  16384    10.00      93.89   

But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough
that anyone with an alpha running SMP can test it. Note that this
software and the server in question were tested to run at 10+ megabytes
per second with x86 boxes. The server is a PIII500 running 2.4.4, so its
not like I'm comparing apples to oranges. The second x86 is an athlon
600.

So even factoring out wuftp is not helping much here. I'm fairly
convinced that something is strange because after the de4x5 test, the
box locked up. So either a> I have two identically boned 4 cpu boxen
or b> the interprocessor/locking/resource management has some kind of
problem. Note that under uniprocessor I get near identical to x86
performance, clock for clock and no lock ups.


-- 
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[Andrea Arcangeli]

On Fri, May 25, 2001 at 05:25:03PM -0700, Jay Thorne wrote:
> But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough
> that anyone with an alpha running SMP can test it. Note that this

My smp alpha box drives a single tulip over 12MB/sec in full duplex
using tcp without any problem at all. So I definitely cannot reproduce.
You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not
tried it yet you can consider also using egcs 1.1.2 as compiler just in
case.

You may also want to keep an eye on the VM, on alpha I see very weird
things happening.

Andrea

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[Michal Jaegermann]

On Fri, May 25, 2001 at 02:50:07PM -0700, Jay Thorne wrote:
> [1.] One line summary of the problem:
> Kernel 2.4.4 ac15
....
> Using a quad 400Mhz Dodge/Rawhide machine with Tulip or VIARhine cards,
....
[ description of a slowdown skipped ].

Well, it looks that you have at least something to slow down.  I could
not get a single packet through my tulip on Alpha from at least
2.4.4-ac11 and up.  You can consider that an ultimate slowdown.  I tried
also a driver from http://sourceforge.net/projects/tulip/ and results
are the same.  This NIC, Digital DS21143 Tulip rev 65, works just fine
with various earlier kernels, including assorted 2.4.3 variants.
It is on 10baseT netwok - which may, or may not, be relevant here.

  Michal

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[George France]

Hello Andrea,

Jay, if the problem still exist in 2.4.5-pre6aa1 (please try the new kernel), 
then I will have tech op's check this on Tuesday (Monday is a US holiday).  
We should be able to duplicate this in the hardware lab and find the problem 
with a logic analyser.

Best Regards,


--George

On Friday 25 May 2001 20:51, Andrea Arcangeli wrote:
> On Fri, May 25, 2001 at 05:25:03PM -0700, Jay Thorne wrote:
> > But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough
> > that anyone with an alpha running SMP can test it. Note that this
>
> My smp alpha box drives a single tulip over 12MB/sec in full duplex
> using tcp without any problem at all. So I definitely cannot reproduce.
> You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not
> tried it yet you can consider also using egcs 1.1.2 as compiler just in
> case.
>
> You may also want to keep an eye on the VM, on alpha I see very weird
> things happening.
>
> Andrea
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: PROBLEM: Alpha SMP Low Outbound Bandwidth

[Urban Widmark]

On 25 May 2001, Jay Thorne wrote:

> Netperf is a pretty good idea. Should not be a cpu bottleneck. Thats a
> good thing. So pretty much the same results as wu-ftpd: Note that I used
> the 466 mhz quad with a via-rhine, since the 400 locked up and was still
> fscking when I started this test.
> 
>              Recv   Send    Send                          
>              Socket Socket  Message  Elapsed              
>              Size   Size    Size     Time     Throughput  
>              bytes  bytes   bytes    secs.    10^6bits/sec  
> 
> To alpha     87380  16384  16384    10.02      39.25   
> x86 local    87380  16384  16384    9.99      559.46
> alpha local  87380  16384  16384    10.01     547.27   
> alp to x86   87380  16384  16384    10.01      25.77   
> another x86  87380  16384  16384    9.99      553.67   
> to same x86  87380  16384  16384    10.00      82.79   
> and back     87380  16384  16384    10.00      93.89   

What type of NIC is in the x86'es? If they are not the same, what happens
if you put one of those in the alphas?
(and what happens with the via-rhine if put in one of the x86'es?)


Alphas don't like unaligned memory accesses (not sure how bad those are).  
I think that you can get some idea on the frequence of those with 'cat
/proc/cpuinfo' where it should say "kernel unaligned acc" and then some
numbers.

If that first number keeps going up then the driver(s) or something isn't
being nice. Well, just a though. Oh yes, your original message showed a
value of 1646246 for kernel space unaligned accesses. Is that high?

The userspace value is 0, so I assume that is high. The value for 'pc='
should give the address of where the last unaligned access took place
(look it up in System.map or /proc/ksyms)

Does the value grow faster when you run your netperf tests?


That still doesn't explain the SMP vs UP difference.

/Urban

[SOLVED] PROBLEM: Alpha SMP Low Outbound Bandwidth

[Jay Thorne]

Problem solved, thanks to the rawhide patch from Richard Henderson
(rth@twiddle.net) posted on Sunday. Performance is ~10megs/second both
directions, using tulip, de4x5 or via-rhine.

Using 2.4.4-ac15 it works fine. I'm now trying 2.4.5

Andrea, 2.4.5aa1 oopses just after probing the scsi cards. I've tried
the 2.4.4 series aa patches and had similar failure on boot. 

Its too fast to see the error, so I'm building a serial console version
to capture it. Is an easy way to tell an alpha to stop dead so I can
copy the oops?


On 25 May 2001 23:16:34 -0400, George France wrote:
> Hello Andrea,
> 
> Jay, if the problem still exist in 2.4.5-pre6aa1 (please try the new kernel), 
> then I will have tech op's check this on Tuesday (Monday is a US holiday).  
> We should be able to duplicate this in the hardware lab and find the problem 
> with a logic analyser.
> 
> Best Regards,
> 
> 
> --George
> 
> On Friday 25 May 2001 20:51, Andrea Arcangeli wrote:
> > On Fri, May 25, 2001 at 05:25:03PM -0700, Jay Thorne wrote:
> > > But Wu-ftpd is an easy to set up test bench, and is ubiquitous enough
> > > that anyone with an alpha running SMP can test it. Note that this
> >
> > My smp alpha box drives a single tulip over 12MB/sec in full duplex
> > using tcp without any problem at all. So I definitely cannot reproduce.
> > You may want to try to reproduce with 2.4.5pre6aa1 btw. If you've not
> > tried it yet you can consider also using egcs 1.1.2 as compiler just in
> > case.
> >
> > You may also want to keep an eye on the VM, on alpha I see very weird
> > things happening.
> >
> > Andrea
> > -
> > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/

-- 
--
Jay Thorne Manager, Systems & Technology, UserFriendly Media, Inc.

Re: [SOLVED] PROBLEM: Alpha SMP Low Outbound Bandwidth

[George France]

On Monday 28 May 2001 13:45, Jay Thorne wrote:
> Problem solved, thanks to the rawhide patch from Richard Henderson
> (rth@twiddle.net) posted on Sunday. Performance is ~10megs/second both
> directions, using tulip, de4x5 or via-rhine.

Well Done, Richard.

>
> Using 2.4.4-ac15 it works fine. I'm now trying 2.4.5
>
> Andrea, 2.4.5aa1 oopses just after probing the scsi cards. I've tried
> the 2.4.4 series aa patches and had similar failure on boot.
>
> Its too fast to see the error, so I'm building a serial console version
> to capture it. Is an easy way to tell an alpha to stop dead so I can
> copy the oops?

try adding 'console=ttyS0,9600 console=tty0' to the comand line args passed 
to the kernel at boot time.  if you are using  SRM and aboot, 'b <dev> -fl i' 
followed by the 'l' command, then a 'b' command.

regards,


--George