Re: sk98lin for 2.6.23-rc1

Re: sk98lin for 2.6.23-rc1

[Daniel J Blueman]

> If anyone still has hang issues with sky2, please send me the hardware
> information (lspci, dmesg output), and capture the debugfs state after hang.
>
> At present the known open skge, sky2 issues are:
[snip]

There is still this active issue which I believe is orthogonal from
the interrupt hang. I just hit it again. The IRQ-hang recovery logic
doesn't seem to kick in, so it's perhaps another issue:

# ifconfig lan0
lan0      Link encap:Ethernet  HWaddr 00:03:2D:05:9C:27
          inet addr:192.168.0.250  Bcast:192.168.0.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:13304841 errors:1 dropped:1 overruns:0 frame:2
          TX packets:7493765 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:232720755 (221.9 MiB)  TX bytes:3964088142 (3.6 GiB)
          Interrupt:16

# ethtool -S lan0
NIC statistics:
     tx_bytes: 44342166229
     rx_bytes: 28807483162
     tx_broadcast: 20071
     rx_broadcast: 3309
     tx_multicast: 0
     rx_multicast: 31
     tx_unicast: 45095221
     rx_unicast: 43331790
     tx_mac_pause: 0
     rx_mac_pause: 278
     collisions: 0
     late_collision: 0
     aborted: 0
     single_collisions: 0
     multi_collisions: 0
     rx_short: 0
     rx_runt: 0
     rx_64_byte_packets: 1815009
     rx_65_to_127_byte_packets: 18626585
     rx_128_to_255_byte_packets: 788459
     rx_256_to_511_byte_packets: 4880174
     rx_512_to_1023_byte_packets: 1016831
     rx_1024_to_1518_byte_packets: 16208350
     rx_1518_to_max_byte_packets: 0
     rx_too_long: 0
     rx_fifo_overflow: 0
     rx_jabber: 0
     rx_fcs_error: 0
     tx_64_byte_packets: 1632797
     tx_65_to_127_byte_packets: 11211285
     tx_128_to_255_byte_packets: 1628446
     tx_256_to_511_byte_packets: 2042646
     tx_512_to_1023_byte_packets: 637342
     tx_1024_to_1518_byte_packets: 27962776
     tx_1519_to_max_byte_packets: 0
     tx_fifo_underrun: 0

# dmesg
[snip]
sky2 0000:01:00.0: v1.16 addr 0xdfbfc000 irq 16 Yukon-EC (0xb6) rev 1
sky2 eth1: addr 00:03:2d:05:9c:27
sky2 lan0: enabling interface
sky2 lan0: ram buffer 48K
[snip]
sky2 lan0: rx error, status 0xad78ad78 length 0
lan0: hw csum failure.
[stack-trace snipped]
lan0: hw csum failure.
...

# cat /debug/sky2/lan0
IRQ src=0 mask=c000001d control=0
Status ring (empty)
Tx ring pending=424...424 report=424 done=424

Rx ring hw get=252 put=405 last=1023

# cat /debug/sky2/lan0
IRQ src=0 mask=c000001d control=0
Status ring (empty)
Tx ring pending=432...432 report=432 done=432

Rx ring hw get=252 put=415 last=1023

# cat /debug/sky2/lan0
IRQ src=0 mask=c000001d control=0
Status ring (empty)
Tx ring pending=451...451 report=451 done=451

Rx ring hw get=316 put=440 last=1023
-- 
Daniel J Blueman

[PATCH 2/3] sk98lin: ethtool perm_addr build fix

[Stephen Hemminger]

Deal with API changes while sk98lin was removed.
ethtool_ops no longer has a perm_addr hook.

Signed-off-by: Stephen Hemminger <shemminger@linux-foundation.org>
---
 drivers/net/sk98lin/skethtool.c |    1 -
 1 files changed, 0 insertions(+), 1 deletions(-)

diff --git a/drivers/net/sk98lin/skethtool.c b/drivers/net/sk98lin/skethtool.c
index 3646069..5a6da89 100644
--- a/drivers/net/sk98lin/skethtool.c
+++ b/drivers/net/sk98lin/skethtool.c
@@ -616,7 +616,6 @@ const struct ethtool_ops SkGeEthtoolOps = {
 	.get_pauseparam		= getPauseParams,
 	.set_pauseparam		= setPauseParams,
 	.get_link		= ethtool_op_get_link,
-	.get_perm_addr		= ethtool_op_get_perm_addr,
 	.get_sg			= ethtool_op_get_sg,
 	.set_sg			= setScatterGather,
 	.get_tx_csum		= ethtool_op_get_tx_csum,
-- 
1.5.2.5

[PATCH 3/3]: sk98lin: neuter device to only SysKonnect boards

[Stephen Hemminger]

The skge driver works better for all boards except older SysKonnect
boards.

Signed-off-by: Stephen Hemminger <shemminger@linux-foundation.org>
---
 drivers/net/sk98lin/skge.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)

diff --git a/drivers/net/sk98lin/skge.c b/drivers/net/sk98lin/skge.c
index bf21862..7dc9c9e 100644
--- a/drivers/net/sk98lin/skge.c
+++ b/drivers/net/sk98lin/skge.c
@@ -5168,10 +5168,17 @@ err_out:
 #endif
 
 static struct pci_device_id skge_pci_tbl[] = {
+#ifdef SK98LIN_ALL_DEVICES
 	{ PCI_VENDOR_ID_3COM, 0x1700, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
 	{ PCI_VENDOR_ID_3COM, 0x80eb, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
+#endif
+#ifdef GENESIS
+	/* Generic SysKonnect SK-98xx Gigabit Ethernet Server Adapter */	
 	{ PCI_VENDOR_ID_SYSKONNECT, 0x4300, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
+#endif
+	/* Generic SysKonnect SK-98xx V2.0 Gigabit Ethernet Adapter */	
 	{ PCI_VENDOR_ID_SYSKONNECT, 0x4320, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
+#ifdef SK98LIN_ALL_DEVICES
 /* DLink card does not have valid VPD so this driver gags
  *	{ PCI_VENDOR_ID_DLINK, 0x4c00, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
  */
@@ -5180,6 +5187,7 @@ static struct pci_device_id skge_pci_tbl[] = {
 	{ PCI_VENDOR_ID_CNET, 0x434e, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
 	{ PCI_VENDOR_ID_LINKSYS, 0x1032, PCI_ANY_ID, 0x0015, },
 	{ PCI_VENDOR_ID_LINKSYS, 0x1064, PCI_ANY_ID, PCI_ANY_ID, 0, 0, 0 },
+#endif
 	{ 0 }
 };
 
-- 
1.5.2.5

Re: [PATCH 3/3]: sk98lin: neuter device to only SysKonnect boards

[Jeff Garzik]

applied 1-3 as a single commit, so that git-bisect will not break

	Jeff

[PATCH 1/3] sk98lin: restore driver

[Stephen Hemminger]

This reverts commit e1abecc48938fbe1966ea6e78267fc673fa59295.

The driver works on some hardware that skge doesn't handle yet.

Signed-off-by: Stephen Hemminger <shemminger@linux-foundation.org>

-----------
Patch too large for mailing list. Download from:
	http://developer.osdl.org/shemminger/patches/sk98lin-2.6.23-restore.patch

Re: sk98lin for 2.6.23-rc1

[Adrian Bunk]

On Tue, Sep 11, 2007 at 10:05:26AM +0200, Stephen Hemminger wrote:
> 
> There are several different problems in this thread:
> 1. The removal of old sk98lin driver caused some users to be forced to use
>     skge. These users have uncovered issues with the dual port fiber based versions
>     of the board.  
>     Short term: The sk98lin driver should be restored to previous state, 
>        and the PCI table should be used to limit the usage to only fiber systems.
>        If Adrian doesn't do it, I'll do it when I return from Germany.
>...

No problem with this, but since it was Jeff's patch it should better be 
him who reverts it (and he's anyway one step nearer to Linus).

But the underlying general problem still remains:

How can we get people to test and report bugs with the new drivers 
before removing the old driver?

That's a question especially for the people who now had problems after 
sk98lin was removed.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: sk98lin for 2.6.23-rc1

[Bill Davidsen]

Adrian Bunk wrote:
> On Tue, Sep 11, 2007 at 10:05:26AM +0200, Stephen Hemminger wrote:
>   
>> There are several different problems in this thread:
>> 1. The removal of old sk98lin driver caused some users to be forced to use
>>     skge. These users have uncovered issues with the dual port fiber based versions
>>     of the board.  
>>     Short term: The sk98lin driver should be restored to previous state, 
>>        and the PCI table should be used to limit the usage to only fiber systems.
>>        If Adrian doesn't do it, I'll do it when I return from Germany.
>> ...
>>     
>
> No problem with this, but since it was Jeff's patch it should better be 
> him who reverts it (and he's anyway one step nearer to Linus).
>
> But the underlying general problem still remains:
>
> How can we get people to test and report bugs with the new drivers 
> before removing the old driver?
>
>   
Sorry for a long answer, I'm trying to provide insight on two recent cases.

Thinking back to several drivers, when e100 was new I tried it because I 
had problems with eepro100 in the area of multiple cards, multiple 
cables on a single card, and jumbo packets. For a while I used both, 
until e100 worked where I need it. So I initially tried it because it 
had features I needed, and then dropped to older driver just to avoid 
having to decide.

With sk98lin, the driver worked flawlessly with all (3-4) systems, so I 
had no reason to try any other. When removing sk98lin was first 
proposed, I tried skge, first measurements showed it was 5-8% slower, 
NOT what I want, so I went back. For me there was no reliability issue, 
but I never tried it in a system with more than on NIC on the driver. 
Would "it's a little slower" be a valid bug report? Or would I have 
gotten "works fine for me" from people not beating it over Gbit? I 
didn't try sky2 until you suggested it, and I have reported my results 
previously, just stops working. Could it be my hardware? I tried it on 
one system, so yes, but sk98lin works for months.
> That's a question especially for the people who now had problems after 
> sk98lin was removed.

So if you want people to try a new driver, I think it really has to have 
some benefits to the users, in terms of performance, reliability, or 
features. "Cleaner design" doesn't motivate, and it does raise the 
question of why the old driver wasn't just cleaned up. I've been doing 
software for decades, I appreciate why, but users in general just want 
to use their system. Which raises the question of why to delete drivers 
which work for many or even most users? Testing a new kernel is no 
longer a drop in a boot operation if modprobe.conf must be edited to get 
the network up, and the typical user isn't going to write that shell 
script to try one or the other driver.

Honestly, new drivers which offer little benefit to most users are the 
exception rather than the rule, so this may a corner case I would like 
to see sk98lin back in the kernel, for a while I can build my own 
kernels and patch it in, but until other drivers are drop-in, I probably 
won't change.

Separate but related: why keep skge and sky2? Are we going through this 
again in a year? Is the benefit worth the effort?

Hope some of this is helpful.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: sk98lin for 2.6.23-rc1

[Adrian Bunk]

On Tue, Sep 11, 2007 at 10:29:47AM -0400, Bill Davidsen wrote:
> Adrian Bunk wrote:
>> On Tue, Sep 11, 2007 at 10:05:26AM +0200, Stephen Hemminger wrote:
>>   
>>> There are several different problems in this thread:
>>> 1. The removal of old sk98lin driver caused some users to be forced to 
>>> use
>>>     skge. These users have uncovered issues with the dual port fiber 
>>> based versions
>>>     of the board.      Short term: The sk98lin driver should be restored 
>>> to previous state,        and the PCI table should be used to limit the 
>>> usage to only fiber systems.
>>>        If Adrian doesn't do it, I'll do it when I return from Germany.
>>> ...
>>>     
>>
>> No problem with this, but since it was Jeff's patch it should better be 
>> him who reverts it (and he's anyway one step nearer to Linus).
>>
>> But the underlying general problem still remains:
>>
>> How can we get people to test and report bugs with the new drivers before 
>> removing the old driver?
>>
>>   
> Sorry for a long answer, I'm trying to provide insight on two recent cases.
>
> Thinking back to several drivers, when e100 was new I tried it because I 
> had problems with eepro100 in the area of multiple cards, multiple cables 
> on a single card, and jumbo packets. For a while I used both, until e100 
> worked where I need it. So I initially tried it because it had features I 
> needed, and then dropped to older driver just to avoid having to decide.
>
> With sk98lin, the driver worked flawlessly with all (3-4) systems, so I had 
> no reason to try any other. When removing sk98lin was first proposed, I 
> tried skge, first measurements showed it was 5-8% slower, NOT what I want, 
> so I went back. For me there was no reliability issue, but I never tried it 
> in a system with more than on NIC on the driver. Would "it's a little 
> slower" be a valid bug report? Or would I have gotten "works fine for me" 
> from people not beating it over Gbit?
>...

If you get less throughput that is a regression, and it should be 
reported and fixed.

I doubt anybody would have told you otherwise.

Is this bug still present as of 2.6.23-rc6?

>> That's a question especially for the people who now had problems after 
>> sk98lin was removed.
>
> So if you want people to try a new driver, I think it really has to have 
> some benefits to the users, in terms of performance, reliability, or 
> features. "Cleaner design" doesn't motivate, and it does raise the question 
> of why the old driver wasn't just cleaned up. I've been doing software for 
> decades, I appreciate why, but users in general just want to use their 
> system. Which raises the question of why to delete drivers which work for 
> many or even most users?

As I already explained, there is a long term advantage for all users if 
there is only one driver in the kernel. Therefore all users should 
switch away from obsolete drivers to the replacement drivers, and the 
obsolete driver will be removed at some point in time. The only question 
is how to do it.

> Testing a new kernel is no longer a drop in a boot 
> operation if modprobe.conf must be edited to get the network up, and the 
> typical user isn't going to write that shell script to try one or the other 
> driver.

The typical user will let his distribution handle this.

And MODULE_ALIAS can also handle this.

> Honestly, new drivers which offer little benefit to most users are the 
> exception rather than the rule, so this may a corner case I would like to 
> see sk98lin back in the kernel, for a while I can build my own kernels and 
> patch it in, but until other drivers are drop-in, I probably won't change.

That a new driver offers benefits that cause most users to switch isn't 
realistic.

You mention e100 as an example - well, I'm using this driver in my 
computer, but I doubt anything would be worse for me if I'd use the 
obsolete eepro100 driver instead since I'm not using any of the fancy 
e100 features you mentioned as advantages.

There is a long term advantage for all users if there is only one driver 
in the kernel. Therefore all users should switch away from obsolete 
drivers to the replacement drivers, and the obsolete driver will be 
removed at some point in time. The only question is how to do it.

> Separate but related: why keep skge and sky2? Are we going through this 
> again in a year? Is the benefit worth the effort?
>...

skge and sky2 support distinct hardware.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: sk98lin for 2.6.23-rc1

[Willy Tarreau]

On Tue, Sep 11, 2007 at 05:03:57PM +0200, Adrian Bunk wrote:
> On Tue, Sep 11, 2007 at 10:29:47AM -0400, Bill Davidsen wrote:
> > So if you want people to try a new driver, I think it really has to have 
> > some benefits to the users, in terms of performance, reliability, or 
> > features. "Cleaner design" doesn't motivate, and it does raise the question 
> > of why the old driver wasn't just cleaned up. I've been doing software for 
> > decades, I appreciate why, but users in general just want to use their 
> > system. Which raises the question of why to delete drivers which work for 
> > many or even most users?
> 
> As I already explained, there is a long term advantage for all users if 
> there is only one driver in the kernel.

Not only that. You have to place the switch in its context with history.
Stephen, please correct me if I'm wrong, but sk98lin has been randomly
working for a very long time. Not 100% the driver's fault, because it
has had to workaround a lot of chips bugs. The fact that this driver
supports *all* chips in the family makes it harder to identify whether
problems are caused by the hardware or by the driver because it is
bloated with tons of if/else.

I've personally encountered random data corruption on the receive path
with PCI-E hardware with sk98lin, as well as random TX stops. Sometimes
it would require one terabyte of data, sometimes just a few hundreds
megs. On other hardware (skge now), UDP would simply stop being sent
and some TCP traffic was necessary to restart UDP! One guy at Marvell
once asked me for more information, but it was not easy to provide
much more, given the randomness of the problems!

Stephen has done an excellent (and thankless) job at restarting from
scratch, and the idea to separate the two chips was a good one IMHO.
The problem is that he might have thought that most of the bugs were
in the driver, while most of them are in the hardware, and this requires
a lot of workarounds, which do not always work the same for everybody
(I remember having tried to disable flow control with sk98lin because
it helped with sky2).

In parallel, sk98lin has improved on the vendor's site. v8 exhibited
all the problems I explained above, but v10 has fixed a lot of them,
making the new sk98lin more reliable. In parallel, sky2 and skge had
got wider acceptance and testing. The nastiest hardware bugs will
slowly surface, a good deal of driver bugs have been detected too
(and that's expected from any new driver).

It is possible that after 2 or 3 patches, a lot of the remaining
problems will suddenly vanish. But it's also possible that the driver
will still not work for 1% of people for 1 or 2 years because of some
obscure hardware combinations which trigger some obscure hardware bugs.

> Therefore all users should 
> switch away from obsolete drivers to the replacement drivers, and the 
> obsolete driver will be removed at some point in time. The only question 
> is how to do it.

Desktop users genreally have no problem experimenting with multiple kernels
or drivers. They can report feedback too, but generally, they're not very
good at downloading alternative drivers and patching their kernel with those.

Server users cannot experiment for a long time. After 2 or 3 losses of
service, they *have* to provide a definitive solution. For some of them
when sky2 fails, it may very well be to switch over to sk98lin. Downloading
from the vendor's site and patching is not a problem for those users, but
it causes them the trouble of updating the kernel for security fixes, so
the old driver must be shipped with the kernel.

However, I remember something which might constitute a solution. In 2.4,
there's a small bug in the kbuild process on alpha. One question is always
asked during make oldconfig. Its saved value is ignored because of the way
it is computed. I don't know if we could do this with 2.6 kbuild. It would
then be nice to always set sk98lin to unset if it was set to "Y" or "M",
so that at each build, the user has to explicitly state he wants it. It's
annoying enough to give the other one a try once in a while, without causing
too much trouble to people who really have no other choice right now.

What we need with this driver is people being fed up with it, not them
being unable to use it as a last resort. Also, given that it has improved
over the last years (probably due to competition pressure from sky2/skge),
users will even less understand why there is such incentive to remove it.

Another trick for obsolete drivers would be to simply remove them from
the usual build system, but have them being available for explicit build.
Eg: make modules will not build them, but make obsolete-modules would do.

> > Testing a new kernel is no longer a drop in a boot 
> > operation if modprobe.conf must be edited to get the network up, and the 
> > typical user isn't going to write that shell script to try one or the other 
> > driver.
> 
> The typical user will let his distribution handle this.
> 
> And MODULE_ALIAS can also handle this.

No system config should be edited to switch back to the alternative,
otherwise it remains in its working state.

> > Honestly, new drivers which offer little benefit to most users are the 
> > exception rather than the rule, so this may a corner case I would like to 
> > see sk98lin back in the kernel, for a while I can build my own kernels and 
> > patch it in, but until other drivers are drop-in, I probably won't change.
> 
> That a new driver offers benefits that cause most users to switch isn't 
> realistic.

Desktop users are curious and have plenty of time to kill. Server users
are frightened and lazy. So I think that annoying the user slightly is
a good solution (eg: make obsolete-modules).

> You mention e100 as an example - well, I'm using this driver in my 
> computer, but I doubt anything would be worse for me if I'd use the 
> obsolete eepro100 driver instead since I'm not using any of the fancy 
> e100 features you mentioned as advantages.

After having been happy with eepro100 for years, I discovered many problems
with its VLAN support in 2.4 (MTU, ...) for which e100 was a solution. It
was a good reason to switch. But the old e100 driver took ages to load (half
of the machine boot time), which was not satisfying. So having a new driver
load faster is another good reason to switch.

> There is a long term advantage for all users if there is only one driver 
> in the kernel. Therefore all users should switch away from obsolete 
> drivers to the replacement drivers, and the obsolete driver will be 
> removed at some point in time. The only question is how to do it.

Hmmm we already read this paragraph above :-)

> > Separate but related: why keep skge and sky2? Are we going through this 
> > again in a year? Is the benefit worth the effort?
> >...
> 
> skge and sky2 support distinct hardware.

... and as such are both smaller than sk98lin which supports both.

Cheers,
Willy

Re: sk98lin for 2.6.23-rc1

[Stephen Hemminger]

If anyone still has hang issues with sky2, please send me the hardware
information (lspci, dmesg output), and capture the debugfs state after hang.

At present the known open skge, sky2 issues are:

1) Skge doesn't work with dual port fiber
2) Sky2 (and vendor sk98lin) don't work on some MSI motherboards
    due to PCI issues
3) Skge and sky2 trigger problems with motherboards that don't
    do >4GB DMA correctly. Not a driver but a PCI quirk problem.
4) Sky2 does polling for lost irq even when interface not up
5) Sky2 polling rate for lost irq maybe greater than TCP timeout so
    connection may be lost even after link recovers
6) Sky2 Yukon extreme support is still experimental and fragile,
    but this hardware isn't in the wild yet
7) Sky2 Yukon EC-U not powering up PHY correctly on some revisions
8) Sky2 suspend to ram not working right on some chips

Hardware nuisances that are being masked:
a) hardware can't DMA to unaligned address
b) polling for lost irq is a hack, but until the root cause
   is found it will have to stay.

Re: sk98lin for 2.6.23-rc1

[Jeff Garzik]

Stephen Hemminger wrote:
> If anyone still has hang issues with sky2, please send me the hardware
> information (lspci, dmesg output), and capture the debugfs state after hang.
> 
> At present the known open skge, sky2 issues are:

Did you add Chris Stromsoe's skge-hangs problem to the list?

	Jeff

Re: sk98lin for 2.6.23-rc1

[Stephen Hemminger]

On Fri, 27 Jul 2007 08:58:45 -0400
Jeff Garzik <jeff@garzik.org> wrote:

> Stephen Hemminger wrote:
> > If anyone still has hang issues with sky2, please send me the hardware
> > information (lspci, dmesg output), and capture the debugfs state after hang.
> > 
> > At present the known open skge, sky2 issues are:
> 
> Did you add Chris Stromsoe's skge-hangs problem to the list?
> 
> 	Jeff

That is the #1 on list, it is the dual port fiber problem.
Seems to be specific to dual port, since can't reproduce on my only
skge fiber board (single port) connected to e1000.