PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Thomas Kear]

USB3 gigabit ethernet adapters with the ASIX AX88179 chipset (LevelOne
USB0401-V3, Plugable USB3-E1000, SIIG JU-NE0211-S1 and others) are
experiencing kernel panics in usb_hcd_map_urb_for_dma since 3.12.  The
issue does not seem to directly correlate with low or high network
activity, occurring seemingly at random.  Some panics occurred less
than 5 minutes from boot and tens of megabytes of network transfer,
while on other occasions it would be stable for multiple days with
tens to hundreds of gigabytes of line-rate throughput and several
sleep/resume cycles.

Both my Sony Vaio Pro 13 and another user reporting this issue on the
Arch forums [1] are Intel-based, my USB controller is an 8086:9c31
(Lynx Point LP), the other is reported as a C210/7 series (unknown
PID).  A third with a Haswell Dell XPS has attempted my workaround and
reports similar success.

I have a mediocre quality photo of my laptop's screen from one of
these panics [2], the call trace - which is similar but not identical
between my machine and that of the other user reporting the issue - is
as follows:

usb_hcd_map_urb_for_dma
usb_hcd_submit_urb
local_bh_enable_ip
selinux_parse_skb
usb_alloc_urb
__kmalloc
usbnet_start_xmit
usbnet_start_xmit
dev_hard_start_xmit
sch_direct_xmit
dev_queue_xmit
ip_finish_output2
ip_finish_output
ip_output
dst_output
ip_local_out
ip_queue_xmit
tcp_transmit_skb
tcp_write_xmit
__tcp_push_pending_frames
tcp_push
tcp_sendmsg
inet_sendmsg
__sock_sendmsg_nosec
sock_sendmsg
set_restore_sigmask
set_restore_sigmask
fget_light
SYSC_sendto
set_restore_sigmask
SyS_sendto
system_call_fastpath

So far as I can tell, the driver is unaffected as late as 3.11.6, but
problematic as of 3.12 (and still affected in 3.13-rc5).  The history
of drivers/net/usb/ax88179_178a.c for this time period yields this
patch, which at least in my somewhat limited understanding appeared a
likely candidate.  I've reverted this on my system - against several
linux-next builds from the last 3-4 weeks - and have had no issues
with this network controller since.

commit 3804fad45411b48233b48003e33a78f290d227c8
Author: Ming Lei <ming.lei@canonical.com>
Date:   Thu Aug 8 21:48:25 2013 +0800

    USBNET: ax88179_178a: enable tso if usb host supports sg dma

    This patch enables 'can_dma_sg' flag for ax88179_178a device
    if the attached host controller supports building packet from
    discontinuous buffers(DMA SG is possible), so TSO can be enabled
    and skb fragment buffers can be passed to usb stack via urb->sg
    directly.

    With the patch, system CPU utilization decreased ~50% and throughput
    increased by ~10% when doing iperf client test on one ARM A15 dual
    core board.

    Cc: Ben Hutchings <bhutchings@solarflare.com>
    Cc: Grant Grundler <grundler@google.com>
    Cc: Oliver Neukum <oneukum@suse.de>
    Cc: Alan Stern <stern@rowland.harvard.edu>
    Cc: Freddy Xin <freddy@asix.com.tw>
    Signed-off-by: Ming Lei <ming.lei@canonical.com>
    Acked-by: Eric Dumazet <edumazet@gmail.com>
    Acked-by: David S. Miller <davem@davemloft.net>
    Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>

Unfortunately I have not retained the built kernel from a broken 3.12
build, so the system information below reflects the patched linux-next
kernel I am running currently.

I understand this may be a somewhat obscure piece of hardware, I am
willing to assist by drop-shipping one to someone from Amazon (or
local country equivalent if the price is not extortionate) should it
be required.



System information:

ver_linux:
Linux neko 3.13.0-rc5-next-20131224+ #1 SMP Sat Dec 28 19:09:27 NZDT
2013 x86_64 Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz GenuineIntel
GNU/Linux

Gnu C                  4.7.3
Gnu make               4.0
binutils               2.24
util-linux             scripts/ver_linux: line 23: fdformat: command not found
mount                  assert
module-init-tools      16
e2fsprogs              1.42.9
jfsutils               1.1.15
reiserfsprogs          3.6.24
reiser4progs           1.0.7
xfsprogs               3.1.11
quota-tools            4.01.
PPP                    2.4.5
Linux C Library        2.17
Dynamic linker (ldd)   2.17
Procps                 3.3.9
Net-tools              1.60_p20130513023548
Kbd                    2.0.1
Sh-utils               8.22
Modules Loaded         bonding rndis_host cdc_ether tun snd_usb_audio
snd_usbmidi_lib snd_rawmidi cdc_acm ctr ccm hidp nfsd rfcomm bnep
iptable_nat nf_nat_ipv4 nf_nat uvcvideo btusb bluetooth hid_multitouch
videobuf2_vmalloc videobuf2_memops videobuf2_core uinput ax88179_178a
usbnet mii rtsx_pci_sdmmc rtsx_pci mmc_core fuse snd_hda_codec_realtek
iwlmvm kvm_intel snd_hda_codec_generic mac80211 kvm pn544_mei mei_phy
iwlwifi pn544 snd_hda_intel hci nfc snd_hda_codec snd_hwdep cfg80211
xhci_hcd

/proc/cpuinfo:
processor       : 3
vendor_id       : GenuineIntel
cpu family      : 6
model           : 69
model name      : Intel(R) Core(TM) i7-4500U CPU @ 1.80GHz
stepping        : 1
microcode       : 0x10
cpu MHz         : 2968.125
cache size      : 4096 KB
physical id     : 0
siblings        : 4
core id         : 1
cpu cores       : 2
apicid          : 3
initial apicid  : 3
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts
rep_good nopl xtopology nonstop_tsc aperfmperf eagerfpu pni pclmulqdq
dtes64 monitor ds_cpl vmx est tm2 ssse3 fma cx16 xtpr pdcm pcid sse4_1
sse4_2 movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand
lahf_lm abm ida arat epb xsaveopt pln pts dtherm tpr_shadow vnmi
flexpriority ept vpid fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms
invpcid
bogomips        : 4788.92
clflush size    : 64
cache_alignment : 64
address sizes   : 39 bits physical, 48 bits virtual
power management:

PCI:
-[0000:00]-+-00.0  Intel Corporation Haswell-ULT DRAM Controller [8086:0a04]
           +-02.0  Intel Corporation Haswell-ULT Integrated Graphics
Controller [8086:0a16]
           +-03.0  Intel Corporation Device [8086:0a0c]
           +-14.0  Intel Corporation Lynx Point-LP USB xHCI HC [8086:9c31]
           +-16.0  Intel Corporation Lynx Point-LP HECI #0 [8086:9c3a]
           +-1b.0  Intel Corporation Lynx Point-LP HD Audio Controller
[8086:9c20]
           +-1c.0-[01]----00.0  Intel Corporation Wireless 7260 [8086:08b1]
           +-1c.3-[02]--
           +-1c.4-[03]----00.0  Samsung Electronics Co Ltd Device [144d:a800]
           +-1d.0  Intel Corporation Lynx Point-LP USB EHCI #1 [8086:9c26]
           +-1f.0  Intel Corporation Lynx Point-LP LPC Controller [8086:9c43]
           \-1f.3  Intel Corporation Lynx Point-LP SMBus Controller [8086:9c22]

USB:
Bus 001 Device 002: ID 8087:8000 Intel Corp.
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 003 Device 025: ID 0b95:1790 ASIX Electronics Corp.
Bus 003 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 027: ID 8087:07dc Intel Corp.
Bus 002 Device 003: ID 04f2:b3be Chicony Electronics Co., Ltd
Bus 002 Device 002: ID 0eef:a108 D-WAV Scientific Co., Ltd
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

/proc/modules:
bonding 84837 0 - Live 0xffffffffc00d5000
rndis_host 5162 0 - Live 0xffffffffc0096000
cdc_ether 4324 1 rndis_host, Live 0xffffffffc006e000
tun 16811 0 - Live 0xffffffffc009e000
snd_usb_audio 102474 0 - Live 0xffffffffc007b000
snd_usbmidi_lib 16542 1 snd_usb_audio, Live 0xffffffffc0072000
snd_rawmidi 15891 1 snd_usbmidi_lib, Live 0xffffffffc0017000
cdc_acm 16166 0 - Live 0xffffffffc05ec000
ctr 3471 2 - Live 0xffffffffc05e8000
ccm 6977 2 - Live 0xffffffffc05e3000
hidp 12989 0 - Live 0xffffffffc05db000
nfsd 192979 13 - Live 0xffffffffc059b000
rfcomm 27704 12 - Live 0xffffffffc058e000
bnep 9055 2 - Live 0xffffffffc0587000
iptable_nat 2550 0 - Live 0xffffffffc0583000
nf_nat_ipv4 3118 1 iptable_nat, Live 0xffffffffc057f000
nf_nat 9984 2 iptable_nat,nf_nat_ipv4, Live 0xffffffffc0577000
uvcvideo 60542 0 - Live 0xffffffffc0562000
btusb 14182 0 - Live 0xffffffffc0519000
bluetooth 200149 23 hidp,rfcomm,bnep,btusb, Live 0xffffffffc04d8000
hid_multitouch 8791 0 - Live 0xffffffffc04d1000
videobuf2_vmalloc 2528 1 uvcvideo, Live 0xffffffffc04cd000
videobuf2_memops 1559 1 videobuf2_vmalloc, Live 0xffffffffc04c9000
videobuf2_core 22473 1 uvcvideo, Live 0xffffffffc04be000
uinput 6657 0 - Live 0xffffffffc04b9000
ax88179_178a 11352 0 - Live 0xffffffffc04b2000
usbnet 17066 3 rndis_host,cdc_ether,ax88179_178a, Live 0xffffffffc04a7000
mii 3427 2 ax88179_178a,usbnet, Live 0xffffffffc04a3000
rtsx_pci_sdmmc 8434 0 - Live 0xffffffffc049d000
rtsx_pci 24242 1 rtsx_pci_sdmmc, Live 0xffffffffc0490000
mmc_core 73734 1 rtsx_pci_sdmmc, Live 0xffffffffc0471000
fuse 65180 0 - Live 0xffffffffc0458000
snd_hda_codec_realtek 37786 1 - Live 0xffffffffc0448000
iwlmvm 99382 0 - Live 0xffffffffc03d9000
kvm_intel 119916 0 - Live 0xffffffffc033e000
snd_hda_codec_generic 39626 1 snd_hda_codec_realtek, Live 0xffffffffc032d000
mac80211 352537 1 iwlmvm, Live 0xffffffffc0239000
kvm 332678 1 kvm_intel, Live 0xffffffffc014f000
pn544_mei 1507 0 - Live 0xffffffffc014b000
mei_phy 1942 1 pn544_mei, Live 0xffffffffc0147000
iwlwifi 70257 1 iwlmvm, Live 0xffffffffc0129000
pn544 6215 1 pn544_mei, Live 0xffffffffc0124000
snd_hda_intel 29721 2 - Live 0xffffffffc0106000
hci 13343 2 mei_phy,pn544, Live 0xffffffffc00fd000
nfc 46459 2 pn544,hci, Live 0xffffffffc00c8000
snd_hda_codec 72588 3
snd_hda_codec_realtek,snd_hda_codec_generic,snd_hda_intel, Live
0xffffffffc00a7000
snd_hwdep 5373 2 snd_usb_audio,snd_hda_codec, Live 0xffffffffc009b000
cfg80211 320801 3 iwlmvm,mac80211,iwlwifi, Live 0xffffffffc001e000
xhci_hcd 88348 0 - Live 0xffffffffc0000000

/proc/iomem:
00000000-00000fff : reserved
00001000-00057fff : System RAM
00058000-00058fff : reserved
00059000-0009dfff : System RAM
0009e000-0009ffff : reserved
000a0000-000bffff : PCI Bus 0000:00
000c0000-000c3fff : PCI Bus 0000:00
000c4000-000c7fff : PCI Bus 0000:00
000c8000-000cbfff : PCI Bus 0000:00
000cc000-000cffff : PCI Bus 0000:00
000d0000-000d3fff : PCI Bus 0000:00
000d4000-000d7fff : PCI Bus 0000:00
000d8000-000dbfff : PCI Bus 0000:00
000dc000-000dffff : PCI Bus 0000:00
000f0000-000fffff : System ROM
00100000-ca4b7fff : System RAM
  06000000-066753ba : Kernel code
  066753bb-06cc83ff : Kernel data
  06dc9000-06ecffff : Kernel bss
ca4b8000-ca4befff : ACPI Non-volatile Storage
ca4bf000-ca8e7fff : System RAM
ca8e8000-cac39fff : reserved
cac3a000-da89bfff : System RAM
da89c000-dab3ffff : reserved
dab40000-dab55fff : ACPI Tables
dab56000-dbaaafff : ACPI Non-volatile Storage
dbaab000-dbffefff : reserved
dbfff000-dbffffff : System RAM
dd000000-df1fffff : reserved
  dd200000-df1fffff : Graphics Stolen Memory
df200000-feafffff : PCI Bus 0000:00
  df200000-df3fffff : PCI Bus 0000:01
  e0000000-efffffff : 0000:00:02.0
    e0000000-e07e8fff : BOOTFB
  f0000000-f09fffff : PCI Bus 0000:03
  f0a00000-f13fffff : PCI Bus 0000:02
  f6400000-f67fffff : 0000:00:02.0
  f6800000-f71fffff : PCI Bus 0000:03
    f6800000-f680ffff : 0000:03:00.0
    f6810000-f6811fff : 0000:03:00.0
      f6810000-f6811fff : ahci
  f7200000-f7bfffff : PCI Bus 0000:02
  f7c00000-f7cfffff : PCI Bus 0000:01
    f7c00000-f7c01fff : 0000:01:00.0
      f7c00000-f7c01fff : iwlwifi
  f7d00000-f7d0ffff : 0000:00:14.0
    f7d00000-f7d0ffff : xhci_hcd
  f7d10000-f7d13fff : 0000:00:1b.0
    f7d10000-f7d13fff : ICH HD audio
  f7d14000-f7d17fff : 0000:00:03.0
  f7d19000-f7d190ff : 0000:00:1f.3
  f7d1a000-f7d1a3ff : 0000:00:1d.0
    f7d1a000-f7d1a3ff : ehci_hcd
  f7d1c000-f7d1c01f : 0000:00:16.0
    f7d1c000-f7d1c01f : mei_me
  f7fef000-f7feffff : pnp 00:0a
  f7ff0000-f7ffffff : pnp 00:0a
  f8000000-fbffffff : PCI MMCONFIG 0000 [bus 00-3f]
    f8000000-fbffffff : reserved
      f8000000-fbffffff : pnp 00:0a
fec00000-fec00fff : reserved
  fec00000-fec003ff : IOAPIC 0
fed00000-fed03fff : reserved
  fed00000-fed003ff : HPET 0
fed10000-fed17fff : pnp 00:0a
fed18000-fed18fff : pnp 00:0a
fed19000-fed19fff : pnp 00:0a
fed1c000-fed1ffff : reserved
  fed1c000-fed1ffff : pnp 00:0a
    fed1f410-fed1f414 : iTCO_wdt
      fed1f410-fed1f414 : iTCO_wdt
fed20000-fed3ffff : pnp 00:0a
fed45000-fed8ffff : pnp 00:0a
fed90000-fed93fff : pnp 00:0a
fee00000-fee00fff : Local APIC
  fee00000-fee00fff : reserved
ff000000-ffffffff : reserved
  ff000000-ffffffff : pnp 00:0a
100000000-21fdfffff : System RAM
21fe00000-21fffffff : RAM buffer

/proc/ioports:
0000-0cf7 : PCI Bus 0000:00
  0000-001f : dma1
  0020-0021 : pic1
  0040-0043 : timer0
  0050-0053 : timer1
  0060-0060 : keyboard
  0062-0062 : EC data
  0064-0064 : keyboard
  0066-0066 : EC cmd
  0070-0077 : rtc0
  0080-008f : dma page reg
  00a0-00a1 : pic2
  00c0-00df : dma2
  00f0-00ff : fpu
  04d0-04d1 : pnp 00:07
  0680-069f : pnp 00:04
0cf8-0cff : PCI conf1
0d00-ffff : PCI Bus 0000:00
  164e-164f : pnp 00:04
  1800-1803 : ACPI PM1a_EVT_BLK
  1804-1805 : ACPI PM1a_CNT_BLK
  1808-180b : ACPI PM_TMR
  1810-1815 : ACPI CPU throttle
  1830-1833 : iTCO_wdt
    1830-1833 : iTCO_wdt
  1850-1850 : ACPI PM2_CNT_BLK
  1854-1857 : pnp 00:06
  1860-187f : iTCO_wdt
    1860-187f : iTCO_wdt
  1880-189f : ACPI GPE0_BLK
  1c00-1cfe : pnp 00:04
  1d00-1dfe : pnp 00:04
  1e00-1efe : pnp 00:04
  1f00-1ffe : pnp 00:04
  2008-200b : pnp 00:04
  3000-3fff : PCI Bus 0000:01
  d000-dfff : PCI Bus 0000:03
  e000-efff : PCI Bus 0000:02
  f000-f03f : 0000:00:02.0
  f040-f05f : 0000:00:1f.3
  ffff-ffff : pnp 00:04
    ffff-ffff : pnp 00:04


Regards,
Thomas

(my apologies for forgetting the plain-text requirement resulting in
this not reaching the list immediately)

[1] https://bbs.archlinux.org/viewtopic.php?pid=1350921
[2] http://i.imgur.com/NCanPUY.jpg

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Ben Hutchings]

On Sat, 2014-01-11 at 08:56 +1300, Thomas Kear wrote:
> USB3 gigabit ethernet adapters with the ASIX AX88179 chipset (LevelOne
> USB0401-V3, Plugable USB3-E1000, SIIG JU-NE0211-S1 and others) are
> experiencing kernel panics in usb_hcd_map_urb_for_dma since 3.12.
[...]
> So far as I can tell, the driver is unaffected as late as 3.11.6, but
> problematic as of 3.12 (and still affected in 3.13-rc5).  The history
> of drivers/net/usb/ax88179_178a.c for this time period yields this
> patch, which at least in my somewhat limited understanding appeared a
> likely candidate.  I've reverted this on my system - against several
> linux-next builds from the last 3-4 weeks - and have had no issues
> with this network controller since.
> 
> commit 3804fad45411b48233b48003e33a78f290d227c8
> Author: Ming Lei <ming.lei-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
> Date:   Thu Aug 8 21:48:25 2013 +0800
> 
>     USBNET: ax88179_178a: enable tso if usb host supports sg dma

Enabling SG DMA here has unfortunately caused a number of regressions as
the XHCI (USB 3 controller) hardware is a bit inflexible and the driver
wasn't quite ready to handle the SG lists generated for net devices.

I don't think I've seen this particular symptom though, and it might
indicate a bug in usbnet.

[...]
> [2] http://i.imgur.com/NCanPUY.jpg

For those joining us on linux-usb, this is a photo of the panic.

Ben.

-- 
Ben Hutchings, Staff Engineer, Solarflare
Not speaking for my employer; that's the marketing department's job.
They asked us to note that Solarflare product names are trademarked.

--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Bjørn Mork]

Ben Hutchings <bhutchings-s/n/eUQHGBpZroRs9YW3xA@public.gmane.org> writes:
> On Sat, 2014-01-11 at 08:56 +1300, Thomas Kear wrote:
>> USB3 gigabit ethernet adapters with the ASIX AX88179 chipset (LevelOne
>> USB0401-V3, Plugable USB3-E1000, SIIG JU-NE0211-S1 and others) are
>> experiencing kernel panics in usb_hcd_map_urb_for_dma since 3.12.
> [...]
>> So far as I can tell, the driver is unaffected as late as 3.11.6, but
>> problematic as of 3.12 (and still affected in 3.13-rc5).  The history
>> of drivers/net/usb/ax88179_178a.c for this time period yields this
>> patch, which at least in my somewhat limited understanding appeared a
>> likely candidate.  I've reverted this on my system - against several
>> linux-next builds from the last 3-4 weeks - and have had no issues
>> with this network controller since.
>> 
>> commit 3804fad45411b48233b48003e33a78f290d227c8
>> Author: Ming Lei <ming.lei-Z7WLFzj8eWMS+FvcfC7Uqw@public.gmane.org>
>> Date:   Thu Aug 8 21:48:25 2013 +0800
>> 
>>     USBNET: ax88179_178a: enable tso if usb host supports sg dma
>
> Enabling SG DMA here has unfortunately caused a number of regressions as
> the XHCI (USB 3 controller) hardware is a bit inflexible and the driver
> wasn't quite ready to handle the SG lists generated for net devices.
>
> I don't think I've seen this particular symptom though, and it might
> indicate a bug in usbnet.

Yes, I believe this code in usbnet is still somewhat unmature and not
much tested.  Unfortunately I don't have any PC with xhci, so I can't do
much about that.

But looking at the code I think I found and obvious miss in the SG list
initialisation.  I'll post a proposed fix for that.  Would be good if
someone was able to test it.



Bjørn
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Thomas Kear]

On Sat, Jan 11, 2014 at 11:09 AM, Bjørn Mork <bjorn-yOkvZcmFvRU@public.gmane.org> wrote:
> But looking at the code I think I found and obvious miss in the SG list
> initialisation.  I'll post a proposed fix for that.  Would be good if
> someone was able to test it.

I've built 3.13.0-rc7-next-20140110 with your patch applied.
Unfortunately since this bug has taken anywhere from minutes to days
to manifest previously I'm not sure how quickly I'll be able to report
on its efficacy.

I currently have the adapter plugged in through a 4-port USB3 hub
(2109:0811, which appears to be a VIA chip) but I will test it
directly attached to the laptop too.

-Thomas
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Bjørn Mork]

Thomas Kear <thomas-hrxM1pTBpuF02iOg2JU2CQ@public.gmane.org> writes:

> On Sat, Jan 11, 2014 at 11:09 AM, Bjørn Mork <bjorn-yOkvZcmFvRU@public.gmane.org> wrote:
>> But looking at the code I think I found and obvious miss in the SG list
>> initialisation.  I'll post a proposed fix for that.  Would be good if
>> someone was able to test it.
>
> I've built 3.13.0-rc7-next-20140110 with your patch applied.
> Unfortunately since this bug has taken anywhere from minutes to days
> to manifest previously I'm not sure how quickly I'll be able to report
> on its efficacy.

Thanks for testing it. 

If I'm correct, then your problem is caused by usbnet incrementing
urb->num_sgs above the value sg_init_table was called with. This happens
if usbnet adds padding to a fragmented skb.  Unfortunately I have no
idea how you can create fragmented skbs with a certain length.  But I'm
sure others here know?

This bug in usbnet makes usb_hcd_map_urb_for_dma() call dma_map_sg()
with nents set past an entry with the SG chain termination bit set. This
bit makes the call to sg_next return NULL, even if there is another non
NULL entry in the list.  So when dma_map_sg does

        for_each_sg(sg, s, nents, i)
                kmemcheck_mark_initialized(sg_virt(s), s->length);

it ends up dereferencing NULL.

> I currently have the adapter plugged in through a 4-port USB3 hub
> (2109:0811, which appears to be a VIA chip) but I will test it
> directly attached to the laptop too.

I don't think that should matter.


Bjørn
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[David Laight]

From: Bjørn Mork
> Thomas Kear <thomas@kear.co.nz> writes:
> 
> > On Sat, Jan 11, 2014 at 11:09 AM, Bjrn Mork <bjorn@mork.no> wrote:
> >> But looking at the code I think I found and obvious miss in the SG list
> >> initialisation.  I'll post a proposed fix for that.  Would be good if
> >> someone was able to test it.
> >
> > I've built 3.13.0-rc7-next-20140110 with your patch applied.
> > Unfortunately since this bug has taken anywhere from minutes to days
> > to manifest previously I'm not sure how quickly I'll be able to report
> > on its efficacy.
> 
> Thanks for testing it.
> 
> If I'm correct, then your problem is caused by usbnet incrementing
> urb->num_sgs above the value sg_init_table was called with. This happens
> if usbnet adds padding to a fragmented skb.  Unfortunately I have no
> idea how you can create fragmented skbs with a certain length.  But I'm
> sure others here know?

I've managed to send fragmented skb, but not of specific lengths.
Maybe Nagle can be used on a TCP to get the required merging.
Send an initial fragment to 'prime' Nagle, then send a few fragments
that get sent together when the timer expires (or send data the other
way to force the send).

The patch you submitted is wrong.
Whoever wrote the sg interface was on crack.
The 'last' marker needs moving as well.

	David

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Bjørn Mork]

David Laight <David.Laight@ACULAB.COM> writes:

> The patch you submitted is wrong.
> Whoever wrote the sg interface was on crack.
> The 'last' marker needs moving as well.

I'm afraid I don't understand what you meant by this.

sg_init_table() set the 'last' marker.  AFAICS, you don't need to change
it unless you want to chain lists.

Care to explain with some code?


Bjørn

RE: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[David Laight]

From: Bjørn Mork
> David Laight <David.Laight@ACULAB.COM> writes:
> 
> > The patch you submitted is wrong.
> > Whoever wrote the sg interface was on crack.
> > The 'last' marker needs moving as well.
> 
> I'm afraid I don't understand what you meant by this.
> 
> sg_init_table() set the 'last' marker.  AFAICS, you don't need to change
> it unless you want to chain lists.
> 
> Care to explain with some code?

Just assuming that there will be some code, somewhere, that will try
to process the entire sg list - so won't like the entry with a
NULL pointer and zero length at the end.

If all the places that process the list are given an explicit
number of entries, or don't care about the NULL it doesn't matter.

	David

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Bjørn Mork]

David Laight <David.Laight-ZS65k/vG3HxXrIkS9f7CXA@public.gmane.org> writes:
> From: Bjørn Mork
>> David Laight <David.Laight-ZS65k/vG3HxXrIkS9f7CXA@public.gmane.org> writes:
>> 
>> > The patch you submitted is wrong.
>> > Whoever wrote the sg interface was on crack.
>> > The 'last' marker needs moving as well.
>> 
>> I'm afraid I don't understand what you meant by this.
>> 
>> sg_init_table() set the 'last' marker.  AFAICS, you don't need to change
>> it unless you want to chain lists.
>> 
>> Care to explain with some code?
>
> Just assuming that there will be some code, somewhere, that will try
> to process the entire sg list - so won't like the entry with a
> NULL pointer and zero length at the end.
>
> If all the places that process the list are given an explicit
> number of entries, or don't care about the NULL it doesn't matter.

I believe all processing use the urb->num_sgs field to limit the number
of entries.  Common interfaces like dma_map_sg() and for_each_sg() limit
their processing to "nents" entries, and the USB code use the value of
urb->num_sgs for this parameter.


Bjørn
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[David Laight]

From: Bjørn Mork [mailto:bjorn@mork.no]
> David Laight <David.Laight@ACULAB.COM> writes:
> > From: Bjørn Mork
> >> David Laight <David.Laight@ACULAB.COM> writes:
> >>
> >> > The patch you submitted is wrong.
> >> > Whoever wrote the sg interface was on crack.
> >> > The 'last' marker needs moving as well.
> >>
> >> I'm afraid I don't understand what you meant by this.
> >>
> >> sg_init_table() set the 'last' marker.  AFAICS, you don't need to change
> >> it unless you want to chain lists.
> >>
> >> Care to explain with some code?
> >
> > Just assuming that there will be some code, somewhere, that will try
> > to process the entire sg list - so won't like the entry with a
> > NULL pointer and zero length at the end.
> >
> > If all the places that process the list are given an explicit
> > number of entries, or don't care about the NULL it doesn't matter.
> 
> I believe all processing use the urb->num_sgs field to limit the number
> of entries.  Common interfaces like dma_map_sg() and for_each_sg() limit
> their processing to "nents" entries, and the USB code use the value of
> urb->num_sgs for this parameter.

Which mostly means that the sg_xxx functions are doing a whole load
of unnecessary instructions and memory accesses...

This probably has a lot to do with the significant difference in the
cpu use for the usb3 and 'normal' ethernet interfaces.

While each bit doesn't seem significant, they soon add up.

	David

Re: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[Ming Lei]

On Mon, Jan 13, 2014 at 9:26 PM, David Laight <David.Laight-JxhZ9S5GRejQT0dZR+AlfA@public.gmane.org> wrote:
>>
>> I believe all processing use the urb->num_sgs field to limit the number
>> of entries.  Common interfaces like dma_map_sg() and for_each_sg() limit
>> their processing to "nents" entries, and the USB code use the value of
>> urb->num_sgs for this parameter.
>
> Which mostly means that the sg_xxx functions are doing a whole load
> of unnecessary instructions and memory accesses...
>
> This probably has a lot to do with the significant difference in the
> cpu use for the usb3 and 'normal' ethernet interfaces.
>
> While each bit doesn't seem significant, they soon add up.

If you plan to remove the 'nents' parameter, I am wondering if it is
a good idea, because sg_nents() should be more heavy. Not mention
sometimes the callers just want to map/unmap part of entries.


Thanks,
-- 
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-usb" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[David Laight]

From: Ming Lei
> On Mon, Jan 13, 2014 at 9:26 PM, David Laight <David.Laight@aculab.com> wrote:
> >>
> >> I believe all processing use the urb->num_sgs field to limit the number
> >> of entries.  Common interfaces like dma_map_sg() and for_each_sg() limit
> >> their processing to "nents" entries, and the USB code use the value of
> >> urb->num_sgs for this parameter.
> >
> > Which mostly means that the sg_xxx functions are doing a whole load
> > of unnecessary instructions and memory accesses...
> >
> > This probably has a lot to do with the significant difference in the
> > cpu use for the usb3 and 'normal' ethernet interfaces.
> >
> > While each bit doesn't seem significant, they soon add up.
> 
> If you plan to remove the 'nents' parameter, I am wondering if it is
> a good idea, because sg_nents() should be more heavy. Not mention
> sometimes the callers just want to map/unmap part of entries.

I was thinking of using a simple address/length array without all
the extra fields and flags 'overpunched' in the low address bits.

I'm not even sure the current use is strictly correct.
The field names of the scatterlist have a page address and offset
but the buffers passed to xhci (at least by usbnet) can span multiple
pages - they are physically contiguous.

IIRC some traces of requests for USB disks show a separate SG entry
for each 4k page - even for both virtually and physically adjacent
pages.

	David

RE: PROBLEM: usbnet / ax88179_178a: Panic in usb_hcd_map_urb_for_dma

[David Laight]

From: Thomas Kear
> USB3 gigabit ethernet adapters with the ASIX AX88179 chipset (LevelOne
> USB0401-V3, Plugable USB3-E1000, SIIG JU-NE0211-S1 and others) are
> experiencing kernel panics in usb_hcd_map_urb_for_dma since 3.12...

I've worked out why I didn't hit that in my ax8179_18a testing.
I was running with a patch to xhci-ring.c that sends the zero length
packet, so the usbnet code wasn't adding the extra zero byte.

That patch is somewhere in Sarah's 'pending' queue.

	David