From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: dvb usb issues since kernel 4.9 From: Mauro Carvalho Chehab Message-Id: <20180109154235.2a42f0a0@vento.lan> Date: Tue, 9 Jan 2018 15:42:35 -0200 To: Linus Torvalds , Peter Zijlstra Cc: Alan Stern , Ingo Molnar , Josef Griebichler , Greg Kroah-Hartman , USB list , Eric Dumazet , Rik van Riel , Paolo Abeni , Hannes Frederic Sowa , Jesper Dangaard Brouer , linux-kernel , netdev , Jonathan Corbet , LMML , David Miller List-ID: RW0gTW9uLCA4IEphbiAyMDE4IDExOjUxOjA0IC0wODAwCkxpbnVzIFRvcnZhbGRzIDx0b3J2YWxk c0BsaW51eC1mb3VuZGF0aW9uLm9yZz4gZXNjcmV2ZXU6Cgo+IE9uIE1vbiwgSmFuIDgsIDIwMTgg YXQgMTE6MTUgQU0sIEFsYW4gU3Rlcm4gPHN0ZXJuQHJvd2xhbmQuaGFydmFyZC5lZHU+IHdyb3Rl Ogo+ID4KPiA+IEJvdGggZHdjMl9oc290ZyBhbmQgZWhjaS1oY2QgdXNlIHRoZSB0YXNrbGV0cyBl bWJlZGRlZCBpbiB0aGUKPiA+IGdpdmViYWNrX3VyYl9iaCBtZW1iZXIgb2Ygc3RydWN0IHVzYl9o Y2QuICBTZWUgdXNiX2hjZF9naXZlYmFja191cmIoKQo+ID4gaW4gZHJpdmVycy91c2IvY29yZS9o Y2QuYzsgdGhlIGNhbGxzIGFyZQo+ID4KPiA+ICAgICAgICAgZWxzZSBpZiAoaGlnaF9wcmlvX2Jo KQo+ID4gICAgICAgICAgICAgICAgIHRhc2tsZXRfaGlfc2NoZWR1bGUoJmJoLT5iaCk7Cj4gPiAg ICAgICAgIGVsc2UKPiA+ICAgICAgICAgICAgICAgICB0YXNrbGV0X3NjaGVkdWxlKCZiaC0+Ymgp Owo+ID4KPiA+IEFzIGl0IHR1cm5zIG91dCwgaGlnaF9wcmlvX2JoIGdldHMgc2V0IGZvciBpbnRl cnJ1cHQgYW5kIGlzb2Nocm9ub3VzCj4gPiBVUkJzIGJ1dCBub3QgZm9yIGJ1bGsgYW5kIGNvbnRy b2wgVVJCcy4gIFRoZSBEVkIgZHJpdmVyIGluIHF1ZXN0aW9uCj4gPiB1c2VzIGJ1bGsgdHJhbnNm ZXJzLiAgCj4gCj4gT2ssIHNvIHdlIGNvdWxkIHRyeSBvdXQgc29tZXRoaW5nIGxpa2UgdGhlIGFw cGVuZGVkPwo+IAo+IE5PVEUhIEkgaGF2ZSBub3QgdGVzdGVkIHRoaXMgYXQgYWxsLiBJdCBMb29r c09idmlvdXModG0pLCBidXQuLi4KPiAKPiAgICAgICAgICAgICAgICAgICAgIExpbnVzCgoKCj4g IGtlcm5lbC9zb2Z0aXJxLmMgfCAxMiArKysrKysrKy0tLS0KPiAgMSBmaWxlIGNoYW5nZWQsIDgg aW5zZXJ0aW9ucygrKSwgNCBkZWxldGlvbnMoLSkKPiAKPiBkaWZmIC0tZ2l0IGEva2VybmVsL3Nv ZnRpcnEuYyBiL2tlcm5lbC9zb2Z0aXJxLmMKPiBpbmRleCAyZjVlODdmMWJhZTIuLjk3YjA4MDk1 NmZlYSAxMDA2NDQKPiAtLS0gYS9rZXJuZWwvc29mdGlycS5jCj4gKysrIGIva2VybmVsL3NvZnRp cnEuYwo+IEBAIC03OSwxMiArNzksMTYgQEAgc3RhdGljIHZvaWQgd2FrZXVwX3NvZnRpcnFkKHZv aWQpCj4gIAo+ICAvKgo+ICAgKiBJZiBrc29mdGlycWQgaXMgc2NoZWR1bGVkLCB3ZSBkbyBub3Qg d2FudCB0byBwcm9jZXNzIHBlbmRpbmcgc29mdGlycXMKPiAtICogcmlnaHQgbm93LiBMZXQga3Nv ZnRpcnFkIGhhbmRsZSB0aGlzIGF0IGl0cyBvd24gcmF0ZSwgdG8gZ2V0IGZhaXJuZXNzLgo+ICsg KiByaWdodCBub3cuIExldCBrc29mdGlycWQgaGFuZGxlIHRoaXMgYXQgaXRzIG93biByYXRlLCB0 byBnZXQgZmFpcm5lc3MsCj4gKyAqIHVubGVzcyB3ZSdyZSBkb2luZyBzb21lIG9mIHRoZSBzeW5j aHJvbm91cyBzb2Z0aXJxcy4KPiAgICovCj4gLXN0YXRpYyBib29sIGtzb2Z0aXJxZF9ydW5uaW5n KHZvaWQpCj4gKyNkZWZpbmUgU09GVElSUV9OT1dfTUFTSyAoKDEgPDwgSElfU09GVElSUSkgfCAo MSA8PCBUQVNLTEVUX1NPRlRJUlEpKQo+ICtzdGF0aWMgYm9vbCBrc29mdGlycWRfcnVubmluZyh1 bnNpZ25lZCBsb25nIHBlbmRpbmcpCj4gIHsKPiAgCXN0cnVjdCB0YXNrX3N0cnVjdCAqdHNrID0g X190aGlzX2NwdV9yZWFkKGtzb2Z0aXJxZCk7Cj4gIAo+ICsJaWYgKHBlbmRpbmcgJiBTT0ZUSVJR X05PV19NQVNLKQo+ICsJCXJldHVybiBmYWxzZTsKPiAgCXJldHVybiB0c2sgJiYgKHRzay0+c3Rh dGUgPT0gVEFTS19SVU5OSU5HKTsKPiAgfQo+ICAKPiBAQCAtMzI1LDcgKzMyOSw3IEBAIGFzbWxp bmthZ2UgX192aXNpYmxlIHZvaWQgZG9fc29mdGlycSh2b2lkKQo+ICAKPiAgCXBlbmRpbmcgPSBs b2NhbF9zb2Z0aXJxX3BlbmRpbmcoKTsKPiAgCj4gLQlpZiAocGVuZGluZyAmJiAha3NvZnRpcnFk X3J1bm5pbmcoKSkKPiArCWlmIChwZW5kaW5nICYmICFrc29mdGlycWRfcnVubmluZyhwZW5kaW5n KSkKPiAgCQlkb19zb2Z0aXJxX293bl9zdGFjaygpOwo+ICAKPiAgCWxvY2FsX2lycV9yZXN0b3Jl KGZsYWdzKTsKPiBAQCAtMzUyLDcgKzM1Niw3IEBAIHZvaWQgaXJxX2VudGVyKHZvaWQpCj4gIAo+ ICBzdGF0aWMgaW5saW5lIHZvaWQgaW52b2tlX3NvZnRpcnEodm9pZCkKPiAgewo+IC0JaWYgKGtz b2Z0aXJxZF9ydW5uaW5nKCkpCj4gKwlpZiAoa3NvZnRpcnFkX3J1bm5pbmcobG9jYWxfc29mdGly cV9wZW5kaW5nKCkpKQo+ICAJCXJldHVybjsKPiAgCj4gIAlpZiAoIWZvcmNlX2lycXRocmVhZHMp IHsKCgpIaSBMaW51cywKClBhdGNoIG1ha2VzIHNlbnNlIHRvIG1lLCBhbHRob3VnaCBJIHdhcyBu b3QgYWJsZSB0byB0ZXN0IGl0IG15c2VsZi4KCkkgc2V0IGEgUlBpMyBtYWNoaW5lIGhlcmUgd2l0 aCB2YW5pbGxhIEtlcm5lbCA0LjE0LjExIHJ1bm5pbmcgYSBzdGFuZGFyZApyYXNwYmlhbiBkaXN0 cmlidXRpb24gKHdpdGggZWxldmF0b3I9ZGVhZGxpbmUpLiBSaWdodCBub3csIEknbSB0cnlpbmcg dG8KcmVwcm9kdWNlIHRoZSBidWcgd2l0aCBkdmJ2NS16YXAuIEkgbWF5IGV2ZW50dWFsbHkgZG8g bW9yZSB0ZXN0cyBvbiAKc29tZSBvdGhlciBzbG93IG1hY2hpbmVzLgoKVXN1YWxseSwgYXBwbGlj YXRpb25zIGxpa2UgdHZoZWFkZW5kIHJlY29yZHMganVzdCBvbmUgY2hhbm5lbC4gU28sIGluc3Rl YWQKb2YgYSB+NTggTWJpdHMvcyBwYXlsb2FkLCBpdCB1c2VzLCB0eXBpY2FsbHksIH4xMSBNYml0 cy9zIGZvciBhIEhEIGNoYW5uZWwuClRoaXMgaXMgdXN1YWxseSBmaWx0ZXJlZCBieSBoYXJkd2Fy ZS4gSGVyZSwgSSdtIGZvcmNpbmcgdG8gcmVjb3JkIHRoZQplbnRpcmUgVFMsIGluIG9yZGVyIHRv IG1ha2UgZWFzaWVyIHRvIHJlcHJvZHVjZSB0aGUgaXNzdWUuIFNvLCBJJ20gZm9yY2luZwphIGNv bmRpdGlvbiB0aGF0IGl0IGlzIHVzdWFsbHkgd29yc2UgdGhhbiByZWFsIHVzZWNhc2VzIChhdCBs YXN0IGZvciBIRCAtIEkKSSBkb24ndCBoYXZlIGFueSBEVkIgc3RyZWFtIGhlcmUgd2l0aCBhIDRL IGNoYW5uZWwpLgoKRnJvbSB3aGF0IEkgY2hlY2tlZCBzbyBmYXIsIHdpdGggdmFuaWxhIHVwc3Ry ZWFtIEtlcm5lbCBvbiBSUGkzLCBqdXN0CnJlY2VpdmluZyBhIERWQiBzdHJlYW0gLSBvciByZWNl aXZpbmcgaXQgYW5kIHdyaXRpbmcgdG8gL2Rldi9udWxsIHdvcmtzCndpdGggb3Igd2l0aG91dCB5 b3VyIHBhdGNoLgoKVGhlIHByb2JsZW0gc3RhcnRzIHRvIGhhcHBlbiB3aGVuIHRoZXJlIGFyZSBj b25jdXJyZW5jeSB3aXRoIHdyaXRlcy4KCk9uIG15IHByZWxpbWluYXIgdGVzdHMsIHdyaXRpbmcg dG8gYSBmaWxlIG9uIGFuIGV4dDQgcGFydGl0aW9uIGF0IGEKVVNCIHN0aWNrIGxvc2VzIGRhdGEg dXAgdG8gdGhlIHBvaW50IHRvIG1ha2UgaXQgdXNlbGVzcyAoMS80IG9mIHRoZSBkYXRhCmlzIGxv c3QhKS4gSG93ZXZlciwgd3JpdGluZyB0byBhIGNsYXNzIDEwIG1pY3JvU0QgY2FyZCBpcyBkb2Fi bGUuCgpJZiB5b3UncmUgY3VyaW91cyBlbm91Z2gsIHRoaXMgaXMgd2hhdCBJJ20gZG9pbmcgKHRo YXQgYXJlIHRoZSByZXN1bHRzCndoaWxlIHVzaW5nIGNsYXNzIDEwIG1pY3JvU0QgY2FyZCk6Cgok IEZJTEU9L3RtcC9vdXQudHM7IGZvciBpIGluICQoc2VxIDEgNik7IGRvIGVjaG8gInN0ZXAgJGki OyBybSAkRklMRSAyPi9kZXYvbnVsbDsgZHZidjUtemFwIC1sIHVuaXZlcnNhbCAtYyB+L3Zpdm8t Y2hhbm5lbHMuY29uZiBOQlIgLW8gJEZJTEUgLVAgLXQ2MCAyPiYxfGdyZXAgLUUgIihidWZmZXJ8 cmVjZWl2ZWQpIjsgZHUgJEZJTEUgMj4vZGV2L251bGw7IGRvbmUgCnN0ZXAgMQogIFNldHRpbmcg YnVmZmVyIGxlbmd0aCB0byA3MjUwMDAwCmJ1ZmZlciBvdmVycnVuCmJ1ZmZlciBvdmVycnVuCmJ1 ZmZlciBvdmVycnVuCmJ1ZmZlciBvdmVycnVuCmJ1ZmZlciBvdmVycnVuCmJ1ZmZlciBvdmVycnVu CmJ1ZmZlciBvdmVycnVuCnJlY2VpdmVkIDM0NzUwNDY1MiBieXRlcyAoNTY1NiBLYnl0ZXMvc2Vj KQozMzkzNjgJL3RtcC9vdXQudHMKc3RlcCAyCiAgU2V0dGluZyBidWZmZXIgbGVuZ3RoIHRvIDcy NTAwMDAKYnVmZmVyIG92ZXJydW4KcmVjZWl2ZWQgNDA4OTk1ODgwIGJ5dGVzICg2NjU2IEtieXRl cy9zZWMpCjM5OTQxNgkvdG1wL291dC50cwpzdGVwIDMKICBTZXR0aW5nIGJ1ZmZlciBsZW5ndGgg dG8gNzI1MDAwMApyZWNlaXZlZCA0MTI5OTk3MTYgYnl0ZXMgKDY3MjIgS2J5dGVzL3NlYykKNDAz MzI4CS90bXAvb3V0LnRzCnN0ZXAgNAogIFNldHRpbmcgYnVmZmVyIGxlbmd0aCB0byA3MjUwMDAw CmJ1ZmZlciBvdmVycnVuCnJlY2VpdmVkIDQxNTU2NDc4OCBieXRlcyAoNjc2MyBLYnl0ZXMvc2Vj KQo0MDU4MzIJL3RtcC9vdXQudHMKc3RlcCA1CiAgU2V0dGluZyBidWZmZXIgbGVuZ3RoIHRvIDcy NTAwMDAKcmVjZWl2ZWQgNDEyOTk5NzE2IGJ5dGVzICg2NzIyIEtieXRlcy9zZWMpCjQwMzMyNAkv dG1wL291dC50cwpzdGVwIDYKICBTZXR0aW5nIGJ1ZmZlciBsZW5ndGggdG8gNzI1MDAwMApyZWNl aXZlZCA0MDgzNjYwODAgYnl0ZXMgKDY2NDYgS2J5dGVzL3NlYykKMzk4Nzk2CS90bXAvb3V0LnRz CgpNeSBwbGFuIGlzIHRvIGRvIG1vcmUgdGVzdHMgYWxvbmcgdGhpcyB3ZWVrLCBhbmQgdHJ5IHRv IHR3ZWFrIGEgbGl0dGxlCmJpdCBib3RoIHVzZXJzcGFjZSBhbmQga2VybmVsc3BhY2UsIGluIG9y ZGVyIHRvIHNlZSBpZiBJIGNhbiBnZXQgYmV0dGVyCnJlc3VsdHMuCgpUaGFua3MsCk1hdXJvCi0t LQpUbyB1bnN1YnNjcmliZSBmcm9tIHRoaXMgbGlzdDogc2VuZCB0aGUgbGluZSAidW5zdWJzY3Jp YmUgbGludXgtdXNiIiBpbgp0aGUgYm9keSBvZiBhIG1lc3NhZ2UgdG8gbWFqb3Jkb21vQHZnZXIu a2VybmVsLm9yZwpNb3JlIG1ham9yZG9tbyBpbmZvIGF0ICBodHRwOi8vdmdlci5rZXJuZWwub3Jn L21ham9yZG9tby1pbmZvLmh0bWwK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from osg.samsung.com ([64.30.133.232]:58869 "EHLO osg.samsung.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754885AbeAIRmo (ORCPT ); Tue, 9 Jan 2018 12:42:44 -0500 Date: Tue, 9 Jan 2018 15:42:35 -0200 From: Mauro Carvalho Chehab To: Linus Torvalds , Peter Zijlstra Cc: Alan Stern , Ingo Molnar , Josef Griebichler , Greg Kroah-Hartman , USB list , Eric Dumazet , Rik van Riel , Paolo Abeni , Hannes Frederic Sowa , Jesper Dangaard Brouer , linux-kernel , netdev , Jonathan Corbet , LMML , David Miller Subject: Re: dvb usb issues since kernel 4.9 Message-ID: <20180109154235.2a42f0a0@vento.lan> In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 8BIT Sender: linux-media-owner@vger.kernel.org List-ID: Em Mon, 8 Jan 2018 11:51:04 -0800 Linus Torvalds escreveu: > On Mon, Jan 8, 2018 at 11:15 AM, Alan Stern wrote: > > > > Both dwc2_hsotg and ehci-hcd use the tasklets embedded in the > > giveback_urb_bh member of struct usb_hcd. See usb_hcd_giveback_urb() > > in drivers/usb/core/hcd.c; the calls are > > > > else if (high_prio_bh) > > tasklet_hi_schedule(&bh->bh); > > else > > tasklet_schedule(&bh->bh); > > > > As it turns out, high_prio_bh gets set for interrupt and isochronous > > URBs but not for bulk and control URBs. The DVB driver in question > > uses bulk transfers. > > Ok, so we could try out something like the appended? > > NOTE! I have not tested this at all. It LooksObvious(tm), but... > > Linus > kernel/softirq.c | 12 ++++++++---- > 1 file changed, 8 insertions(+), 4 deletions(-) > > diff --git a/kernel/softirq.c b/kernel/softirq.c > index 2f5e87f1bae2..97b080956fea 100644 > --- a/kernel/softirq.c > +++ b/kernel/softirq.c > @@ -79,12 +79,16 @@ static void wakeup_softirqd(void) > > /* > * If ksoftirqd is scheduled, we do not want to process pending softirqs > - * right now. Let ksoftirqd handle this at its own rate, to get fairness. > + * right now. Let ksoftirqd handle this at its own rate, to get fairness, > + * unless we're doing some of the synchronous softirqs. > */ > -static bool ksoftirqd_running(void) > +#define SOFTIRQ_NOW_MASK ((1 << HI_SOFTIRQ) | (1 << TASKLET_SOFTIRQ)) > +static bool ksoftirqd_running(unsigned long pending) > { > struct task_struct *tsk = __this_cpu_read(ksoftirqd); > > + if (pending & SOFTIRQ_NOW_MASK) > + return false; > return tsk && (tsk->state == TASK_RUNNING); > } > > @@ -325,7 +329,7 @@ asmlinkage __visible void do_softirq(void) > > pending = local_softirq_pending(); > > - if (pending && !ksoftirqd_running()) > + if (pending && !ksoftirqd_running(pending)) > do_softirq_own_stack(); > > local_irq_restore(flags); > @@ -352,7 +356,7 @@ void irq_enter(void) > > static inline void invoke_softirq(void) > { > - if (ksoftirqd_running()) > + if (ksoftirqd_running(local_softirq_pending())) > return; > > if (!force_irqthreads) { Hi Linus, Patch makes sense to me, although I was not able to test it myself. I set a RPi3 machine here with vanilla Kernel 4.14.11 running a standard raspbian distribution (with elevator=deadline). Right now, I'm trying to reproduce the bug with dvbv5-zap. I may eventually do more tests on some other slow machines. Usually, applications like tvheadend records just one channel. So, instead of a ~58 Mbits/s payload, it uses, typically, ~11 Mbits/s for a HD channel. This is usually filtered by hardware. Here, I'm forcing to record the entire TS, in order to make easier to reproduce the issue. So, I'm forcing a condition that it is usually worse than real usecases (at last for HD - I I don't have any DVB stream here with a 4K channel). >>From what I checked so far, with vanila upstream Kernel on RPi3, just receiving a DVB stream - or receiving it and writing to /dev/null works with or without your patch. The problem starts to happen when there are concurrency with writes. On my preliminar tests, writing to a file on an ext4 partition at a USB stick loses data up to the point to make it useless (1/4 of the data is lost!). However, writing to a class 10 microSD card is doable. If you're curious enough, this is what I'm doing (that are the results while using class 10 microSD card): $ FILE=/tmp/out.ts; for i in $(seq 1 6); do echo "step $i"; rm $FILE 2>/dev/null; dvbv5-zap -l universal -c ~/vivo-channels.conf NBR -o $FILE -P -t60 2>&1|grep -E "(buffer|received)"; du $FILE 2>/dev/null; done step 1 Setting buffer length to 7250000 buffer overrun buffer overrun buffer overrun buffer overrun buffer overrun buffer overrun buffer overrun received 347504652 bytes (5656 Kbytes/sec) 339368 /tmp/out.ts step 2 Setting buffer length to 7250000 buffer overrun received 408995880 bytes (6656 Kbytes/sec) 399416 /tmp/out.ts step 3 Setting buffer length to 7250000 received 412999716 bytes (6722 Kbytes/sec) 403328 /tmp/out.ts step 4 Setting buffer length to 7250000 buffer overrun received 415564788 bytes (6763 Kbytes/sec) 405832 /tmp/out.ts step 5 Setting buffer length to 7250000 received 412999716 bytes (6722 Kbytes/sec) 403324 /tmp/out.ts step 6 Setting buffer length to 7250000 received 408366080 bytes (6646 Kbytes/sec) 398796 /tmp/out.ts My plan is to do more tests along this week, and try to tweak a little bit both userspace and kernelspace, in order to see if I can get better results. Thanks, Mauro