From mboxrd@z Thu Jan 1 00:00:00 1970 Return-path: Received: from us-smtp-2.mimecast.com ([205.139.110.61] helo=us-smtp-delivery-1.mimecast.com) by bombadil.infradead.org with esmtps (Exim 4.92.2 #3 (Red Hat Linux)) id 1iBF0n-0006Gv-SC for ath10k@lists.infradead.org; Fri, 20 Sep 2019 09:15:36 +0000 Received: by mail-ed1-f71.google.com with SMTP id c23so3791234edb.14 for ; Fri, 20 Sep 2019 02:15:28 -0700 (PDT) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= Subject: Re: [PATCH 2/4] mac80211: defer txqs removal from rbtree In-Reply-To: <1b4ab006d9b5c88035845aaac193ef48@codeaurora.org> References: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> <1568639388-27291-2-git-send-email-yiboz@codeaurora.org> <87pnjyiq7o.fsf@toke.dk> <87sgothmpy.fsf@toke.dk> <8cdece5c030fd95817fb099021c38613@codeaurora.org> <87tv98fu6l.fsf@toke.dk> <1b4ab006d9b5c88035845aaac193ef48@codeaurora.org> Date: Fri, 20 Sep 2019 11:15:25 +0200 Message-ID: <8736gre3bm.fsf@toke.dk> MIME-Version: 1.0 List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: base64 Sender: "ath10k" Errors-To: ath10k-bounces+kvalo=adurom.com@lists.infradead.org To: Yibo Zhao Cc: linux-wireless-owner@vger.kernel.org, linux-wireless@vger.kernel.org, ath10k@lists.infradead.org WWlibyBaaGFvIDx5aWJvekBjb2RlYXVyb3JhLm9yZz4gd3JpdGVzOgoKPiBPbiAyMDE5LTA5LTE5 IDE4OjM3LCBUb2tlIEjDuGlsYW5kLUrDuHJnZW5zZW4gd3JvdGU6Cj4+IFlpYm8gWmhhbyA8eWli b3pAY29kZWF1cm9yYS5vcmc+IHdyaXRlczoKPj4gCj4+PiBPbiAyMDE5LTA5LTE4IDE5OjIzLCBU b2tlIEjDuGlsYW5kLUrDuHJnZW5zZW4gd3JvdGU6Cj4+Pj4gWWlibyBaaGFvIDx5aWJvekBjb2Rl YXVyb3JhLm9yZz4gd3JpdGVzOgo+Pj4+IAo+Pj4+PiBPbiAyMDE5LTA5LTE4IDA1OjEwLCBUb2tl IEjDuGlsYW5kLUrDuHJnZW5zZW4gd3JvdGU6Cj4+Pj4+PiBZaWJvIFpoYW8gPHlpYm96QGNvZGVh dXJvcmEub3JnPiB3cml0ZXM6Cj4+Pj4+PiAKPj4+Pj4+PiBJbiBhIGxvb3AgdHhxcyBkZXF1ZXVl IHNjZW5hcmlvLCBpZiB0aGUgZmlyc3QgdHhxIGluIHRoZSByYnRyZWUgCj4+Pj4+Pj4gZ2V0cwo+ Pj4+Pj4+IHJlbW92ZWQgZnJvbSByYnRyZWUgaW1tZWRpYXRlbHkgaW4gdGhlIGllZWU4MDIxMV9y ZXR1cm5fdHhxKCksIHRoZQo+Pj4+Pj4+IGxvb3Agd2lsbCBicmVhayBzb29uIGluIHRoZSBpZWVl ODAyMTFfbmV4dF90eHEoKSBkdWUgdG8gCj4+Pj4+Pj4gc2NoZWR1bGVfcG9zCj4+Pj4+Pj4gbm90 IGxlYWRpbmcgdG8gdGhlIHNlY29uZCB0eHEgaW4gdGhlIHJidHJlZS4gVGh1cywgZGVmZXJpbmcg dGhlCj4+Pj4+Pj4gcmVtb3ZhbCByaWdodCBiZWZvcmUgdGhlIGVuZCBvZiB0aGlzIHNjaGVkdWxl IHJvdW5kLgo+Pj4+Pj4+IAo+Pj4+Pj4+IENvLWRldmVsb3BlZC1ieTogWWlibyBaaGFvIDx5aWJv ekBjb2RlYXVyb3JhLm9yZz4KPj4+Pj4+PiBTaWduZWQtb2ZmLWJ5OiBZaWJvIFpoYW8gPHlpYm96 QGNvZGVhdXJvcmEub3JnPgo+Pj4+Pj4+IFNpZ25lZC1vZmYtYnk6IFRva2UgSMO4aWxhbmQtSsO4 cmdlbnNlbiA8dG9rZUB0b2tlLmRrPgo+Pj4+Pj4gCj4+Pj4+PiBJIGRpZG4ndCB3cml0ZSB0aGlz IHBhdGNoLCBzbyBwbGVhc2UgZG9uJ3QgdXNlIG15IHNpZ24tb2ZmLiBJJ2xsIAo+Pj4+Pj4gYWRk Cj4+Pj4+PiBhY2sgb3IgcmV2aWV3IHRhZ3MgYXMgYXBwcm9wcmlhdGUgaW4gcmVwbHk7IGJ1dCBh IGZldyBjb21tZW50cyAKPj4+Pj4+IGZpcnN0Ogo+Pj4+Pj4gCj4+Pj4+Pj4gLS0tCj4+Pj4+Pj4g IGluY2x1ZGUvbmV0L21hYzgwMjExLmggICAgIHwgMTYgKysrKysrKysrKy0tCj4+Pj4+Pj4gIG5l dC9tYWM4MDIxMS9pZWVlODAyMTFfaS5oIHwgIDMgKysrCj4+Pj4+Pj4gIG5ldC9tYWM4MDIxMS9t YWluLmMgICAgICAgIHwgIDYgKysrKysKPj4+Pj4+PiAgbmV0L21hYzgwMjExL3R4LmMgICAgICAg ICAgfCA2Mwo+Pj4+Pj4+ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysrKysr KystLS0KPj4+Pj4+PiAgNCBmaWxlcyBjaGFuZ2VkLCA4MyBpbnNlcnRpb25zKCspLCA1IGRlbGV0 aW9ucygtKQo+Pj4+Pj4+IAo+Pj4+Pj4+IGRpZmYgLS1naXQgYS9pbmNsdWRlL25ldC9tYWM4MDIx MS5oIGIvaW5jbHVkZS9uZXQvbWFjODAyMTEuaAo+Pj4+Pj4+IGluZGV4IGFjMmVkOGUuLmJhNWEz NDUgMTAwNjQ0Cj4+Pj4+Pj4gLS0tIGEvaW5jbHVkZS9uZXQvbWFjODAyMTEuaAo+Pj4+Pj4+ICsr KyBiL2luY2x1ZGUvbmV0L21hYzgwMjExLmgKPj4+Pj4+PiBAQCAtOTI1LDYgKzkyNSw4IEBAIHN0 cnVjdCBpZWVlODAyMTFfdHhfcmF0ZSB7Cj4+Pj4+Pj4gCj4+Pj4+Pj4gICNkZWZpbmUgSUVFRTgw MjExX01BWF9UWF9SRVRSWQkJMzEKPj4+Pj4+PiAKPj4+Pj4+PiArI2RlZmluZSBJRUVFODAyMTFf QUlSVElNRV9UWFFfUk1fQ0hLX0lOVFZfSU5fTVMgMTAwCj4+Pj4+Pj4gKwo+Pj4+Pj4+ICBzdGF0 aWMgaW5saW5lIHZvaWQgaWVlZTgwMjExX3JhdGVfc2V0X3ZodChzdHJ1Y3QgCj4+Pj4+Pj4gaWVl ZTgwMjExX3R4X3JhdGUKPj4+Pj4+PiAqcmF0ZSwKPj4+Pj4+PiAgCQkJCQkgIHU4IG1jcywgdTgg bnNzKQo+Pj4+Pj4+ICB7Cj4+Pj4+Pj4gQEAgLTYyMzIsNyArNjIzNCw4IEBAIHN0cnVjdCBza19i dWZmICppZWVlODAyMTFfdHhfZGVxdWV1ZShzdHJ1Y3QKPj4+Pj4+PiBpZWVlODAyMTFfaHcgKmh3 LAo+Pj4+Pj4+ICAgKiBAYWM6IEFDIG51bWJlciB0byByZXR1cm4gcGFja2V0cyBmcm9tLgo+Pj4+ Pj4+ICAgKgo+Pj4+Pj4+ICAgKiBTaG91bGQgb25seSBiZSBjYWxsZWQgYmV0d2VlbiBjYWxscyB0 bwo+Pj4+Pj4+IGllZWU4MDIxMV90eHFfc2NoZWR1bGVfc3RhcnQoKQo+Pj4+Pj4+IC0gKiBhbmQg aWVlZTgwMjExX3R4cV9zY2hlZHVsZV9lbmQoKS4KPj4+Pj4+PiArICogYW5kIGllZWU4MDIxMV90 eHFfc2NoZWR1bGVfZW5kKCkuIElmIHRoZSB0eHEgaXMgZW1wdHksIGl0IHdpbGwKPj4+Pj4+PiBi ZQo+Pj4+Pj4+IGFkZGVkCj4+Pj4+Pj4gKyAqIHRvIGEgcmVtb3ZlIGxpc3QgYW5kIGdldCByZW1v dmVkIGxhdGVyLgo+Pj4+Pj4+ICAgKiBSZXR1cm5zIHRoZSBuZXh0IHR4cSBpZiBzdWNjZXNzZnVs LCAlTlVMTCBpZiBubyBxdWV1ZSBpcwo+Pj4+Pj4+IGVsaWdpYmxlLgo+Pj4+Pj4+IElmIGEgdHhx Cj4+Pj4+Pj4gICAqIGlzIHJldHVybmVkLCBpdCBzaG91bGQgYmUgcmV0dXJuZWQgd2l0aCBpZWVl ODAyMTFfcmV0dXJuX3R4cSgpCj4+Pj4+Pj4gYWZ0ZXIgdGhlCj4+Pj4+Pj4gICAqIGRyaXZlciBo YXMgZmluaXNoZWQgc2NoZWR1bGluZyBpdC4KPj4+Pj4+PiBAQCAtNjI2OCw3ICs2MjcxLDggQEAg dm9pZCBpZWVlODAyMTFfdHhxX3NjaGVkdWxlX3N0YXJ0KHN0cnVjdAo+Pj4+Pj4+IGllZWU4MDIx MV9odyAqaHcsIHU4IGFjKQo+Pj4+Pj4+ICAgKiBAaHc6IHBvaW50ZXIgYXMgb2J0YWluZWQgZnJv bSBpZWVlODAyMTFfYWxsb2NfaHcoKQo+Pj4+Pj4+ICAgKiBAYWM6IEFDIG51bWJlciB0byBhY3F1 aXJlIGxvY2tzIGZvcgo+Pj4+Pj4+ICAgKgo+Pj4+Pj4+IC0gKiBSZWxlYXNlIGxvY2tzIHByZXZp b3VzbHkgYWNxdWlyZWQgYnkKPj4+Pj4+PiBpZWVlODAyMTFfdHhxX3NjaGVkdWxlX2VuZCgpLgo+ Pj4+Pj4+ICsgKiBSZWxlYXNlIGxvY2tzIHByZXZpb3VzbHkgYWNxdWlyZWQgYnkKPj4+Pj4+PiBp ZWVlODAyMTFfdHhxX3NjaGVkdWxlX2VuZCgpLgo+Pj4+Pj4+IENoZWNrCj4+Pj4+Pj4gKyAqIGFu ZCByZW1vdmUgdGhlIGVtcHR5IHR4cSBmcm9tIHJiLXRyZWUuCj4+Pj4+Pj4gICAqLwo+Pj4+Pj4+ ICB2b2lkIGllZWU4MDIxMV90eHFfc2NoZWR1bGVfZW5kKHN0cnVjdCBpZWVlODAyMTFfaHcgKmh3 LCB1OCBhYykKPj4+Pj4+PiAgCV9fcmVsZWFzZXModHhxX2xvY2spOwo+Pj4+Pj4+IEBAIC02Mjg3 LDYgKzYyOTEsMTQgQEAgdm9pZCBpZWVlODAyMTFfc2NoZWR1bGVfdHhxKHN0cnVjdAo+Pj4+Pj4+ IGllZWU4MDIxMV9odwo+Pj4+Pj4+ICpodywgc3RydWN0IGllZWU4MDIxMV90eHEgKnR4cSkKPj4+ Pj4+PiAgCV9fYWNxdWlyZXModHhxX2xvY2spIF9fcmVsZWFzZXModHhxX2xvY2spOwo+Pj4+Pj4+ IAo+Pj4+Pj4+ICAvKioKPj4+Pj4+PiArICogaWVlZTgwMjExX3R4cXNfY2hlY2sgLSBDaGVjayB0 eHFzIHdhaXRpbmcgZm9yIHJlbW92YWwKPj4+Pj4+PiArICoKPj4+Pj4+PiArICogQHRtcjogcG9p bnRlciBhcyBvYnRhaW5lZCBmcm9tIGxvY2FsCj4+Pj4+Pj4gKyAqCj4+Pj4+Pj4gKyAqLwo+Pj4+ Pj4+ICt2b2lkIGllZWU4MDIxMV90eHFzX2NoZWNrKHN0cnVjdCB0aW1lcl9saXN0ICp0bXIpOwo+ Pj4+Pj4+ICsKPj4+Pj4+PiArLyoqCj4+Pj4+Pj4gICAqIGllZWU4MDIxMV90eHFfbWF5X3RyYW5z bWl0IC0gY2hlY2sgd2hldGhlciBUWFEgaXMgYWxsb3dlZCB0bwo+Pj4+Pj4+IHRyYW5zbWl0Cj4+ Pj4+Pj4gICAqCj4+Pj4+Pj4gICAqIFRoaXMgZnVuY3Rpb24gaXMgdXNlZCB0byBjaGVjayB3aGV0 aGVyIGdpdmVuIHR4cSBpcyBhbGxvd2VkIHRvCj4+Pj4+Pj4gdHJhbnNtaXQgYnkKPj4+Pj4+PiBk aWZmIC0tZ2l0IGEvbmV0L21hYzgwMjExL2llZWU4MDIxMV9pLmggCj4+Pj4+Pj4gYi9uZXQvbWFj ODAyMTEvaWVlZTgwMjExX2kuaAo+Pj4+Pj4+IGluZGV4IGE0NTU2ZjkuLjQ5YWExNDNlIDEwMDY0 NAo+Pj4+Pj4+IC0tLSBhL25ldC9tYWM4MDIxMS9pZWVlODAyMTFfaS5oCj4+Pj4+Pj4gKysrIGIv bmV0L21hYzgwMjExL2llZWU4MDIxMV9pLmgKPj4+Pj4+PiBAQCAtODQ3LDYgKzg0Nyw3IEBAIHN0 cnVjdCB0eHFfaW5mbyB7Cj4+Pj4+Pj4gIAlzdHJ1Y3QgY29kZWxfc3RhdHMgY3N0YXRzOwo+Pj4+ Pj4+ICAJc3RydWN0IHNrX2J1ZmZfaGVhZCBmcmFnczsKPj4+Pj4+PiAgCXN0cnVjdCByYl9ub2Rl IHNjaGVkdWxlX29yZGVyOwo+Pj4+Pj4+ICsJc3RydWN0IGxpc3RfaGVhZCBjYW5kaWRhdGU7Cj4+ Pj4+Pj4gIAl1bnNpZ25lZCBsb25nIGZsYWdzOwo+Pj4+Pj4+IAo+Pj4+Pj4+ICAJLyoga2VlcCBs YXN0ISAqLwo+Pj4+Pj4+IEBAIC0xMTQ1LDYgKzExNDYsOCBAQCBzdHJ1Y3QgaWVlZTgwMjExX2xv Y2FsIHsKPj4+Pj4+PiAgCXU2NCBhaXJ0aW1lX3ZfdFtJRUVFODAyMTFfTlVNX0FDU107Cj4+Pj4+ Pj4gIAl1NjQgYWlydGltZV93ZWlnaHRfc3VtW0lFRUU4MDIxMV9OVU1fQUNTXTsKPj4+Pj4+PiAK Pj4+Pj4+PiArCXN0cnVjdCBsaXN0X2hlYWQgcmVtb3ZlX2xpc3RbSUVFRTgwMjExX05VTV9BQ1Nd Owo+Pj4+Pj4+ICsJc3RydWN0IHRpbWVyX2xpc3QgcmVtb3ZlX3RpbWVyOwo+Pj4+Pj4+ICAJdTE2 IGFpcnRpbWVfZmxhZ3M7Cj4+Pj4+Pj4gCj4+Pj4+Pj4gIAljb25zdCBzdHJ1Y3QgaWVlZTgwMjEx X29wcyAqb3BzOwo+Pj4+Pj4+IGRpZmYgLS1naXQgYS9uZXQvbWFjODAyMTEvbWFpbi5jIGIvbmV0 L21hYzgwMjExL21haW4uYwo+Pj4+Pj4+IGluZGV4IGU5ZmZhOGUuLjc4ZmUyNGEgMTAwNjQ0Cj4+ Pj4+Pj4gLS0tIGEvbmV0L21hYzgwMjExL21haW4uYwo+Pj4+Pj4+ICsrKyBiL25ldC9tYWM4MDIx MS9tYWluLmMKPj4+Pj4+PiBAQCAtNjY3LDEwICs2NjcsMTUgQEAgc3RydWN0IGllZWU4MDIxMV9o dwo+Pj4+Pj4+ICppZWVlODAyMTFfYWxsb2NfaHdfbm0oc2l6ZV90IHByaXZfZGF0YV9sZW4sCj4+ Pj4+Pj4gCj4+Pj4+Pj4gIAlmb3IgKGkgPSAwOyBpIDwgSUVFRTgwMjExX05VTV9BQ1M7IGkrKykg ewo+Pj4+Pj4+ICAJCWxvY2FsLT5hY3RpdmVfdHhxc1tpXSA9IFJCX1JPT1RfQ0FDSEVEOwo+Pj4+ Pj4+ICsJCUlOSVRfTElTVF9IRUFEKCZsb2NhbC0+cmVtb3ZlX2xpc3RbaV0pOwo+Pj4+Pj4+ICAJ CXNwaW5fbG9ja19pbml0KCZsb2NhbC0+YWN0aXZlX3R4cV9sb2NrW2ldKTsKPj4+Pj4+PiAgCX0K Pj4+Pj4+PiAgCWxvY2FsLT5haXJ0aW1lX2ZsYWdzID0gQUlSVElNRV9VU0VfVFggfCBBSVJUSU1F X1VTRV9SWDsKPj4+Pj4+PiAKPj4+Pj4+PiArCXRpbWVyX3NldHVwKCZsb2NhbC0+cmVtb3ZlX3Rp bWVyLCBpZWVlODAyMTFfdHhxc19jaGVjaywgMCk7Cj4+Pj4+Pj4gKwltb2RfdGltZXIoJmxvY2Fs LT5yZW1vdmVfdGltZXIsCj4+Pj4+Pj4gKwkJICBqaWZmaWVzICsKPj4+Pj4+PiBtc2Vjc190b19q aWZmaWVzKElFRUU4MDIxMV9BSVJUSU1FX1RYUV9STV9DSEtfSU5UVl9JTl9NUykpOwo+Pj4+Pj4+ ICsKPj4+Pj4+PiAgCUlOSVRfTElTVF9IRUFEKCZsb2NhbC0+Y2hhbmN0eF9saXN0KTsKPj4+Pj4+ PiAgCW11dGV4X2luaXQoJmxvY2FsLT5jaGFuY3R4X210eCk7Cj4+Pj4+Pj4gCj4+Pj4+Pj4gQEAg LTEzMDUsNiArMTMxMCw3IEBAIHZvaWQgaWVlZTgwMjExX3VucmVnaXN0ZXJfaHcoc3RydWN0Cj4+ Pj4+Pj4gaWVlZTgwMjExX2h3Cj4+Pj4+Pj4gKmh3KQo+Pj4+Pj4+ICAJdGFza2xldF9raWxsKCZs b2NhbC0+dHhfcGVuZGluZ190YXNrbGV0KTsKPj4+Pj4+PiAgCXRhc2tsZXRfa2lsbCgmbG9jYWwt PnRhc2tsZXQpOwo+Pj4+Pj4+IAo+Pj4+Pj4+ICsJZGVsX3RpbWVyX3N5bmMoJmxvY2FsLT5yZW1v dmVfdGltZXIpOwo+Pj4+Pj4+ICAjaWZkZWYgQ09ORklHX0lORVQKPj4+Pj4+PiAgCXVucmVnaXN0 ZXJfaW5ldGFkZHJfbm90aWZpZXIoJmxvY2FsLT5pZmFfbm90aWZpZXIpOwo+Pj4+Pj4+ICAjZW5k aWYKPj4+Pj4+PiBkaWZmIC0tZ2l0IGEvbmV0L21hYzgwMjExL3R4LmMgYi9uZXQvbWFjODAyMTEv dHguYwo+Pj4+Pj4+IGluZGV4IGQwMGJhYWEuLjQyY2EwMTAgMTAwNjQ0Cj4+Pj4+Pj4gLS0tIGEv bmV0L21hYzgwMjExL3R4LmMKPj4+Pj4+PiArKysgYi9uZXQvbWFjODAyMTEvdHguYwo+Pj4+Pj4+ IEBAIC0xNDUwLDYgKzE0NTAsNyBAQCB2b2lkIGllZWU4MDIxMV90eHFfaW5pdChzdHJ1Y3QKPj4+ Pj4+PiBpZWVlODAyMTFfc3ViX2lmX2RhdGEgKnNkYXRhLAo+Pj4+Pj4+ICAJY29kZWxfc3RhdHNf aW5pdCgmdHhxaS0+Y3N0YXRzKTsKPj4+Pj4+PiAgCV9fc2tiX3F1ZXVlX2hlYWRfaW5pdCgmdHhx aS0+ZnJhZ3MpOwo+Pj4+Pj4+ICAJUkJfQ0xFQVJfTk9ERSgmdHhxaS0+c2NoZWR1bGVfb3JkZXIp Owo+Pj4+Pj4+ICsJSU5JVF9MSVNUX0hFQUQoJnR4cWktPmNhbmRpZGF0ZSk7Cj4+Pj4+Pj4gCj4+ Pj4+Pj4gIAl0eHFpLT50eHEudmlmID0gJnNkYXRhLT52aWY7Cj4+Pj4+Pj4gCj4+Pj4+Pj4gQEAg LTM3MjQsNiArMzcyNSw5IEBAIHZvaWQgaWVlZTgwMjExX3NjaGVkdWxlX3R4cShzdHJ1Y3QKPj4+ Pj4+PiBpZWVlODAyMTFfaHcKPj4+Pj4+PiAqaHcsCj4+Pj4+Pj4gCj4+Pj4+Pj4gIAlzcGluX2xv Y2tfYmgoJmxvY2FsLT5hY3RpdmVfdHhxX2xvY2tbYWNdKTsKPj4+Pj4+PiAKPj4+Pj4+PiArCWlm ICghbGlzdF9lbXB0eSgmdHhxaS0+Y2FuZGlkYXRlKSkKPj4+Pj4+PiArCQlsaXN0X2RlbF9pbml0 KCZ0eHFpLT5jYW5kaWRhdGUpOwo+Pj4+Pj4+ICsKPj4+Pj4+PiAgCWlmICghUkJfRU1QVFlfTk9E RSgmdHhxaS0+c2NoZWR1bGVfb3JkZXIpKQo+Pj4+Pj4+ICAJCWdvdG8gb3V0Owo+Pj4+Pj4+IAo+ Pj4+Pj4+IEBAIC0zNzgzLDYgKzM3ODcsMjAgQEAgc3RhdGljIHZvaWQgCj4+Pj4+Pj4gX19pZWVl ODAyMTFfdW5zY2hlZHVsZV90eHEoc3RydWN0Cj4+Pj4+Pj4gaWVlZTgwMjExX2h3ICpodywKPj4+ Pj4+PiAgCVJCX0NMRUFSX05PREUoJnR4cWktPnNjaGVkdWxlX29yZGVyKTsKPj4+Pj4+PiAgfQo+ Pj4+Pj4+IAo+Pj4+Pj4+ICt2b2lkIGllZWU4MDIxMV9yZW1vdmVfdHhxKHN0cnVjdCBpZWVlODAy MTFfaHcgKmh3LAo+Pj4+Pj4+ICsJCQkgIHN0cnVjdCBpZWVlODAyMTFfdHhxICp0eHEpCj4+Pj4+ Pj4gK3sKPj4+Pj4+PiArCXN0cnVjdCBpZWVlODAyMTFfbG9jYWwgKmxvY2FsID0gaHdfdG9fbG9j YWwoaHcpOwo+Pj4+Pj4+ICsJc3RydWN0IHR4cV9pbmZvICp0eHFpID0gdG9fdHhxX2luZm8odHhx KTsKPj4+Pj4+PiArCj4+Pj4+Pj4gKwlsb2NrZGVwX2Fzc2VydF9oZWxkKCZsb2NhbC0+YWN0aXZl X3R4cV9sb2NrW3R4cS0+YWNdKTsKPj4+Pj4+PiArCj4+Pj4+Pj4gKwlpZiAoIVJCX0VNUFRZX05P REUoJnR4cWktPnNjaGVkdWxlX29yZGVyKSkgewo+Pj4+Pj4+ICsJCV9faWVlZTgwMjExX3Vuc2No ZWR1bGVfdHhxKGh3LCB0eHEpOwo+Pj4+Pj4+ICsJCWxpc3RfZGVsX2luaXQoJnR4cWktPmNhbmRp ZGF0ZSk7Cj4+Pj4+Pj4gKwl9Cj4+Pj4+Pj4gK30KPj4+Pj4+PiArCj4+Pj4+Pj4gIHZvaWQgaWVl ZTgwMjExX3Vuc2NoZWR1bGVfdHhxKHN0cnVjdCBpZWVlODAyMTFfaHcgKmh3LAo+Pj4+Pj4+ICAJ CQkgICAgICBzdHJ1Y3QgaWVlZTgwMjExX3R4cSAqdHhxKQo+Pj4+Pj4+ICAJX19hY3F1aXJlcyh0 eHFfbG9jaykgX19yZWxlYXNlcyh0eHFfbG9jaykKPj4+Pj4+PiBAQCAtMzc5MCw3ICszODA4LDcg QEAgdm9pZCBpZWVlODAyMTFfdW5zY2hlZHVsZV90eHEoc3RydWN0Cj4+Pj4+Pj4gaWVlZTgwMjEx X2h3ICpodywKPj4+Pj4+PiAgCXN0cnVjdCBpZWVlODAyMTFfbG9jYWwgKmxvY2FsID0gaHdfdG9f bG9jYWwoaHcpOwo+Pj4+Pj4+IAo+Pj4+Pj4+ICAJc3Bpbl9sb2NrX2JoKCZsb2NhbC0+YWN0aXZl X3R4cV9sb2NrW3R4cS0+YWNdKTsKPj4+Pj4+PiAtCV9faWVlZTgwMjExX3Vuc2NoZWR1bGVfdHhx KGh3LCB0eHEpOwo+Pj4+Pj4+ICsJaWVlZTgwMjExX3JlbW92ZV90eHEoaHcsIHR4cSk7Cj4+Pj4+ Pj4gIAlzcGluX3VubG9ja19iaCgmbG9jYWwtPmFjdGl2ZV90eHFfbG9ja1t0eHEtPmFjXSk7Cj4+ Pj4+Pj4gIH0KPj4+Pj4+PiAKPj4+Pj4+PiBAQCAtMzgwMywxMSArMzgyMSw0OCBAQCB2b2lkIGll ZWU4MDIxMV9yZXR1cm5fdHhxKHN0cnVjdAo+Pj4+Pj4+IGllZWU4MDIxMV9odwo+Pj4+Pj4+ICpo dywKPj4+Pj4+PiAgCWxvY2tkZXBfYXNzZXJ0X2hlbGQoJmxvY2FsLT5hY3RpdmVfdHhxX2xvY2tb dHhxLT5hY10pOwo+Pj4+Pj4+IAo+Pj4+Pj4+ICAJaWYgKCFSQl9FTVBUWV9OT0RFKCZ0eHFpLT5z Y2hlZHVsZV9vcmRlcikgJiYKPj4+Pj4+PiAtCSAgICAoc2tiX3F1ZXVlX2VtcHR5KCZ0eHFpLT5m cmFncykgJiYgCj4+Pj4+Pj4gIXR4cWktPnRpbi5iYWNrbG9nX3BhY2tldHMpKQo+Pj4+Pj4+IC0J CV9faWVlZTgwMjExX3Vuc2NoZWR1bGVfdHhxKGh3LCB0eHEpOwo+Pj4+Pj4+ICsJCSF0eHFfaGFz X3F1ZXVlKCZ0eHFpLT50eHEpICYmCj4+Pj4+Pj4gKwkJbGlzdF9lbXB0eSgmdHhxaS0+Y2FuZGlk YXRlKSkKPj4+Pj4+PiArCQlsaXN0X2FkZF90YWlsKCZ0eHFpLT5jYW5kaWRhdGUsICZsb2NhbC0+ cmVtb3ZlX2xpc3RbdHhxLT5hY10pOwo+Pj4+Pj4+ICsKPj4+Pj4+PiAgfQo+Pj4+Pj4+ICBFWFBP UlRfU1lNQk9MKGllZWU4MDIxMV9yZXR1cm5fdHhxKTsKPj4+Pj4+PiAKPj4+Pj4+PiArdm9pZCBf X2llZWU4MDIxMV9jaGVja190eHFzKHN0cnVjdCBpZWVlODAyMTFfbG9jYWwgKmxvY2FsLCBpbnQg Cj4+Pj4+Pj4gYWMpCj4+Pj4+Pj4gK3sKPj4+Pj4+PiArCXN0cnVjdCB0eHFfaW5mbyAqaXRlciwg KnRtcDsKPj4+Pj4+PiArCXN0cnVjdCBzdGFfaW5mbyAqc3RhOwo+Pj4+Pj4+ICsKPj4+Pj4+PiAr CWxvY2tkZXBfYXNzZXJ0X2hlbGQoJmxvY2FsLT5hY3RpdmVfdHhxX2xvY2tbYWNdKTsKPj4+Pj4+ PiArCj4+Pj4+Pj4gKwlsaXN0X2Zvcl9lYWNoX2VudHJ5X3NhZmUoaXRlciwgdG1wLCAmbG9jYWwt PnJlbW92ZV9saXN0W2FjXSwKPj4+Pj4+PiArCQkJCSBjYW5kaWRhdGUpIHsKPj4+Pj4+PiArCQlz dGEgPSBjb250YWluZXJfb2YoaXRlci0+dHhxLnN0YSwgc3RydWN0IHN0YV9pbmZvLCBzdGEpOwo+ Pj4+Pj4+ICsKPj4+Pj4+PiArCQlpZiAodHhxX2hhc19xdWV1ZSgmaXRlci0+dHhxKSkKPj4+Pj4+ PiArCQkJbGlzdF9kZWxfaW5pdCgmaXRlci0+Y2FuZGlkYXRlKTsKPj4+Pj4+PiArCQllbHNlCj4+ Pj4+Pj4gKwkJCWllZWU4MDIxMV9yZW1vdmVfdHhxKCZsb2NhbC0+aHcsICZpdGVyLT50eHEpOwo+ Pj4+Pj4+ICsJfQo+Pj4+Pj4+ICt9Cj4+Pj4+Pj4gKwo+Pj4+Pj4+ICt2b2lkIGllZWU4MDIxMV90 eHFzX2NoZWNrKHN0cnVjdCB0aW1lcl9saXN0ICp0KQo+Pj4+Pj4+ICt7Cj4+Pj4+Pj4gKwlzdHJ1 Y3QgaWVlZTgwMjExX2xvY2FsICpsb2NhbCA9IGZyb21fdGltZXIobG9jYWwsIHQsCj4+Pj4+Pj4g cmVtb3ZlX3RpbWVyKTsKPj4+Pj4+PiArCXN0cnVjdCB0eHFfaW5mbyAqaXRlciwgKnRtcDsKPj4+ Pj4+PiArCXN0cnVjdCBzdGFfaW5mbyAqc3RhOwo+Pj4+Pj4+ICsJaW50IGFjOwo+Pj4+Pj4+ICsK Pj4+Pj4+PiArCWZvciAoYWMgPSAwOyBhYyA8IElFRUU4MDIxMV9OVU1fQUNTOyBhYysrKSB7Cj4+ Pj4+Pj4gKwkJc3Bpbl9sb2NrX2JoKCZsb2NhbC0+YWN0aXZlX3R4cV9sb2NrW2FjXSk7Cj4+Pj4+ Pj4gKwkJX19pZWVlODAyMTFfY2hlY2tfdHhxcyhsb2NhbCwgYWMpOwo+Pj4+Pj4+ICsJCXNwaW5f dW5sb2NrX2JoKCZsb2NhbC0+YWN0aXZlX3R4cV9sb2NrW2FjXSk7Cj4+Pj4+Pj4gKwl9Cj4+Pj4+ Pj4gKwo+Pj4+Pj4+ICsJbW9kX3RpbWVyKCZsb2NhbC0+cmVtb3ZlX3RpbWVyLAo+Pj4+Pj4+ICsJ CSAgamlmZmllcyArCj4+Pj4+Pj4gbXNlY3NfdG9famlmZmllcyhJRUVFODAyMTFfQUlSVElNRV9U WFFfUk1fQ0hLX0lOVFZfSU5fTVMpKTsKPj4+Pj4+PiArfQo+Pj4+Pj4gCj4+Pj4+PiBJJ2xsIGFz ayB0aGUgc2FtZSBhcyBJIGRpZCBsYXN0IHRpbWUgKHdoZXJlIHlvdSB0b2xkIG1lIHRvIGhvbGQg b2ZmCj4+Pj4+PiB1bnRpbCB0aGlzIHJvdW5kKToKPj4+Pj4+IAo+Pj4+Pj4gV2h5IGRvIHlvdSBu ZWVkIHRoZSB0aW1lciBhbmQgdGhlIHBlcmlvZGljIGNoZWNrPyBJZiBUWFFzIGFyZSBhZGRlZAo+ Pj4+Pj4gdG8KPj4+Pj4+IHRoZSByZW1vdmUgbGlzdCBkdXJpbmcgdGhlIHNjaGVkdWxpbmcgcnVu LCBhbmQKPj4+Pj4+IF9faWVlZTgwMjExX2NoZWNrX3R4cXMoKQo+Pj4+Pj4gaXMgcnVuIGZyb20g c2NoZWR1bGVfZW5kKCksIGlzbid0IHRoYXQgc3VmZmljaWVudCB0byBjbGVhciB0aGUgCj4+Pj4+ PiBsaXN0Pwo+Pj4+PiBJcyBpdCBwb3NzaWJsZSB0aGF0IGEgdHhxIGlzIG5vdCBhZGRlZCB0byB0 aGUgcmVtb3ZlIGxpc3QgYnV0IHRoZW4KPj4+Pj4gcGFja2V0cyBpbiBpdCBhcmUgZHJvcHBlZCBi eSBmcV9jb2RlbCBhbGdvPyBMaWtlIHRoZSBzdGF0aW9uCj4+Pj4+IGRpc2Nvbm5lY3RzCj4+Pj4+ IHdpdGhvdXQgYW55IG5vdGlmaWNhdGlvbi4KPj4+PiAKPj4+PiBXZWxsIGFzIGxvbmcgYXMgYWxs IHRoZSBvdGhlciBjbGVhbnVwIHBhdGhzIGNhbGwgZGlyZWN0bHkgaW50bwo+Pj4+IF9fdW5zY2hl ZHVsZV90eHEoKSwgdGhhdCBzaG91bGQgcmVtb3ZlIHN0YXRpb25zIGZyb20gdGhlIHNjaGVkdWxl ciAKPj4+PiB3aGVuCj4+Pj4gdGhleSBkaXNjb25uZWN0IGV0Yy4KPj4+IFllcywgdGhlIGRpc2Nv bm5lY3Qgc2NlbmFyaW8gaXMgYSBiYWQgZXhhbXBsZS4gTXkgY29uY2VybiBpcywgc2F5LCB3ZQo+ Pj4gaGF2ZSAxMCBzdGF0aW9ucyBhbmQgb25seSBvbmUgb2YgdGhlbSBpcyBhc3NpZ25lZCBhIHZl cnkgc21hbGwgd2VpZ2h0Cj4+PiBjb21wYXJlZCB3aXRoIHRoYXQgb2Ygb3RoZXJzLiBTdXBwb3Nl LCBhZnRlciBpdHMgY2hhbmNlIG9mIFR4LCBpdCBpcwo+Pj4gbW9zdCBsaWtlbHkgdG8gYmUgcGxh Y2VkIGluIHRoZSByaWdodG1vc3Qoc3RpbGwgaGFzIHNvbWUgcGFja2V0cyBpbiAKPj4+IHRoZQo+ Pj4gdHhxKSBhbmQgbm8gbW9yZSBpbmNvbWluZyBkYXRhIGZvciBpdC4gVGhlIHJlbWFpbmluZyBw YWNrZXRzIGluIHR4cSAKPj4+IHdpbGwKPj4+IGJlIGRyb3BwZWQgZHVlIHRvIHRpbWVvdXQgYWxn byBpbiBjb2RlbChjb3JyZWN0IG1lIGlmIEkgYW0gd3JvbmcpIGJ1dAo+Pj4gdGhpcyBlbXB0eSB0 eHEgd2lsbCBzdGF5IG9uIHRoZSByYnRyZWUgdW50aWwgb3RoZXIgdHhxcyBnZXQgZHJhaW5lZCBv cgo+Pj4gZ2xvYmFsIHZ0IGNhdGNoIHVwIHdpdGggaXRzIHZ0LiBUaGUgc3RheWluZyB0aW1lIGNv dWxkIGJlIGxvbmcgaWYgCj4+PiB3ZWlnaHQKPj4+IGlzIGV4dHJlbWVseSBzbWFsbC4gVGhlbiBk byB3ZSBuZWVkIHRpbWVyIHRvIGNoZWNrIG9yIGFueSBvdGhlciBiZXR0ZXIKPj4+IHNvbHV0aW9u Pwo+PiAKPj4gQWgsIEkgc2VlIHdoYXQgeW91IG1lYW4uIE5vLCBJIGRvbid0IHRoaW5rIHRoaXMg d2lsbCBiZSBhIHByb2JsZW07IHRoZQo+PiBzY2VuYXJpbyB5b3UncmUgZGVzY3JpYmluZyB3b3Vs ZCBwbGF5IG91dCBsaWtlIHRoaXM6Cj4+IAo+PiAxLiBTdGF0aW9uIGVuZHMgdHJhbnNtaXR0aW5n LCBzdGlsbCBoYXMgYSBzaW5nbGUgcGFja2V0IHF1ZXVlZCwgZ2V0cwo+PiAgICBtb3ZlZCB0byB0 aGUgZW5kIG9mIHRoZSByYnRyZWUgKGFuZCBzdGF5cyB0aGVyZSBmb3IgYSB3aGlsZSkuCj4+IAo+ PiAyLiBXaGVuIHdlIGZpbmFsbHkgZ2V0IHRvIHRoZSBwb2ludCB3aGVyZSB0aGlzIHN0YXRpb24g Z2V0cyBhbm90aGVyCj4+ICAgIGNoYW5jZSB0byB0cmFuc21pdCwgdGhlIENvRGVsIGRyb3AgdGlt ZXIgdHJpZ2dlcnMgYW5kIHRoZSBsYXN0IAo+PiBwYWNrZXQKPj4gICAgaXMgZHJvcHBlZFswXS4g VGhpcyBtZWFucyB0aGF0IHRoZSBxdWV1ZSB3aWxsIGp1c3QgYmUgZW1wdHkKPj4gICAgKGFuZCBp ZWVlODAyMTFfdHhfZGVxdWV1ZSgpIHdpbGwgcmV0dXJuIE5VTEwpLgo+PiAKPj4gMy4gQmVjYXVz ZSB0aGUgcXVldWUgaXMgZW1wdHksIGllZWU4MDIxMV9yZXR1cm5fdHhxKCkgd2lsbCBub3QgcHV0 IGl0Cj4+ICAgIGJhY2sgb24gdGhlIHJidHJlZS4KPj4gCj4+IENydWNpYWxseSwgaW4gMi4gdGhl IENvRGVsIGFsZ29yaXRobSBkb2Vzbid0IGtpY2sgaW4gdW50aWwgdGhlIHBvaW50IG9mCj4+IHBh Y2tldCBkZXF1ZXVlLiBCdXQgZXZlbiBpZiBhbiBlbXB0eSBxdWV1ZSBzdGF5cyBvbiB0aGUgcmJ0 cmVlIGZvciBhCj4+IHdoaWxlLCB0aGVyZSBpcyBubyBoYXJtIGluIHRoYXQ6IGV2ZW50dWFsbHkg aXQgd2lsbCBnZXQgaXRzIHR1cm4sIGl0Cj4+IHdpbGwgdHVybiBvdXQgdG8gYmUgZW1wdHksIGFu ZCBqdXN0IGJlIHNraXBwZWQgb3Zlci4KPiBUaGVuIHRoYXQgd2lsbCBiZSBmaW5lLiBUaGFua3Mg Zm9yIHRoZSBleHBsYW5hdGlvbiBvZiB0aGUgZHJvcHBpbmcgcGFydCAKPiBpbiBDb0RlbCBhbGdv cml0aG0uCgpZdXAsIHRoaW5rIHNvLiBBbmQgeW91J3JlIHdlbGNvbWUgOikKCj4+IFRoZSBpc3N1 ZSB3ZSBuZWVkIHRvIGJlIGNvbmNlcm5lZCBhYm91dCBpcyB0aGUgb3Bwb3NpdGU6IElmIHdlIGhh dmUgYQo+PiBxdWV1ZSB0aGF0ICpkb2VzKiBoYXZlIHBhY2tldHMgcXVldWVkLCBidXQgd2hpY2gg aXMgKm5vdCogc2NoZWR1bGVkIGZvcgo+PiB0cmFuc21pc3Npb24sIHRoYXQgd2lsbCBzdGFsbCBU WC4KPiBJcyBpdCBieSBkZXNpZ24gc2luY2UgaXRzIHZ0IGlzIG1vcmUgdGhhbiBnbG9iYWwgdnQs IHJpZ2h0PyBUaGUgbGF0dGVuY3kgCj4gbWF5IHNvbWVob3cgZ2V0IGltcGFjdGVkIHRob3VnaC4K CldlbGwsIGl0IHNob3VsZCBzdGlsbCBzdGF5IG9uIHRoZSByYnRyZWUgYXMgbG9uZyBhcyBpdCBo YXMgcGFja2V0cwpxdWV1ZWQuIFdlIGRvbid0IGhhdmUgYSBjaGVjayBhbnl3aGVyZSB0aGF0IHJl c2NoZWR1bGVzIFRYUXMgd2hvc2Ugdl90CmRyb3BzIGJlbG93IGdsb2JhbCB2X3QuLi4KCj4+IFsw XSBDb0RlbCBpbiBtb3N0IGNhc2VzIG9ubHkgZHJvcHMgYSBzaW5nbGUgcGFja2V0IGF0IGEgdGlt ZSwgc28gaXQgCj4+IHdpbGwKPj4gbm90IGNsZWFyIG91dCBhbiBlbnRpcmUgcXVldWUgd2l0aCBt dWx0aXBsZSBwYWNrZXRzIGluIG9uZSBnby4gQnV0IHlvdQo+PiBhcmUgcmlnaHQgdGhhdCBpdCBj b3VsZCBjb25jZWl2YWJseSBkcm9wIHRoZSBsYXN0IHBhY2tldCBpbiBhIHF1ZXVlLgo+PiAKPj4+ PiBXZSBvbmx5IG5lZWQgdG8gZGVmZXIgcmVtb3ZhbCBpbnNpZGUgYSBzaW5nbGUgInNjaGVkdWxp bmcgcm91bmQiIAo+Pj4+IChpLmUuLAo+Pj4+IGJldHdlZW4gYSBwYWlyIG9mIGllZWU4MDIxMV90 eHFfc2NoZWR1bGVfc3RhcnQvZW5kLiBTbyBpZiB3ZSBqdXN0IAo+Pj4+IHdhbGsKPj4+PiB0aGUg cmVtb3ZlIGxpc3QgaW4gc2NoZWR1bGVfZW5kKCkgd2Ugc2hvdWxkIGJlIGVub3VnaCwgbm8/Cj4+ Pj4gCj4+Pj4gSG1tLCBvciBtYXliZSBhIHNpbXBsZXIgd2F5IHRvIGZpeCB0aGUgb3JpZ2luYWwg aXNzdWUgaXMganVzdCB0byBoYXZlCj4+Pj4gdW5zY2hlZHVsZV90eHEoKSB1cGRhdGUgdGhlIHNj aGVkdWxlX3BvcygpIHBvaW50ZXI/Cj4+Pj4gCj4+Pj4gSS5lLiwgdW5zY2hlZHVsZV90eHEgY2hl Y2tzIGlmIHRoZSB0eHEgYmVpbmcgcmVtb3ZlZCBpcyBjdXJyZW50bHkgCj4+Pj4gYmVpbmcKPj4+ PiBwb2ludGVkIHRvIGJ5IHNjaGVkdWxlX3Bvc1thY10sIGFuZCBpZiBpdCBpcywgaXQgdXBkYXRl cyBzY2hlZHVsZV9wb3MKPj4+PiB0bwo+Pj4+IGJlIHRoZSByYl9uZXh0IG9mIHRoZSBjdXJyZW50 IHZhbHVlPwo+Pj4gQWN0dWFsbHksIGlmIHNjaGVkdWxlX3BvcyBpcyB1cGRhdGVkIHRvIHJiX25l eHQgb2YgdGhlIGN1cnJlbnQgdmFsdWUsCj4+PiB0aGVuIGluIHRoZSBuZXh0X3R4cSgpIHdoZXJl IHdlIGFyZSBnb2luZyB0byB1c2UgcmJfbmV4dCBhZ2FpbiBhbmQKPj4+IGZpbmFsbHkgcGljayB0 aGUgbmV4dCBub2RlIG9mIHRoZSBub2RlIHdlIHJlYWxseSB3YW50LiBJcyBpdCBmaW5lIHRvCj4+ PiB1cGRhdGUgc2NoZWR1bGVfcG9zIHRvIE5VTEw/Cj4+IAo+PiBIbW0sIHllYWgsIGdvb2QgcG9p bnQuCj4+IAo+PiBJZiB3ZSBkbyBlbmQgdXAgc2V0dGluZyBzY2hlZHVsZV9wb3MgdG8gTlVMTCBp biB0aGUgbWlkZGxlIG9mIGEKPj4gc2NoZWR1bGluZyByb3VuZCwgdGhhdCB3aWxsIG1ha2UgbmV4 dF90eHEoKSAic3RhcnQgb3ZlciIsIGFuZCBkbyAKPj4gYW5vdGhlcgo+PiBsb29wIHRocm91Z2gg dGhlIHdob2xlIHRoaW5nLiBJIGd1ZXNzIHdlIG1heSBiZSBhYmxlIGhpdCBhIGNhc2Ugd2hlcmUK Pj4gdGhpbmdzIGNhbiBvc2NpbGxhdGUgYmFjayBhbmQgZm9ydGggYmV0d2VlbiBhZGRpdGlvbiBh bmQgcmVtb3ZhbAo+PiByZXN1bHRpbmcgaW4gYW4gaW5maW5pdGUgbG9vcD8gTm90IHN1cmUsIGJ1 dCBhdCBsZWFzdCBJIGNhbid0IHNlZW0gdG8KPj4gY29udmluY2UgbXlzZWxmIHRoYXQgdGhpcyBj YW4ndCBoYXBwZW4uCj4KPiBBcyB0aGUgbG9vcCBvZiBuZXh0X3R4cSB1bmRlciBsb2NrIHByb3Rl Y3Rpb24gYXMgYmVsb3csCj4KPiB0eHFfc2NoZWR1bGVfc3RhcnQoKTsKPiB3aGlsZSh0eHE9bmV4 dF90eHEoKSl7Cj4gLi4uCj4gcmV0dXJuX3R4cSh0eHEpOwo+IH0KPiB0eHFfc2NoZWR1bGVfZW5k KCk7Cj4KPiBJIGRvIG5vdCBzZWUgYW55IGNoYW5jZSBvZiBhZGRpdGlvbiwgbm8/CgpBcyB5b3Ug bm90ZWQgaW4geW91ciBvdGhlciBlbWFpbCwgRmVsaXggcmVkdWNlZCB0aGUgbG9ja2luZy4gQW5k IHllYWgsCndlIG5lZWQgdG8gcmViYXNlIHRoaXMgc2VyaWVzIHRvIGFsc28gaW5jb3Jwb3JhdGUg dGhhdC4gSSBmaWd1cmUgSSBjYW4Kc2VuZCBhbiB1cGRhdGVkIHZlcnNpb24gb2YgdGhlIGZpcnN0 IHBhdGNoIGluIHRoZSBzZXJpZXMgb25jZSB3ZSd2ZQp3b3JrZWQgb3V0IHRoZSByZW1haW5pbmcg aXNzdWVzIHdpdGggeW91ciBmb2xsb3ctdXAgcGF0Y2hlcy4KCj4gSW4gYXRoMTBrLCB3ZSB3aWxs IHVzdWFsbHkgcHVzaCBwYWNrZXRzIG9mIGZpcnN0IHR4cSBhcyBtYW55IGFzIHdlIGNhbgo+IHVu dGlsIGl0IGlzIGRyYWluZWQgYW5kIHRoZW4gbW92ZSB0byB0aGUgbmV4dCBvbmUuIFNvIGlmIGEg dHhxIGdldHMKPiByZW1vdmVkIGluIHRoZSByZXR1cm5fdHhxLCBpdCBzaG91bGQgYWx3YXlzIGJl IHRoZSBsZWZ0bW9zdC4gQW5kCj4gZHVyaW5nIHRoaXMgcGVyaW9kLCBuZWl0aGVyIHZ0IG9mIGFu eSBzdGF0aW9uIG9yIGdsb2JhbCB2dCBjYW4gYmUKPiB1cGRhdGVkIGR1ZSB0byBsb2NrIHByb3Rl Y3Rpb24uCj4KPj4gCj4+IEJ1dCBpbiB0aGF0IGNhc2UsIHdlIGNvdWxkIGZpeCBpdCBieSBqdXN0 IGNvbmRpdGlvbmFsbHkgYXNzaWduaW5nIAo+PiBlaXRoZXIKPj4gcmJfbmV4dCBvciByYl9wcmV2 IHRvIHRoZSBzY2hlZHVsZV9wb3MgaW4gdW5zY2hlZHVsZV90eHEoKT8gSS5lLiwKPj4gc29tZXRo aW5nIGxpa2U6Cj4+IAo+PiBsb2NhbC0+c2NoZWR1bGVfcG9zW2FjXSA9IHJiX25leHQobm9kZSkg PzogcmJfcHJldihub2RlKTsKPiBJIGFtIG5vdCBzdXJlIEkgYW0gZ2V0dGluZyB5b3VyIHBvaW50 LiBTdGlsbCBpbiBuZXh0X3R4cSwgCj4gc2NoZWR1bGVfcG9zW2FjXSB3aWxsIGxlYWQgdXMgdG8g dGhlIG5leHQgbm9kZSBvZiB0aGUgb25lIHdlIHdhbnQuCgpUaGUgbG9naWMgaW4gbmV4dF90eHEg aXMgZGlmZmVyZW50IHdoZW4gc2NoZWR1bGVfcG9zW2FjXSBpcyBOVUxMLCB2cwp3aGVuIHJiX25l eHQoc2NoZWR1bGVfcG9zW2FjXSkgaXMgTlVMTC4gVGhlIGZvcm1lciByZXN0YXJ0cyBhIG5ldwpz Y2hlZHVsaW5nIHJvdW5kLCB3aGlsZSB0aGUgbGF0dGVyIGVuZHMgdGhlIGN1cnJlbnQgcm91bmQu CgotVG9rZQoKCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f CmF0aDEwayBtYWlsaW5nIGxpc3QKYXRoMTBrQGxpc3RzLmluZnJhZGVhZC5vcmcKaHR0cDovL2xp c3RzLmluZnJhZGVhZC5vcmcvbWFpbG1hbi9saXN0aW5mby9hdGgxMGsK From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org X-Spam-Level: X-Spam-Status: No, score=-6.8 required=3.0 tests=DKIM_SIGNED,DKIM_VALID, DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,INCLUDES_PATCH,MAILING_LIST_MULTI, SIGNED_OFF_BY,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.0 Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id CF7C9C49ED7 for ; Fri, 20 Sep 2019 09:15:33 +0000 (UTC) Received: from vger.kernel.org (vger.kernel.org [209.132.180.67]) by mail.kernel.org (Postfix) with ESMTP id 89D5A20665 for ; Fri, 20 Sep 2019 09:15:33 +0000 (UTC) Authentication-Results: mail.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="FRQXnMFO" Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405604AbfITJPc (ORCPT ); Fri, 20 Sep 2019 05:15:32 -0400 Received: from us-smtp-delivery-1.mimecast.com ([207.211.31.120]:28387 "EHLO us-smtp-1.mimecast.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S2405594AbfITJPc (ORCPT ); Fri, 20 Sep 2019 05:15:32 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1568970930; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=E1Ezn1BdF0YFuGEcSb/GM7sGojAhk6Wl8orvnAxflNA=; b=FRQXnMFOKYjfGWXU6JtUvJvXHc9xh69UQN7z/wIUcWF85xA+bYgil7A0GdOnVsfQvYiKvg 1sXoPw3SEJfs9YIo/duFlGxRSygFXiynlynm5iZRoQo3XGLLCmLlofR9QkHWCfeIhpcA4D BWBiyg7H1OYJAPkLwwHdjoZu48HgzJw= Received: from mail-ed1-f69.google.com (mail-ed1-f69.google.com [209.85.208.69]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-129-nEwJ7x0JM6GKvC_T7nQePg-1; Fri, 20 Sep 2019 05:15:29 -0400 Received: by mail-ed1-f69.google.com with SMTP id 38so3813986edr.4 for ; Fri, 20 Sep 2019 02:15:27 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:in-reply-to:references:date :message-id:mime-version:content-transfer-encoding; bh=3rJvTKddHx2y/5FptgU8KPzL9bL71L3WG1N88NnOp5Y=; b=rbn2hVCKxLsSO/i1zUQTpxkKaR0qQgaGYC2dkAl/Hqiodamg2v9e51JSSeBawUx38u 18cbq/NTk0VZs3rzlP+daOjU+5EFyXmZvSVdGrTkfqZ8UigvzI+aQzmtoAqp6nppfyZM /mvZoLvEq4f2f57KPakyy2/IbH6cWbOvtgIMwCmCTTRNGnCJ5KIeVpy4DTV6KaGW28jS VLa16KGteB6KmhzV/GLaayOX4PZIru2N4z7/d7B5oYnxEoWjndK829bRa/YTJ94+RJiu 5x7BBBXxPA8Ne2y6jKv2vJhN92royHPO+pDSHFr699YdSDgSqhxUuHhmP1lzGYftE8rr kQLA== X-Gm-Message-State: APjAAAXgp/dAMNYXA9OIqxpQfQDY6sboKSAIVqc+IYUPKNfxo4pbhr0B jdkG6aCcuyGWy/zd1/63AoUkVb0n0Z3E+7v4UGEhxCFzJy9wNbconBmgsR+OhkmA+0qHb1crNlp sa5rE8wfEIxYk5CZuDOwMjvLqIPo= X-Received: by 2002:a17:906:fc0c:: with SMTP id ov12mr17890540ejb.86.1568970926998; Fri, 20 Sep 2019 02:15:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqyD5VwO8ouG0F9ZTF1fBv4eCDgy07s0y9FzmZVVW27fU7oS0cgLYlCjsaCH1AVEPO+UwUJsdw== X-Received: by 2002:a17:906:fc0c:: with SMTP id ov12mr17890519ejb.86.1568970926668; Fri, 20 Sep 2019 02:15:26 -0700 (PDT) Received: from alrua-x1.borgediget.toke.dk ([2a00:7660:6da:443::2]) by smtp.gmail.com with ESMTPSA id k20sm219803eds.66.2019.09.20.02.15.25 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 20 Sep 2019 02:15:25 -0700 (PDT) Received: by alrua-x1.borgediget.toke.dk (Postfix, from userid 1000) id 24F4718063E; Fri, 20 Sep 2019 11:15:25 +0200 (CEST) From: Toke =?utf-8?Q?H=C3=B8iland-J=C3=B8rgensen?= To: Yibo Zhao Cc: ath10k@lists.infradead.org, linux-wireless@vger.kernel.org, linux-wireless-owner@vger.kernel.org Subject: Re: [PATCH 2/4] mac80211: defer txqs removal from rbtree In-Reply-To: <1b4ab006d9b5c88035845aaac193ef48@codeaurora.org> References: <1568639388-27291-1-git-send-email-yiboz@codeaurora.org> <1568639388-27291-2-git-send-email-yiboz@codeaurora.org> <87pnjyiq7o.fsf@toke.dk> <87sgothmpy.fsf@toke.dk> <8cdece5c030fd95817fb099021c38613@codeaurora.org> <87tv98fu6l.fsf@toke.dk> <1b4ab006d9b5c88035845aaac193ef48@codeaurora.org> X-Clacks-Overhead: GNU Terry Pratchett Date: Fri, 20 Sep 2019 11:15:25 +0200 Message-ID: <8736gre3bm.fsf@toke.dk> MIME-Version: 1.0 X-MC-Unique: nEwJ7x0JM6GKvC_T7nQePg-1 X-Mimecast-Spam-Score: 0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sender: linux-wireless-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-wireless@vger.kernel.org Yibo Zhao writes: > On 2019-09-19 18:37, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >> Yibo Zhao writes: >>=20 >>> On 2019-09-18 19:23, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >>>> Yibo Zhao writes: >>>>=20 >>>>> On 2019-09-18 05:10, Toke H=C3=B8iland-J=C3=B8rgensen wrote: >>>>>> Yibo Zhao writes: >>>>>>=20 >>>>>>> In a loop txqs dequeue scenario, if the first txq in the rbtree=20 >>>>>>> gets >>>>>>> removed from rbtree immediately in the ieee80211_return_txq(), the >>>>>>> loop will break soon in the ieee80211_next_txq() due to=20 >>>>>>> schedule_pos >>>>>>> not leading to the second txq in the rbtree. Thus, defering the >>>>>>> removal right before the end of this schedule round. >>>>>>>=20 >>>>>>> Co-developed-by: Yibo Zhao >>>>>>> Signed-off-by: Yibo Zhao >>>>>>> Signed-off-by: Toke H=C3=B8iland-J=C3=B8rgensen >>>>>>=20 >>>>>> I didn't write this patch, so please don't use my sign-off. I'll=20 >>>>>> add >>>>>> ack or review tags as appropriate in reply; but a few comments=20 >>>>>> first: >>>>>>=20 >>>>>>> --- >>>>>>> include/net/mac80211.h | 16 ++++++++++-- >>>>>>> net/mac80211/ieee80211_i.h | 3 +++ >>>>>>> net/mac80211/main.c | 6 +++++ >>>>>>> net/mac80211/tx.c | 63 >>>>>>> +++++++++++++++++++++++++++++++++++++++++++--- >>>>>>> 4 files changed, 83 insertions(+), 5 deletions(-) >>>>>>>=20 >>>>>>> diff --git a/include/net/mac80211.h b/include/net/mac80211.h >>>>>>> index ac2ed8e..ba5a345 100644 >>>>>>> --- a/include/net/mac80211.h >>>>>>> +++ b/include/net/mac80211.h >>>>>>> @@ -925,6 +925,8 @@ struct ieee80211_tx_rate { >>>>>>>=20 >>>>>>> #define IEEE80211_MAX_TX_RETRY=09=0931 >>>>>>>=20 >>>>>>> +#define IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS 100 >>>>>>> + >>>>>>> static inline void ieee80211_rate_set_vht(struct=20 >>>>>>> ieee80211_tx_rate >>>>>>> *rate, >>>>>>> =09=09=09=09=09 u8 mcs, u8 nss) >>>>>>> { >>>>>>> @@ -6232,7 +6234,8 @@ struct sk_buff *ieee80211_tx_dequeue(struct >>>>>>> ieee80211_hw *hw, >>>>>>> * @ac: AC number to return packets from. >>>>>>> * >>>>>>> * Should only be called between calls to >>>>>>> ieee80211_txq_schedule_start() >>>>>>> - * and ieee80211_txq_schedule_end(). >>>>>>> + * and ieee80211_txq_schedule_end(). If the txq is empty, it will >>>>>>> be >>>>>>> added >>>>>>> + * to a remove list and get removed later. >>>>>>> * Returns the next txq if successful, %NULL if no queue is >>>>>>> eligible. >>>>>>> If a txq >>>>>>> * is returned, it should be returned with ieee80211_return_txq() >>>>>>> after the >>>>>>> * driver has finished scheduling it. >>>>>>> @@ -6268,7 +6271,8 @@ void ieee80211_txq_schedule_start(struct >>>>>>> ieee80211_hw *hw, u8 ac) >>>>>>> * @hw: pointer as obtained from ieee80211_alloc_hw() >>>>>>> * @ac: AC number to acquire locks for >>>>>>> * >>>>>>> - * Release locks previously acquired by >>>>>>> ieee80211_txq_schedule_end(). >>>>>>> + * Release locks previously acquired by >>>>>>> ieee80211_txq_schedule_end(). >>>>>>> Check >>>>>>> + * and remove the empty txq from rb-tree. >>>>>>> */ >>>>>>> void ieee80211_txq_schedule_end(struct ieee80211_hw *hw, u8 ac) >>>>>>> =09__releases(txq_lock); >>>>>>> @@ -6287,6 +6291,14 @@ void ieee80211_schedule_txq(struct >>>>>>> ieee80211_hw >>>>>>> *hw, struct ieee80211_txq *txq) >>>>>>> =09__acquires(txq_lock) __releases(txq_lock); >>>>>>>=20 >>>>>>> /** >>>>>>> + * ieee80211_txqs_check - Check txqs waiting for removal >>>>>>> + * >>>>>>> + * @tmr: pointer as obtained from local >>>>>>> + * >>>>>>> + */ >>>>>>> +void ieee80211_txqs_check(struct timer_list *tmr); >>>>>>> + >>>>>>> +/** >>>>>>> * ieee80211_txq_may_transmit - check whether TXQ is allowed to >>>>>>> transmit >>>>>>> * >>>>>>> * This function is used to check whether given txq is allowed to >>>>>>> transmit by >>>>>>> diff --git a/net/mac80211/ieee80211_i.h=20 >>>>>>> b/net/mac80211/ieee80211_i.h >>>>>>> index a4556f9..49aa143e 100644 >>>>>>> --- a/net/mac80211/ieee80211_i.h >>>>>>> +++ b/net/mac80211/ieee80211_i.h >>>>>>> @@ -847,6 +847,7 @@ struct txq_info { >>>>>>> =09struct codel_stats cstats; >>>>>>> =09struct sk_buff_head frags; >>>>>>> =09struct rb_node schedule_order; >>>>>>> +=09struct list_head candidate; >>>>>>> =09unsigned long flags; >>>>>>>=20 >>>>>>> =09/* keep last! */ >>>>>>> @@ -1145,6 +1146,8 @@ struct ieee80211_local { >>>>>>> =09u64 airtime_v_t[IEEE80211_NUM_ACS]; >>>>>>> =09u64 airtime_weight_sum[IEEE80211_NUM_ACS]; >>>>>>>=20 >>>>>>> +=09struct list_head remove_list[IEEE80211_NUM_ACS]; >>>>>>> +=09struct timer_list remove_timer; >>>>>>> =09u16 airtime_flags; >>>>>>>=20 >>>>>>> =09const struct ieee80211_ops *ops; >>>>>>> diff --git a/net/mac80211/main.c b/net/mac80211/main.c >>>>>>> index e9ffa8e..78fe24a 100644 >>>>>>> --- a/net/mac80211/main.c >>>>>>> +++ b/net/mac80211/main.c >>>>>>> @@ -667,10 +667,15 @@ struct ieee80211_hw >>>>>>> *ieee80211_alloc_hw_nm(size_t priv_data_len, >>>>>>>=20 >>>>>>> =09for (i =3D 0; i < IEEE80211_NUM_ACS; i++) { >>>>>>> =09=09local->active_txqs[i] =3D RB_ROOT_CACHED; >>>>>>> +=09=09INIT_LIST_HEAD(&local->remove_list[i]); >>>>>>> =09=09spin_lock_init(&local->active_txq_lock[i]); >>>>>>> =09} >>>>>>> =09local->airtime_flags =3D AIRTIME_USE_TX | AIRTIME_USE_RX; >>>>>>>=20 >>>>>>> +=09timer_setup(&local->remove_timer, ieee80211_txqs_check, 0); >>>>>>> +=09mod_timer(&local->remove_timer, >>>>>>> +=09=09 jiffies + >>>>>>> msecs_to_jiffies(IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS)); >>>>>>> + >>>>>>> =09INIT_LIST_HEAD(&local->chanctx_list); >>>>>>> =09mutex_init(&local->chanctx_mtx); >>>>>>>=20 >>>>>>> @@ -1305,6 +1310,7 @@ void ieee80211_unregister_hw(struct >>>>>>> ieee80211_hw >>>>>>> *hw) >>>>>>> =09tasklet_kill(&local->tx_pending_tasklet); >>>>>>> =09tasklet_kill(&local->tasklet); >>>>>>>=20 >>>>>>> +=09del_timer_sync(&local->remove_timer); >>>>>>> #ifdef CONFIG_INET >>>>>>> =09unregister_inetaddr_notifier(&local->ifa_notifier); >>>>>>> #endif >>>>>>> diff --git a/net/mac80211/tx.c b/net/mac80211/tx.c >>>>>>> index d00baaa..42ca010 100644 >>>>>>> --- a/net/mac80211/tx.c >>>>>>> +++ b/net/mac80211/tx.c >>>>>>> @@ -1450,6 +1450,7 @@ void ieee80211_txq_init(struct >>>>>>> ieee80211_sub_if_data *sdata, >>>>>>> =09codel_stats_init(&txqi->cstats); >>>>>>> =09__skb_queue_head_init(&txqi->frags); >>>>>>> =09RB_CLEAR_NODE(&txqi->schedule_order); >>>>>>> +=09INIT_LIST_HEAD(&txqi->candidate); >>>>>>>=20 >>>>>>> =09txqi->txq.vif =3D &sdata->vif; >>>>>>>=20 >>>>>>> @@ -3724,6 +3725,9 @@ void ieee80211_schedule_txq(struct >>>>>>> ieee80211_hw >>>>>>> *hw, >>>>>>>=20 >>>>>>> =09spin_lock_bh(&local->active_txq_lock[ac]); >>>>>>>=20 >>>>>>> +=09if (!list_empty(&txqi->candidate)) >>>>>>> +=09=09list_del_init(&txqi->candidate); >>>>>>> + >>>>>>> =09if (!RB_EMPTY_NODE(&txqi->schedule_order)) >>>>>>> =09=09goto out; >>>>>>>=20 >>>>>>> @@ -3783,6 +3787,20 @@ static void=20 >>>>>>> __ieee80211_unschedule_txq(struct >>>>>>> ieee80211_hw *hw, >>>>>>> =09RB_CLEAR_NODE(&txqi->schedule_order); >>>>>>> } >>>>>>>=20 >>>>>>> +void ieee80211_remove_txq(struct ieee80211_hw *hw, >>>>>>> +=09=09=09 struct ieee80211_txq *txq) >>>>>>> +{ >>>>>>> +=09struct ieee80211_local *local =3D hw_to_local(hw); >>>>>>> +=09struct txq_info *txqi =3D to_txq_info(txq); >>>>>>> + >>>>>>> +=09lockdep_assert_held(&local->active_txq_lock[txq->ac]); >>>>>>> + >>>>>>> +=09if (!RB_EMPTY_NODE(&txqi->schedule_order)) { >>>>>>> +=09=09__ieee80211_unschedule_txq(hw, txq); >>>>>>> +=09=09list_del_init(&txqi->candidate); >>>>>>> +=09} >>>>>>> +} >>>>>>> + >>>>>>> void ieee80211_unschedule_txq(struct ieee80211_hw *hw, >>>>>>> =09=09=09 struct ieee80211_txq *txq) >>>>>>> =09__acquires(txq_lock) __releases(txq_lock) >>>>>>> @@ -3790,7 +3808,7 @@ void ieee80211_unschedule_txq(struct >>>>>>> ieee80211_hw *hw, >>>>>>> =09struct ieee80211_local *local =3D hw_to_local(hw); >>>>>>>=20 >>>>>>> =09spin_lock_bh(&local->active_txq_lock[txq->ac]); >>>>>>> -=09__ieee80211_unschedule_txq(hw, txq); >>>>>>> +=09ieee80211_remove_txq(hw, txq); >>>>>>> =09spin_unlock_bh(&local->active_txq_lock[txq->ac]); >>>>>>> } >>>>>>>=20 >>>>>>> @@ -3803,11 +3821,48 @@ void ieee80211_return_txq(struct >>>>>>> ieee80211_hw >>>>>>> *hw, >>>>>>> =09lockdep_assert_held(&local->active_txq_lock[txq->ac]); >>>>>>>=20 >>>>>>> =09if (!RB_EMPTY_NODE(&txqi->schedule_order) && >>>>>>> -=09 (skb_queue_empty(&txqi->frags) &&=20 >>>>>>> !txqi->tin.backlog_packets)) >>>>>>> -=09=09__ieee80211_unschedule_txq(hw, txq); >>>>>>> +=09=09!txq_has_queue(&txqi->txq) && >>>>>>> +=09=09list_empty(&txqi->candidate)) >>>>>>> +=09=09list_add_tail(&txqi->candidate, &local->remove_list[txq->ac]= ); >>>>>>> + >>>>>>> } >>>>>>> EXPORT_SYMBOL(ieee80211_return_txq); >>>>>>>=20 >>>>>>> +void __ieee80211_check_txqs(struct ieee80211_local *local, int=20 >>>>>>> ac) >>>>>>> +{ >>>>>>> +=09struct txq_info *iter, *tmp; >>>>>>> +=09struct sta_info *sta; >>>>>>> + >>>>>>> +=09lockdep_assert_held(&local->active_txq_lock[ac]); >>>>>>> + >>>>>>> +=09list_for_each_entry_safe(iter, tmp, &local->remove_list[ac], >>>>>>> +=09=09=09=09 candidate) { >>>>>>> +=09=09sta =3D container_of(iter->txq.sta, struct sta_info, sta); >>>>>>> + >>>>>>> +=09=09if (txq_has_queue(&iter->txq)) >>>>>>> +=09=09=09list_del_init(&iter->candidate); >>>>>>> +=09=09else >>>>>>> +=09=09=09ieee80211_remove_txq(&local->hw, &iter->txq); >>>>>>> +=09} >>>>>>> +} >>>>>>> + >>>>>>> +void ieee80211_txqs_check(struct timer_list *t) >>>>>>> +{ >>>>>>> +=09struct ieee80211_local *local =3D from_timer(local, t, >>>>>>> remove_timer); >>>>>>> +=09struct txq_info *iter, *tmp; >>>>>>> +=09struct sta_info *sta; >>>>>>> +=09int ac; >>>>>>> + >>>>>>> +=09for (ac =3D 0; ac < IEEE80211_NUM_ACS; ac++) { >>>>>>> +=09=09spin_lock_bh(&local->active_txq_lock[ac]); >>>>>>> +=09=09__ieee80211_check_txqs(local, ac); >>>>>>> +=09=09spin_unlock_bh(&local->active_txq_lock[ac]); >>>>>>> +=09} >>>>>>> + >>>>>>> +=09mod_timer(&local->remove_timer, >>>>>>> +=09=09 jiffies + >>>>>>> msecs_to_jiffies(IEEE80211_AIRTIME_TXQ_RM_CHK_INTV_IN_MS)); >>>>>>> +} >>>>>>=20 >>>>>> I'll ask the same as I did last time (where you told me to hold off >>>>>> until this round): >>>>>>=20 >>>>>> Why do you need the timer and the periodic check? If TXQs are added >>>>>> to >>>>>> the remove list during the scheduling run, and >>>>>> __ieee80211_check_txqs() >>>>>> is run from schedule_end(), isn't that sufficient to clear the=20 >>>>>> list? >>>>> Is it possible that a txq is not added to the remove list but then >>>>> packets in it are dropped by fq_codel algo? Like the station >>>>> disconnects >>>>> without any notification. >>>>=20 >>>> Well as long as all the other cleanup paths call directly into >>>> __unschedule_txq(), that should remove stations from the scheduler=20 >>>> when >>>> they disconnect etc. >>> Yes, the disconnect scenario is a bad example. My concern is, say, we >>> have 10 stations and only one of them is assigned a very small weight >>> compared with that of others. Suppose, after its chance of Tx, it is >>> most likely to be placed in the rightmost(still has some packets in=20 >>> the >>> txq) and no more incoming data for it. The remaining packets in txq=20 >>> will >>> be dropped due to timeout algo in codel(correct me if I am wrong) but >>> this empty txq will stay on the rbtree until other txqs get drained or >>> global vt catch up with its vt. The staying time could be long if=20 >>> weight >>> is extremely small. Then do we need timer to check or any other better >>> solution? >>=20 >> Ah, I see what you mean. No, I don't think this will be a problem; the >> scenario you're describing would play out like this: >>=20 >> 1. Station ends transmitting, still has a single packet queued, gets >> moved to the end of the rbtree (and stays there for a while). >>=20 >> 2. When we finally get to the point where this station gets another >> chance to transmit, the CoDel drop timer triggers and the last=20 >> packet >> is dropped[0]. This means that the queue will just be empty >> (and ieee80211_tx_dequeue() will return NULL). >>=20 >> 3. Because the queue is empty, ieee80211_return_txq() will not put it >> back on the rbtree. >>=20 >> Crucially, in 2. the CoDel algorithm doesn't kick in until the point of >> packet dequeue. But even if an empty queue stays on the rbtree for a >> while, there is no harm in that: eventually it will get its turn, it >> will turn out to be empty, and just be skipped over. > Then that will be fine. Thanks for the explanation of the dropping part= =20 > in CoDel algorithm. Yup, think so. And you're welcome :) >> The issue we need to be concerned about is the opposite: If we have a >> queue that *does* have packets queued, but which is *not* scheduled for >> transmission, that will stall TX. > Is it by design since its vt is more than global vt, right? The lattency= =20 > may somehow get impacted though. Well, it should still stay on the rbtree as long as it has packets queued. We don't have a check anywhere that reschedules TXQs whose v_t drops below global v_t... >> [0] CoDel in most cases only drops a single packet at a time, so it=20 >> will >> not clear out an entire queue with multiple packets in one go. But you >> are right that it could conceivably drop the last packet in a queue. >>=20 >>>> We only need to defer removal inside a single "scheduling round"=20 >>>> (i.e., >>>> between a pair of ieee80211_txq_schedule_start/end. So if we just=20 >>>> walk >>>> the remove list in schedule_end() we should be enough, no? >>>>=20 >>>> Hmm, or maybe a simpler way to fix the original issue is just to have >>>> unschedule_txq() update the schedule_pos() pointer? >>>>=20 >>>> I.e., unschedule_txq checks if the txq being removed is currently=20 >>>> being >>>> pointed to by schedule_pos[ac], and if it is, it updates schedule_pos >>>> to >>>> be the rb_next of the current value? >>> Actually, if schedule_pos is updated to rb_next of the current value, >>> then in the next_txq() where we are going to use rb_next again and >>> finally pick the next node of the node we really want. Is it fine to >>> update schedule_pos to NULL? >>=20 >> Hmm, yeah, good point. >>=20 >> If we do end up setting schedule_pos to NULL in the middle of a >> scheduling round, that will make next_txq() "start over", and do=20 >> another >> loop through the whole thing. I guess we may be able hit a case where >> things can oscillate back and forth between addition and removal >> resulting in an infinite loop? Not sure, but at least I can't seem to >> convince myself that this can't happen. > > As the loop of next_txq under lock protection as below, > > txq_schedule_start(); > while(txq=3Dnext_txq()){ > ... > return_txq(txq); > } > txq_schedule_end(); > > I do not see any chance of addition, no? As you noted in your other email, Felix reduced the locking. And yeah, we need to rebase this series to also incorporate that. I figure I can send an updated version of the first patch in the series once we've worked out the remaining issues with your follow-up patches. > In ath10k, we will usually push packets of first txq as many as we can > until it is drained and then move to the next one. So if a txq gets > removed in the return_txq, it should always be the leftmost. And > during this period, neither vt of any station or global vt can be > updated due to lock protection. > >>=20 >> But in that case, we could fix it by just conditionally assigning=20 >> either >> rb_next or rb_prev to the schedule_pos in unschedule_txq()? I.e., >> something like: >>=20 >> local->schedule_pos[ac] =3D rb_next(node) ?: rb_prev(node); > I am not sure I am getting your point. Still in next_txq,=20 > schedule_pos[ac] will lead us to the next node of the one we want. The logic in next_txq is different when schedule_pos[ac] is NULL, vs when rb_next(schedule_pos[ac]) is NULL. The former restarts a new scheduling round, while the latter ends the current round. -Toke