From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: [v8,1/3] dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended) From: Angelo Dureghello Message-Id: <20180803193244.12084-1-angelo@sysam.it> Date: Fri, 3 Aug 2018 21:32:42 +0200 To: vinod.koul@linaro.org Cc: dmaengine@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-m68k@vger.kernel.org, stefan@agner.ch, krzk@kernel.org, Angelo Dureghello List-ID: VGhpcyBwYXRjaCBhZGRzIGEgbmV3IGZzbC1lZG1hLWNvbW1vbiBtb2R1bGUgdG8gYWxsb3cgbmV3 Cm1jZi1lZG1hIG1vZHVsZSBjb2RlIHRvIHVzZSBtb3N0IG9mIHRoZSBmc2wtZWRtYSBjb2RlLgoK U2lnbmVkLW9mZi1ieTogQW5nZWxvIER1cmVnaGVsbG8gPGFuZ2Vsb0BzeXNhbS5pdD4KLS0tCkNo YW5nZXMgZm9yIHYyOgotIHBhdGNoIHNwbGl0dGVkIGludG8gNAotIGFkZCBtY2YtZWRtYSBhcyBt aW5pbWFsIGRpZmZlcmVudCBwYXJ0cyBmcm9tIGZzbC1lZG1hCgpDaGFuZ2VzIGZvciB2MzoKbm9u ZQoKQ2hhbmdlcyBmb3IgdjQ6Ci0gcGF0Y2ggc2ltcGxpZmllZCBmcm9tIDQvNCBpbnRvIDIvMgot IGNvbGxlY3RpbmcgYWxsIHRoZSBtY2YtZWRtYS1yZWxhdGVkIGNoYW5nZXMKCkNoYW5nZXMgZm9y IHY1Ogpub25lCgpDaGFuZ2VzIGZvciB2NjoKLSBhZGp1c3RlZCBjb21tZW50IGhlYWRlcgotIGZp eGVkIGJpdCBzaGlmdCB3aXRoIEJJVCgpCi0gd2UgbmVlZCB0byBmcmVlIHRoZSBpbnRlcnJ1cHRz IGF0IHJlbW92ZSgpLCBzbyByZW1vdmVkIGFsbCBkZXZtXwogIGludGVycnVwdCByZWxhdGVkIGNh bGxzCgpDaGFuZ2VzIGZvciB2NzoKbm9uZQoKQ2hhbmdlcyBmb3Igdjg6Ci0gcGF0Y2ggcmV3cml0 dGVuIGZyb20gc2NyYXRjaCwgc3BsaXR0ZWQgaW50byAzLCBjb21tb24gY29kZSBpc29sYXRlZCwK ICBtaW5pbWFsIGNoYW5nZXMgZnJvbSB0aGUgb3JpZ2luYWwgRnJlZXNjYWxlIGNvZGUgaGF2ZSBi ZWVuIGRvbmUuCiAgVGhlIHBhdGNoIGhhcyBiZWVuIHRlc3RlZCB3aXRoIGJvdGggSXJpcyArIENv bGlicmkgVnlicmlkIFZGNTAgYW5kCiAgc3RtYXJrMi9tY2Y1NDQxNSBDb2xkZmlyZSBib2FyZHMu Ci0tLQogZHJpdmVycy9kbWEvTWFrZWZpbGUgICAgICAgICAgfCAgIDIgKy0KIGRyaXZlcnMvZG1h L2ZzbC1lZG1hLWNvbW1vbi5jIHwgNTc2ICsrKysrKysrKysrKysrKysrKysrKysrKysrKysKIGRy aXZlcnMvZG1hL2ZzbC1lZG1hLWNvbW1vbi5oIHwgMTk2ICsrKysrKysrKysKIGRyaXZlcnMvZG1h L2ZzbC1lZG1hLmMgICAgICAgIHwgNjk3ICstLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0tLS0t LS0KIDQgZmlsZXMgY2hhbmdlZCwgNzc0IGluc2VydGlvbnMoKyksIDY5NyBkZWxldGlvbnMoLSkK IGNyZWF0ZSBtb2RlIDEwMDY0NCBkcml2ZXJzL2RtYS9mc2wtZWRtYS1jb21tb24uYwogY3JlYXRl IG1vZGUgMTAwNjQ0IGRyaXZlcnMvZG1hL2ZzbC1lZG1hLWNvbW1vbi5oCgpkaWZmIC0tZ2l0IGEv ZHJpdmVycy9kbWEvTWFrZWZpbGUgYi9kcml2ZXJzL2RtYS9NYWtlZmlsZQppbmRleCAyMDNhOTlk NjgzMTUuLjY2MDIyZjU5ZmNhNCAxMDA2NDQKLS0tIGEvZHJpdmVycy9kbWEvTWFrZWZpbGUKKysr IGIvZHJpdmVycy9kbWEvTWFrZWZpbGUKQEAgLTMxLDcgKzMxLDcgQEAgb2JqLSQoQ09ORklHX0RX X0FYSV9ETUFDKSArPSBkdy1heGktZG1hYy8KIG9iai0kKENPTkZJR19EV19ETUFDX0NPUkUpICs9 IGR3Lwogb2JqLSQoQ09ORklHX0VQOTNYWF9ETUEpICs9IGVwOTN4eF9kbWEubwogb2JqLSQoQ09O RklHX0ZTTF9ETUEpICs9IGZzbGRtYS5vCi1vYmotJChDT05GSUdfRlNMX0VETUEpICs9IGZzbC1l ZG1hLm8KK29iai0kKENPTkZJR19GU0xfRURNQSkgKz0gZnNsLWVkbWEubyBmc2wtZWRtYS1jb21t b24ubwogb2JqLSQoQ09ORklHX0ZTTF9SQUlEKSArPSBmc2xfcmFpZC5vCiBvYmotJChDT05GSUdf SFNVX0RNQSkgKz0gaHN1Lwogb2JqLSQoQ09ORklHX0lNR19NRENfRE1BKSArPSBpbWctbWRjLWRt YS5vCmRpZmYgLS1naXQgYS9kcml2ZXJzL2RtYS9mc2wtZWRtYS1jb21tb24uYyBiL2RyaXZlcnMv ZG1hL2ZzbC1lZG1hLWNvbW1vbi5jCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAw MDAwMC4uMGFlNzA5NGY0NzdhCi0tLSAvZGV2L251bGwKKysrIGIvZHJpdmVycy9kbWEvZnNsLWVk bWEtY29tbW9uLmMKQEAgLTAsMCArMSw1NzYgQEAKKy8vIFNQRFgtTGljZW5zZS1JZGVudGlmaWVy OiBHUEwtMi4wCisvLworLy8gQ29weXJpZ2h0IChjKSAyMDEzLTIwMTQgRnJlZXNjYWxlIFNlbWlj b25kdWN0b3IsIEluYworLy8gQ29weXJpZ2h0IChjKSAyMDE3IFN5c2FtLCBBbmdlbG8gRHVyZWdo ZWxsbyAgPGFuZ2Vsb0BzeXNhbS5pdD4KKworI2luY2x1ZGUgPGxpbnV4L2RtYXBvb2wuaD4KKyNp bmNsdWRlIDxsaW51eC9tb2R1bGUuaD4KKyNpbmNsdWRlIDxsaW51eC9zbGFiLmg+CisKKyNpbmNs dWRlICJmc2wtZWRtYS1jb21tb24uaCIKKworLyoKKyAqIFIvVyBmdW5jdGlvbnMgZm9yIGJpZy0g b3IgbGl0dGxlLWVuZGlhbiByZWdpc3RlcnM6CisgKiBUaGUgZURNQSBjb250cm9sbGVyJ3MgZW5k aWFuIGlzIGluZGVwZW5kZW50IG9mIHRoZSBDUFUgY29yZSdzIGVuZGlhbi4KKyAqIEZvciB0aGUg YmlnLWVuZGlhbiBJUCBtb2R1bGUsIHRoZSBvZmZzZXQgZm9yIDgtYml0IG9yIDE2LWJpdCByZWdp c3RlcnMKKyAqIHNob3VsZCBhbHNvIGJlIHN3YXBwZWQgb3Bwb3NpdGUgdG8gdGhhdCBpbiBsaXR0 bGUtZW5kaWFuIElQLgorICovCit1MzIgZWRtYV9yZWFkbChzdHJ1Y3QgZnNsX2VkbWFfZW5naW5l ICplZG1hLCB2b2lkIF9faW9tZW0gKmFkZHIpCit7CisJaWYgKGVkbWEtPmJpZ19lbmRpYW4pCisJ CXJldHVybiBpb3JlYWQzMmJlKGFkZHIpOworCWVsc2UKKwkJcmV0dXJuIGlvcmVhZDMyKGFkZHIp OworfQorRVhQT1JUX1NZTUJPTF9HUEwoZWRtYV9yZWFkbCk7CisKK3ZvaWQgZWRtYV93cml0ZWIo c3RydWN0IGZzbF9lZG1hX2VuZ2luZSAqZWRtYSwgdTggdmFsLCB2b2lkIF9faW9tZW0gKmFkZHIp Cit7CisJLyogc3dhcCB0aGUgcmVnIG9mZnNldCBmb3IgdGhlc2UgaW4gYmlnLWVuZGlhbiBtb2Rl ICovCisJaWYgKGVkbWEtPmJpZ19lbmRpYW4pCisJCWlvd3JpdGU4KHZhbCwgKHZvaWQgX19pb21l bSAqKSgodW5zaWduZWQgbG9uZylhZGRyIF4gMHgzKSk7CisJZWxzZQorCQlpb3dyaXRlOCh2YWws IGFkZHIpOworfQorRVhQT1JUX1NZTUJPTF9HUEwoZWRtYV93cml0ZWIpOworCit2b2lkIGVkbWFf d3JpdGV3KHN0cnVjdCBmc2xfZWRtYV9lbmdpbmUgKmVkbWEsIHUxNiB2YWwsIHZvaWQgX19pb21l bSAqYWRkcikKK3sKKwkvKiBzd2FwIHRoZSByZWcgb2Zmc2V0IGZvciB0aGVzZSBpbiBiaWctZW5k aWFuIG1vZGUgKi8KKwlpZiAoZWRtYS0+YmlnX2VuZGlhbikKKwkJaW93cml0ZTE2YmUodmFsLCAo dm9pZCBfX2lvbWVtICopKCh1bnNpZ25lZCBsb25nKWFkZHIgXiAweDIpKTsKKwllbHNlCisJCWlv d3JpdGUxNih2YWwsIGFkZHIpOworfQorRVhQT1JUX1NZTUJPTF9HUEwoZWRtYV93cml0ZXcpOwor Cit2b2lkIGVkbWFfd3JpdGVsKHN0cnVjdCBmc2xfZWRtYV9lbmdpbmUgKmVkbWEsIHUzMiB2YWws IHZvaWQgX19pb21lbSAqYWRkcikKK3sKKwlpZiAoZWRtYS0+YmlnX2VuZGlhbikKKwkJaW93cml0 ZTMyYmUodmFsLCBhZGRyKTsKKwllbHNlCisJCWlvd3JpdGUzMih2YWwsIGFkZHIpOworfQorRVhQ T1JUX1NZTUJPTF9HUEwoZWRtYV93cml0ZWwpOworCitzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAqdG9f ZnNsX2VkbWFfY2hhbihzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4pCit7CisJcmV0dXJuIGNvbnRhaW5l cl9vZihjaGFuLCBzdHJ1Y3QgZnNsX2VkbWFfY2hhbiwgdmNoYW4uY2hhbik7Cit9CitFWFBPUlRf U1lNQk9MX0dQTCh0b19mc2xfZWRtYV9jaGFuKTsKKworc3RydWN0IGZzbF9lZG1hX2Rlc2MgKnRv X2ZzbF9lZG1hX2Rlc2Moc3RydWN0IHZpcnRfZG1hX2Rlc2MgKnZkKQoreworCXJldHVybiBjb250 YWluZXJfb2YodmQsIHN0cnVjdCBmc2xfZWRtYV9kZXNjLCB2ZGVzYyk7Cit9CitFWFBPUlRfU1lN Qk9MX0dQTCh0b19mc2xfZWRtYV9kZXNjKTsKKworc3RhdGljIHZvaWQgZnNsX2VkbWFfZW5hYmxl X3JlcXVlc3Qoc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9jaGFuKQoreworCXZvaWQgX19pb21l bSAqYWRkciA9IGZzbF9jaGFuLT5lZG1hLT5tZW1iYXNlOworCXUzMiBjaCA9IGZzbF9jaGFuLT52 Y2hhbi5jaGFuLmNoYW5faWQ7CisKKwllZG1hX3dyaXRlYihmc2xfY2hhbi0+ZWRtYSwgRURNQV9T RUVJX1NFRUkoY2gpLCBhZGRyICsgRURNQV9TRUVJKTsKKwllZG1hX3dyaXRlYihmc2xfY2hhbi0+ ZWRtYSwgY2gsIGFkZHIgKyBFRE1BX1NFUlEpOworfQorCit2b2lkIGZzbF9lZG1hX2Rpc2FibGVf cmVxdWVzdChzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAqZnNsX2NoYW4pCit7CisJdm9pZCBfX2lvbWVt ICphZGRyID0gZnNsX2NoYW4tPmVkbWEtPm1lbWJhc2U7CisJdTMyIGNoID0gZnNsX2NoYW4tPnZj aGFuLmNoYW4uY2hhbl9pZDsKKworCWVkbWFfd3JpdGViKGZzbF9jaGFuLT5lZG1hLCBjaCwgYWRk ciArIEVETUFfQ0VSUSk7CisJZWRtYV93cml0ZWIoZnNsX2NoYW4tPmVkbWEsIEVETUFfQ0VFSV9D RUVJKGNoKSwgYWRkciArIEVETUFfQ0VFSSk7Cit9CitFWFBPUlRfU1lNQk9MX0dQTChmc2xfZWRt YV9kaXNhYmxlX3JlcXVlc3QpOworCit2b2lkIGZzbF9lZG1hX2NoYW5fbXV4KHN0cnVjdCBmc2xf ZWRtYV9jaGFuICpmc2xfY2hhbiwKKwkJCXVuc2lnbmVkIGludCBzbG90LCBib29sIGVuYWJsZSkK K3sKKwl1MzIgY2ggPSBmc2xfY2hhbi0+dmNoYW4uY2hhbi5jaGFuX2lkOworCXZvaWQgX19pb21l bSAqbXV4YWRkcjsKKwl1bnNpZ25lZCBpbnQgY2hhbnNfcGVyX211eCwgY2hfb2ZmOworCisJY2hh bnNfcGVyX211eCA9IGZzbF9jaGFuLT5lZG1hLT5uX2NoYW5zIC8gRE1BTVVYX05SOworCWNoX29m ZiA9IGZzbF9jaGFuLT52Y2hhbi5jaGFuLmNoYW5faWQgJSBjaGFuc19wZXJfbXV4OworCW11eGFk ZHIgPSBmc2xfY2hhbi0+ZWRtYS0+bXV4YmFzZVtjaCAvIGNoYW5zX3Blcl9tdXhdOworCXNsb3Qg PSBFRE1BTVVYX0NIQ0ZHX1NPVVJDRShzbG90KTsKKworCWlmIChlbmFibGUpCisJCWlvd3JpdGU4 KEVETUFNVVhfQ0hDRkdfRU5CTCB8IHNsb3QsIG11eGFkZHIgKyBjaF9vZmYpOworCWVsc2UKKwkJ aW93cml0ZTgoRURNQU1VWF9DSENGR19ESVMsIG11eGFkZHIgKyBjaF9vZmYpOworfQorRVhQT1JU X1NZTUJPTF9HUEwoZnNsX2VkbWFfY2hhbl9tdXgpOworCitzdGF0aWMgdW5zaWduZWQgaW50IGZz bF9lZG1hX2dldF90Y2RfYXR0cihlbnVtIGRtYV9zbGF2ZV9idXN3aWR0aCBhZGRyX3dpZHRoKQor eworCXN3aXRjaCAoYWRkcl93aWR0aCkgeworCWNhc2UgMToKKwkJcmV0dXJuIEVETUFfVENEX0FU VFJfU1NJWkVfOEJJVCB8IEVETUFfVENEX0FUVFJfRFNJWkVfOEJJVDsKKwljYXNlIDI6CisJCXJl dHVybiBFRE1BX1RDRF9BVFRSX1NTSVpFXzE2QklUIHwgRURNQV9UQ0RfQVRUUl9EU0laRV8xNkJJ VDsKKwljYXNlIDQ6CisJCXJldHVybiBFRE1BX1RDRF9BVFRSX1NTSVpFXzMyQklUIHwgRURNQV9U Q0RfQVRUUl9EU0laRV8zMkJJVDsKKwljYXNlIDg6CisJCXJldHVybiBFRE1BX1RDRF9BVFRSX1NT SVpFXzY0QklUIHwgRURNQV9UQ0RfQVRUUl9EU0laRV82NEJJVDsKKwlkZWZhdWx0OgorCQlyZXR1 cm4gRURNQV9UQ0RfQVRUUl9TU0laRV8zMkJJVCB8IEVETUFfVENEX0FUVFJfRFNJWkVfMzJCSVQ7 CisJfQorfQorCit2b2lkIGZzbF9lZG1hX2ZyZWVfZGVzYyhzdHJ1Y3QgdmlydF9kbWFfZGVzYyAq dmRlc2MpCit7CisJc3RydWN0IGZzbF9lZG1hX2Rlc2MgKmZzbF9kZXNjOworCWludCBpOworCisJ ZnNsX2Rlc2MgPSB0b19mc2xfZWRtYV9kZXNjKHZkZXNjKTsKKwlmb3IgKGkgPSAwOyBpIDwgZnNs X2Rlc2MtPm5fdGNkczsgaSsrKQorCQlkbWFfcG9vbF9mcmVlKGZzbF9kZXNjLT5lY2hhbi0+dGNk X3Bvb2wsIGZzbF9kZXNjLT50Y2RbaV0udnRjZCwKKwkJCSAgICAgIGZzbF9kZXNjLT50Y2RbaV0u cHRjZCk7CisJa2ZyZWUoZnNsX2Rlc2MpOworfQorRVhQT1JUX1NZTUJPTF9HUEwoZnNsX2VkbWFf ZnJlZV9kZXNjKTsKKworaW50IGZzbF9lZG1hX3Rlcm1pbmF0ZV9hbGwoc3RydWN0IGRtYV9jaGFu ICpjaGFuKQoreworCXN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1h X2NoYW4oY2hhbik7CisJdW5zaWduZWQgbG9uZyBmbGFnczsKKwlMSVNUX0hFQUQoaGVhZCk7CisK KwlzcGluX2xvY2tfaXJxc2F2ZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKKwlmc2xf ZWRtYV9kaXNhYmxlX3JlcXVlc3QoZnNsX2NoYW4pOworCWZzbF9jaGFuLT5lZGVzYyA9IE5VTEw7 CisJZnNsX2NoYW4tPmlkbGUgPSB0cnVlOworCXZjaGFuX2dldF9hbGxfZGVzY3JpcHRvcnMoJmZz bF9jaGFuLT52Y2hhbiwgJmhlYWQpOworCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJmZzbF9jaGFu LT52Y2hhbi5sb2NrLCBmbGFncyk7CisJdmNoYW5fZG1hX2Rlc2NfZnJlZV9saXN0KCZmc2xfY2hh bi0+dmNoYW4sICZoZWFkKTsKKwlyZXR1cm4gMDsKK30KK0VYUE9SVF9TWU1CT0xfR1BMKGZzbF9l ZG1hX3Rlcm1pbmF0ZV9hbGwpOworCitpbnQgZnNsX2VkbWFfcGF1c2Uoc3RydWN0IGRtYV9jaGFu ICpjaGFuKQoreworCXN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1h X2NoYW4oY2hhbik7CisJdW5zaWduZWQgbG9uZyBmbGFnczsKKworCXNwaW5fbG9ja19pcnFzYXZl KCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOworCWlmIChmc2xfY2hhbi0+ZWRlc2MpIHsK KwkJZnNsX2VkbWFfZGlzYWJsZV9yZXF1ZXN0KGZzbF9jaGFuKTsKKwkJZnNsX2NoYW4tPnN0YXR1 cyA9IERNQV9QQVVTRUQ7CisJCWZzbF9jaGFuLT5pZGxlID0gdHJ1ZTsKKwl9CisJc3Bpbl91bmxv Y2tfaXJxcmVzdG9yZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKKwlyZXR1cm4gMDsK K30KK0VYUE9SVF9TWU1CT0xfR1BMKGZzbF9lZG1hX3BhdXNlKTsKKworaW50IGZzbF9lZG1hX3Jl c3VtZShzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4pCit7CisJc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZz bF9jaGFuID0gdG9fZnNsX2VkbWFfY2hhbihjaGFuKTsKKwl1bnNpZ25lZCBsb25nIGZsYWdzOwor CisJc3Bpbl9sb2NrX2lycXNhdmUoJmZzbF9jaGFuLT52Y2hhbi5sb2NrLCBmbGFncyk7CisJaWYg KGZzbF9jaGFuLT5lZGVzYykgeworCQlmc2xfZWRtYV9lbmFibGVfcmVxdWVzdChmc2xfY2hhbik7 CisJCWZzbF9jaGFuLT5zdGF0dXMgPSBETUFfSU5fUFJPR1JFU1M7CisJCWZzbF9jaGFuLT5pZGxl ID0gZmFsc2U7CisJfQorCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJmZzbF9jaGFuLT52Y2hhbi5s b2NrLCBmbGFncyk7CisJcmV0dXJuIDA7Cit9CitFWFBPUlRfU1lNQk9MX0dQTChmc2xfZWRtYV9y ZXN1bWUpOworCitpbnQgZnNsX2VkbWFfc2xhdmVfY29uZmlnKHN0cnVjdCBkbWFfY2hhbiAqY2hh biwKKwkJCQkgc3RydWN0IGRtYV9zbGF2ZV9jb25maWcgKmNmZykKK3sKKwlzdHJ1Y3QgZnNsX2Vk bWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOworCisJZnNsX2NoYW4t PmZzYy5kaXIgPSBjZmctPmRpcmVjdGlvbjsKKwlpZiAoY2ZnLT5kaXJlY3Rpb24gPT0gRE1BX0RF Vl9UT19NRU0pIHsKKwkJZnNsX2NoYW4tPmZzYy5kZXZfYWRkciA9IGNmZy0+c3JjX2FkZHI7CisJ CWZzbF9jaGFuLT5mc2MuYWRkcl93aWR0aCA9IGNmZy0+c3JjX2FkZHJfd2lkdGg7CisJCWZzbF9j aGFuLT5mc2MuYnVyc3QgPSBjZmctPnNyY19tYXhidXJzdDsKKwkJZnNsX2NoYW4tPmZzYy5hdHRy ID0gZnNsX2VkbWFfZ2V0X3RjZF9hdHRyKGNmZy0+c3JjX2FkZHJfd2lkdGgpOworCX0gZWxzZSBp ZiAoY2ZnLT5kaXJlY3Rpb24gPT0gRE1BX01FTV9UT19ERVYpIHsKKwkJZnNsX2NoYW4tPmZzYy5k ZXZfYWRkciA9IGNmZy0+ZHN0X2FkZHI7CisJCWZzbF9jaGFuLT5mc2MuYWRkcl93aWR0aCA9IGNm Zy0+ZHN0X2FkZHJfd2lkdGg7CisJCWZzbF9jaGFuLT5mc2MuYnVyc3QgPSBjZmctPmRzdF9tYXhi dXJzdDsKKwkJZnNsX2NoYW4tPmZzYy5hdHRyID0gZnNsX2VkbWFfZ2V0X3RjZF9hdHRyKGNmZy0+ ZHN0X2FkZHJfd2lkdGgpOworCX0gZWxzZQorCQlyZXR1cm4gLUVJTlZBTDsKKworCXJldHVybiAw OworfQorRVhQT1JUX1NZTUJPTF9HUEwoZnNsX2VkbWFfc2xhdmVfY29uZmlnKTsKKworc3RhdGlj IHNpemVfdCBmc2xfZWRtYV9kZXNjX3Jlc2lkdWUoc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9j aGFuLAorCQlzdHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmRlc2MsIGJvb2wgaW5fcHJvZ3Jlc3MpCit7 CisJc3RydWN0IGZzbF9lZG1hX2Rlc2MgKmVkZXNjID0gZnNsX2NoYW4tPmVkZXNjOworCXZvaWQg X19pb21lbSAqYWRkciA9IGZzbF9jaGFuLT5lZG1hLT5tZW1iYXNlOworCXUzMiBjaCA9IGZzbF9j aGFuLT52Y2hhbi5jaGFuLmNoYW5faWQ7CisJZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uIGRp ciA9IGZzbF9jaGFuLT5mc2MuZGlyOworCWRtYV9hZGRyX3QgY3VyX2FkZHIsIGRtYV9hZGRyOwor CXNpemVfdCBsZW4sIHNpemU7CisJaW50IGk7CisKKwkvKiBjYWxjdWxhdGUgdGhlIHRvdGFsIHNp emUgaW4gdGhpcyBkZXNjICovCisJZm9yIChsZW4gPSBpID0gMDsgaSA8IGZzbF9jaGFuLT5lZGVz Yy0+bl90Y2RzOyBpKyspCisJCWxlbiArPSBsZTMyX3RvX2NwdShlZGVzYy0+dGNkW2ldLnZ0Y2Qt Pm5ieXRlcykKKwkJCSogbGUxNl90b19jcHUoZWRlc2MtPnRjZFtpXS52dGNkLT5iaXRlcik7CisK KwlpZiAoIWluX3Byb2dyZXNzKQorCQlyZXR1cm4gbGVuOworCisJaWYgKGRpciA9PSBETUFfTUVN X1RPX0RFVikKKwkJY3VyX2FkZHIgPSBlZG1hX3JlYWRsKAorCQkJZnNsX2NoYW4tPmVkbWEsIGFk ZHIgKyBFRE1BX1RDRF9TQUREUihjaCkpOworCWVsc2UKKwkJY3VyX2FkZHIgPSBlZG1hX3JlYWRs KAorCQkJZnNsX2NoYW4tPmVkbWEsIGFkZHIgKyBFRE1BX1RDRF9EQUREUihjaCkpOworCisJLyog ZmlndXJlIG91dCB0aGUgZmluaXNoZWQgYW5kIGNhbGN1bGF0ZSB0aGUgcmVzaWR1ZSAqLworCWZv ciAoaSA9IDA7IGkgPCBmc2xfY2hhbi0+ZWRlc2MtPm5fdGNkczsgaSsrKSB7CisJCXNpemUgPSBs ZTMyX3RvX2NwdShlZGVzYy0+dGNkW2ldLnZ0Y2QtPm5ieXRlcykKKwkJCSogbGUxNl90b19jcHUo ZWRlc2MtPnRjZFtpXS52dGNkLT5iaXRlcik7CisJCWlmIChkaXIgPT0gRE1BX01FTV9UT19ERVYp CisJCQlkbWFfYWRkciA9IGxlMzJfdG9fY3B1KGVkZXNjLT50Y2RbaV0udnRjZC0+c2FkZHIpOwor CQllbHNlCisJCQlkbWFfYWRkciA9IGxlMzJfdG9fY3B1KGVkZXNjLT50Y2RbaV0udnRjZC0+ZGFk ZHIpOworCisJCWxlbiAtPSBzaXplOworCQlpZiAoY3VyX2FkZHIgPj0gZG1hX2FkZHIgJiYgY3Vy X2FkZHIgPCBkbWFfYWRkciArIHNpemUpIHsKKwkJCWxlbiArPSBkbWFfYWRkciArIHNpemUgLSBj dXJfYWRkcjsKKwkJCWJyZWFrOworCQl9CisJfQorCisJcmV0dXJuIGxlbjsKK30KKworZW51bSBk bWFfc3RhdHVzIGZzbF9lZG1hX3R4X3N0YXR1cyhzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4sCisJCWRt YV9jb29raWVfdCBjb29raWUsIHN0cnVjdCBkbWFfdHhfc3RhdGUgKnR4c3RhdGUpCit7CisJc3Ry dWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9jaGFuID0gdG9fZnNsX2VkbWFfY2hhbihjaGFuKTsKKwlz dHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmRlc2M7CisJZW51bSBkbWFfc3RhdHVzIHN0YXR1czsKKwl1 bnNpZ25lZCBsb25nIGZsYWdzOworCisJc3RhdHVzID0gZG1hX2Nvb2tpZV9zdGF0dXMoY2hhbiwg Y29va2llLCB0eHN0YXRlKTsKKwlpZiAoc3RhdHVzID09IERNQV9DT01QTEVURSkKKwkJcmV0dXJu IHN0YXR1czsKKworCWlmICghdHhzdGF0ZSkKKwkJcmV0dXJuIGZzbF9jaGFuLT5zdGF0dXM7CisK KwlzcGluX2xvY2tfaXJxc2F2ZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKKwl2ZGVz YyA9IHZjaGFuX2ZpbmRfZGVzYygmZnNsX2NoYW4tPnZjaGFuLCBjb29raWUpOworCWlmIChmc2xf Y2hhbi0+ZWRlc2MgJiYgY29va2llID09IGZzbF9jaGFuLT5lZGVzYy0+dmRlc2MudHguY29va2ll KQorCQl0eHN0YXRlLT5yZXNpZHVlID0KKwkJCWZzbF9lZG1hX2Rlc2NfcmVzaWR1ZShmc2xfY2hh biwgdmRlc2MsIHRydWUpOworCWVsc2UgaWYgKHZkZXNjKQorCQl0eHN0YXRlLT5yZXNpZHVlID0K KwkJCWZzbF9lZG1hX2Rlc2NfcmVzaWR1ZShmc2xfY2hhbiwgdmRlc2MsIGZhbHNlKTsKKwllbHNl CisJCXR4c3RhdGUtPnJlc2lkdWUgPSAwOworCisJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmZnNs X2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKKworCXJldHVybiBmc2xfY2hhbi0+c3RhdHVzOwor fQorRVhQT1JUX1NZTUJPTF9HUEwoZnNsX2VkbWFfdHhfc3RhdHVzKTsKKworc3RhdGljIHZvaWQg ZnNsX2VkbWFfc2V0X3RjZF9yZWdzKHN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiwKKwkJ CQkgIHN0cnVjdCBmc2xfZWRtYV9od190Y2QgKnRjZCkKK3sKKwlzdHJ1Y3QgZnNsX2VkbWFfZW5n aW5lICplZG1hID0gZnNsX2NoYW4tPmVkbWE7CisJdm9pZCBfX2lvbWVtICphZGRyID0gZnNsX2No YW4tPmVkbWEtPm1lbWJhc2U7CisJdTMyIGNoID0gZnNsX2NoYW4tPnZjaGFuLmNoYW4uY2hhbl9p ZDsKKworCS8qCisJICogVENEIHBhcmFtZXRlcnMgYXJlIHN0b3JlZCBpbiBzdHJ1Y3QgZnNsX2Vk bWFfaHdfdGNkIGluIGxpdHRsZQorCSAqIGVuZGlhbiBmb3JtYXQuIEhvd2V2ZXIsIHdlIG5lZWQg dG8gbG9hZCB0aGUgVENEIHJlZ2lzdGVycyBpbgorCSAqIGJpZy0gb3IgbGl0dGxlLWVuZGlhbiBv YmV5aW5nIHRoZSBlRE1BIGVuZ2luZSBtb2RlbCBlbmRpYW4uCisJICovCisJZWRtYV93cml0ZXco ZWRtYSwgMCwgYWRkciArIEVETUFfVENEX0NTUihjaCkpOworCWVkbWFfd3JpdGVsKGVkbWEsIGxl MzJfdG9fY3B1KHRjZC0+c2FkZHIpLCBhZGRyICsgRURNQV9UQ0RfU0FERFIoY2gpKTsKKwllZG1h X3dyaXRlbChlZG1hLCBsZTMyX3RvX2NwdSh0Y2QtPmRhZGRyKSwgYWRkciArIEVETUFfVENEX0RB RERSKGNoKSk7CisKKwllZG1hX3dyaXRldyhlZG1hLCBsZTE2X3RvX2NwdSh0Y2QtPmF0dHIpLCBh ZGRyICsgRURNQV9UQ0RfQVRUUihjaCkpOworCWVkbWFfd3JpdGV3KGVkbWEsIGxlMTZfdG9fY3B1 KHRjZC0+c29mZiksIGFkZHIgKyBFRE1BX1RDRF9TT0ZGKGNoKSk7CisKKwllZG1hX3dyaXRlbChl ZG1hLCBsZTMyX3RvX2NwdSh0Y2QtPm5ieXRlcyksIGFkZHIgKyBFRE1BX1RDRF9OQllURVMoY2gp KTsKKwllZG1hX3dyaXRlbChlZG1hLCBsZTMyX3RvX2NwdSh0Y2QtPnNsYXN0KSwgYWRkciArIEVE TUFfVENEX1NMQVNUKGNoKSk7CisKKwllZG1hX3dyaXRldyhlZG1hLCBsZTE2X3RvX2NwdSh0Y2Qt PmNpdGVyKSwgYWRkciArIEVETUFfVENEX0NJVEVSKGNoKSk7CisJZWRtYV93cml0ZXcoZWRtYSwg bGUxNl90b19jcHUodGNkLT5iaXRlciksIGFkZHIgKyBFRE1BX1RDRF9CSVRFUihjaCkpOworCWVk bWFfd3JpdGV3KGVkbWEsIGxlMTZfdG9fY3B1KHRjZC0+ZG9mZiksIGFkZHIgKyBFRE1BX1RDRF9E T0ZGKGNoKSk7CisKKwllZG1hX3dyaXRlbChlZG1hLAorCQkgICAgbGUzMl90b19jcHUodGNkLT5k bGFzdF9zZ2EpLCBhZGRyICsgRURNQV9UQ0RfRExBU1RfU0dBKGNoKSk7CisKKwllZG1hX3dyaXRl dyhlZG1hLCBsZTE2X3RvX2NwdSh0Y2QtPmNzciksIGFkZHIgKyBFRE1BX1RDRF9DU1IoY2gpKTsK K30KKworc3RhdGljIGlubGluZQordm9pZCBmc2xfZWRtYV9maWxsX3RjZChzdHJ1Y3QgZnNsX2Vk bWFfaHdfdGNkICp0Y2QsIHUzMiBzcmMsIHUzMiBkc3QsCisJCSAgICAgICB1MTYgYXR0ciwgdTE2 IHNvZmYsIHUzMiBuYnl0ZXMsIHUzMiBzbGFzdCwgdTE2IGNpdGVyLAorCQkgICAgICAgdTE2IGJp dGVyLCB1MTYgZG9mZiwgdTMyIGRsYXN0X3NnYSwgYm9vbCBtYWpvcl9pbnQsCisJCSAgICAgICBi b29sIGRpc2FibGVfcmVxLCBib29sIGVuYWJsZV9zZykKK3sKKwl1MTYgY3NyID0gMDsKKworCS8q CisJICogZURNQSBoYXJkd2FyZSBTR3MgcmVxdWlyZSB0aGUgVENEcyB0byBiZSBzdG9yZWQgaW4g bGl0dGxlCisJICogZW5kaWFuIGZvcm1hdCBpcnJlc3BlY3RpdmUgb2YgdGhlIHJlZ2lzdGVyIGVu ZGlhbiBtb2RlbC4KKwkgKiBTbyB3ZSBwdXQgdGhlIHZhbHVlIGluIGxpdHRsZSBlbmRpYW4gaW4g bWVtb3J5LCB3YWl0aW5nCisJICogZm9yIGZzbF9lZG1hX3NldF90Y2RfcmVncyBkb2luZyB0aGUg c3dhcC4KKwkgKi8KKwl0Y2QtPnNhZGRyID0gY3B1X3RvX2xlMzIoc3JjKTsKKwl0Y2QtPmRhZGRy ID0gY3B1X3RvX2xlMzIoZHN0KTsKKworCXRjZC0+YXR0ciA9IGNwdV90b19sZTE2KGF0dHIpOwor CisJdGNkLT5zb2ZmID0gY3B1X3RvX2xlMTYoRURNQV9UQ0RfU09GRl9TT0ZGKHNvZmYpKTsKKwor CXRjZC0+bmJ5dGVzID0gY3B1X3RvX2xlMzIoRURNQV9UQ0RfTkJZVEVTX05CWVRFUyhuYnl0ZXMp KTsKKwl0Y2QtPnNsYXN0ID0gY3B1X3RvX2xlMzIoRURNQV9UQ0RfU0xBU1RfU0xBU1Qoc2xhc3Qp KTsKKworCXRjZC0+Y2l0ZXIgPSBjcHVfdG9fbGUxNihFRE1BX1RDRF9DSVRFUl9DSVRFUihjaXRl cikpOworCXRjZC0+ZG9mZiA9IGNwdV90b19sZTE2KEVETUFfVENEX0RPRkZfRE9GRihkb2ZmKSk7 CisKKwl0Y2QtPmRsYXN0X3NnYSA9IGNwdV90b19sZTMyKEVETUFfVENEX0RMQVNUX1NHQV9ETEFT VF9TR0EoZGxhc3Rfc2dhKSk7CisKKwl0Y2QtPmJpdGVyID0gY3B1X3RvX2xlMTYoRURNQV9UQ0Rf QklURVJfQklURVIoYml0ZXIpKTsKKwlpZiAobWFqb3JfaW50KQorCQljc3IgfD0gRURNQV9UQ0Rf Q1NSX0lOVF9NQUpPUjsKKworCWlmIChkaXNhYmxlX3JlcSkKKwkJY3NyIHw9IEVETUFfVENEX0NT Ul9EX1JFUTsKKworCWlmIChlbmFibGVfc2cpCisJCWNzciB8PSBFRE1BX1RDRF9DU1JfRV9TRzsK KworCXRjZC0+Y3NyID0gY3B1X3RvX2xlMTYoY3NyKTsKK30KKworc3RhdGljIHN0cnVjdCBmc2xf ZWRtYV9kZXNjICpmc2xfZWRtYV9hbGxvY19kZXNjKHN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xf Y2hhbiwKKwkJaW50IHNnX2xlbikKK3sKKwlzdHJ1Y3QgZnNsX2VkbWFfZGVzYyAqZnNsX2Rlc2M7 CisJaW50IGk7CisKKwlmc2xfZGVzYyA9IGt6YWxsb2Moc2l6ZW9mKCpmc2xfZGVzYykgKworCQkJ ICAgc2l6ZW9mKHN0cnVjdCBmc2xfZWRtYV9zd190Y2QpICoKKwkJCSAgIHNnX2xlbiwgR0ZQX05P V0FJVCk7CisJaWYgKCFmc2xfZGVzYykKKwkJcmV0dXJuIE5VTEw7CisKKwlmc2xfZGVzYy0+ZWNo YW4gPSBmc2xfY2hhbjsKKwlmc2xfZGVzYy0+bl90Y2RzID0gc2dfbGVuOworCWZvciAoaSA9IDA7 IGkgPCBzZ19sZW47IGkrKykgeworCQlmc2xfZGVzYy0+dGNkW2ldLnZ0Y2QgPSBkbWFfcG9vbF9h bGxvYyhmc2xfY2hhbi0+dGNkX3Bvb2wsCisJCQkJCUdGUF9OT1dBSVQsICZmc2xfZGVzYy0+dGNk W2ldLnB0Y2QpOworCQlpZiAoIWZzbF9kZXNjLT50Y2RbaV0udnRjZCkKKwkJCWdvdG8gZXJyOwor CX0KKwlyZXR1cm4gZnNsX2Rlc2M7CisKK2VycjoKKwl3aGlsZSAoLS1pID49IDApCisJCWRtYV9w b29sX2ZyZWUoZnNsX2NoYW4tPnRjZF9wb29sLCBmc2xfZGVzYy0+dGNkW2ldLnZ0Y2QsCisJCQkJ ZnNsX2Rlc2MtPnRjZFtpXS5wdGNkKTsKKwlrZnJlZShmc2xfZGVzYyk7CisJcmV0dXJuIE5VTEw7 Cit9CisKK3N0cnVjdCBkbWFfYXN5bmNfdHhfZGVzY3JpcHRvciAqZnNsX2VkbWFfcHJlcF9kbWFf Y3ljbGljKAorCQlzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4sIGRtYV9hZGRyX3QgZG1hX2FkZHIsIHNp emVfdCBidWZfbGVuLAorCQlzaXplX3QgcGVyaW9kX2xlbiwgZW51bSBkbWFfdHJhbnNmZXJfZGly ZWN0aW9uIGRpcmVjdGlvbiwKKwkJdW5zaWduZWQgbG9uZyBmbGFncykKK3sKKwlzdHJ1Y3QgZnNs X2VkbWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOworCXN0cnVjdCBm c2xfZWRtYV9kZXNjICpmc2xfZGVzYzsKKwlkbWFfYWRkcl90IGRtYV9idWZfbmV4dDsKKwlpbnQg c2dfbGVuLCBpOworCXUzMiBzcmNfYWRkciwgZHN0X2FkZHIsIGxhc3Rfc2csIG5ieXRlczsKKwl1 MTYgc29mZiwgZG9mZiwgaXRlcjsKKworCWlmICghaXNfc2xhdmVfZGlyZWN0aW9uKGZzbF9jaGFu LT5mc2MuZGlyKSkKKwkJcmV0dXJuIE5VTEw7CisKKwlzZ19sZW4gPSBidWZfbGVuIC8gcGVyaW9k X2xlbjsKKwlmc2xfZGVzYyA9IGZzbF9lZG1hX2FsbG9jX2Rlc2MoZnNsX2NoYW4sIHNnX2xlbik7 CisJaWYgKCFmc2xfZGVzYykKKwkJcmV0dXJuIE5VTEw7CisJZnNsX2Rlc2MtPmlzY3ljbGljID0g dHJ1ZTsKKworCWRtYV9idWZfbmV4dCA9IGRtYV9hZGRyOworCW5ieXRlcyA9IGZzbF9jaGFuLT5m c2MuYWRkcl93aWR0aCAqIGZzbF9jaGFuLT5mc2MuYnVyc3Q7CisJaXRlciA9IHBlcmlvZF9sZW4g LyBuYnl0ZXM7CisKKwlmb3IgKGkgPSAwOyBpIDwgc2dfbGVuOyBpKyspIHsKKwkJaWYgKGRtYV9i dWZfbmV4dCA+PSBkbWFfYWRkciArIGJ1Zl9sZW4pCisJCQlkbWFfYnVmX25leHQgPSBkbWFfYWRk cjsKKworCQkvKiBnZXQgbmV4dCBzZydzIHBoeXNpY2FsIGFkZHJlc3MgKi8KKwkJbGFzdF9zZyA9 IGZzbF9kZXNjLT50Y2RbKGkgKyAxKSAlIHNnX2xlbl0ucHRjZDsKKworCQlpZiAoZnNsX2NoYW4t PmZzYy5kaXIgPT0gRE1BX01FTV9UT19ERVYpIHsKKwkJCXNyY19hZGRyID0gZG1hX2J1Zl9uZXh0 OworCQkJZHN0X2FkZHIgPSBmc2xfY2hhbi0+ZnNjLmRldl9hZGRyOworCQkJc29mZiA9IGZzbF9j aGFuLT5mc2MuYWRkcl93aWR0aDsKKwkJCWRvZmYgPSAwOworCQl9IGVsc2UgeworCQkJc3JjX2Fk ZHIgPSBmc2xfY2hhbi0+ZnNjLmRldl9hZGRyOworCQkJZHN0X2FkZHIgPSBkbWFfYnVmX25leHQ7 CisJCQlzb2ZmID0gMDsKKwkJCWRvZmYgPSBmc2xfY2hhbi0+ZnNjLmFkZHJfd2lkdGg7CisJCX0K KworCQlmc2xfZWRtYV9maWxsX3RjZChmc2xfZGVzYy0+dGNkW2ldLnZ0Y2QsIHNyY19hZGRyLCBk c3RfYWRkciwKKwkJCQkgIGZzbF9jaGFuLT5mc2MuYXR0ciwgc29mZiwgbmJ5dGVzLCAwLCBpdGVy LAorCQkJCSAgaXRlciwgZG9mZiwgbGFzdF9zZywgdHJ1ZSwgZmFsc2UsIHRydWUpOworCQlkbWFf YnVmX25leHQgKz0gcGVyaW9kX2xlbjsKKwl9CisKKwlyZXR1cm4gdmNoYW5fdHhfcHJlcCgmZnNs X2NoYW4tPnZjaGFuLCAmZnNsX2Rlc2MtPnZkZXNjLCBmbGFncyk7Cit9CitFWFBPUlRfU1lNQk9M X0dQTChmc2xfZWRtYV9wcmVwX2RtYV9jeWNsaWMpOworCitzdHJ1Y3QgZG1hX2FzeW5jX3R4X2Rl c2NyaXB0b3IgKmZzbF9lZG1hX3ByZXBfc2xhdmVfc2coCisJCXN0cnVjdCBkbWFfY2hhbiAqY2hh biwgc3RydWN0IHNjYXR0ZXJsaXN0ICpzZ2wsCisJCXVuc2lnbmVkIGludCBzZ19sZW4sIGVudW0g ZG1hX3RyYW5zZmVyX2RpcmVjdGlvbiBkaXJlY3Rpb24sCisJCXVuc2lnbmVkIGxvbmcgZmxhZ3Ms IHZvaWQgKmNvbnRleHQpCit7CisJc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9jaGFuID0gdG9f ZnNsX2VkbWFfY2hhbihjaGFuKTsKKwlzdHJ1Y3QgZnNsX2VkbWFfZGVzYyAqZnNsX2Rlc2M7CisJ c3RydWN0IHNjYXR0ZXJsaXN0ICpzZzsKKwl1MzIgc3JjX2FkZHIsIGRzdF9hZGRyLCBsYXN0X3Nn LCBuYnl0ZXM7CisJdTE2IHNvZmYsIGRvZmYsIGl0ZXI7CisJaW50IGk7CisKKwlpZiAoIWlzX3Ns YXZlX2RpcmVjdGlvbihmc2xfY2hhbi0+ZnNjLmRpcikpCisJCXJldHVybiBOVUxMOworCisJZnNs X2Rlc2MgPSBmc2xfZWRtYV9hbGxvY19kZXNjKGZzbF9jaGFuLCBzZ19sZW4pOworCWlmICghZnNs X2Rlc2MpCisJCXJldHVybiBOVUxMOworCWZzbF9kZXNjLT5pc2N5Y2xpYyA9IGZhbHNlOworCisJ bmJ5dGVzID0gZnNsX2NoYW4tPmZzYy5hZGRyX3dpZHRoICogZnNsX2NoYW4tPmZzYy5idXJzdDsK Kwlmb3JfZWFjaF9zZyhzZ2wsIHNnLCBzZ19sZW4sIGkpIHsKKwkJLyogZ2V0IG5leHQgc2cncyBw aHlzaWNhbCBhZGRyZXNzICovCisJCWxhc3Rfc2cgPSBmc2xfZGVzYy0+dGNkWyhpICsgMSkgJSBz Z19sZW5dLnB0Y2Q7CisKKwkJaWYgKGZzbF9jaGFuLT5mc2MuZGlyID09IERNQV9NRU1fVE9fREVW KSB7CisJCQlzcmNfYWRkciA9IHNnX2RtYV9hZGRyZXNzKHNnKTsKKwkJCWRzdF9hZGRyID0gZnNs X2NoYW4tPmZzYy5kZXZfYWRkcjsKKwkJCXNvZmYgPSBmc2xfY2hhbi0+ZnNjLmFkZHJfd2lkdGg7 CisJCQlkb2ZmID0gMDsKKwkJfSBlbHNlIHsKKwkJCXNyY19hZGRyID0gZnNsX2NoYW4tPmZzYy5k ZXZfYWRkcjsKKwkJCWRzdF9hZGRyID0gc2dfZG1hX2FkZHJlc3Moc2cpOworCQkJc29mZiA9IDA7 CisJCQlkb2ZmID0gZnNsX2NoYW4tPmZzYy5hZGRyX3dpZHRoOworCQl9CisKKwkJaXRlciA9IHNn X2RtYV9sZW4oc2cpIC8gbmJ5dGVzOworCQlpZiAoaSA8IHNnX2xlbiAtIDEpIHsKKwkJCWxhc3Rf c2cgPSBmc2xfZGVzYy0+dGNkWyhpICsgMSldLnB0Y2Q7CisJCQlmc2xfZWRtYV9maWxsX3RjZChm c2xfZGVzYy0+dGNkW2ldLnZ0Y2QsIHNyY19hZGRyLAorCQkJCQkgIGRzdF9hZGRyLCBmc2xfY2hh bi0+ZnNjLmF0dHIsIHNvZmYsCisJCQkJCSAgbmJ5dGVzLCAwLCBpdGVyLCBpdGVyLCBkb2ZmLCBs YXN0X3NnLAorCQkJCQkgIGZhbHNlLCBmYWxzZSwgdHJ1ZSk7CisJCX0gZWxzZSB7CisJCQlsYXN0 X3NnID0gMDsKKwkJCWZzbF9lZG1hX2ZpbGxfdGNkKGZzbF9kZXNjLT50Y2RbaV0udnRjZCwgc3Jj X2FkZHIsCisJCQkJCSAgZHN0X2FkZHIsIGZzbF9jaGFuLT5mc2MuYXR0ciwgc29mZiwKKwkJCQkJ ICBuYnl0ZXMsIDAsIGl0ZXIsIGl0ZXIsIGRvZmYsIGxhc3Rfc2csCisJCQkJCSAgdHJ1ZSwgdHJ1 ZSwgZmFsc2UpOworCQl9CisJfQorCisJcmV0dXJuIHZjaGFuX3R4X3ByZXAoJmZzbF9jaGFuLT52 Y2hhbiwgJmZzbF9kZXNjLT52ZGVzYywgZmxhZ3MpOworfQorRVhQT1JUX1NZTUJPTF9HUEwoZnNs X2VkbWFfcHJlcF9zbGF2ZV9zZyk7CisKK3ZvaWQgZnNsX2VkbWFfeGZlcl9kZXNjKHN0cnVjdCBm c2xfZWRtYV9jaGFuICpmc2xfY2hhbikKK3sKKwlzdHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmRlc2M7 CisKKwl2ZGVzYyA9IHZjaGFuX25leHRfZGVzYygmZnNsX2NoYW4tPnZjaGFuKTsKKwlpZiAoIXZk ZXNjKQorCQlyZXR1cm47CisJZnNsX2NoYW4tPmVkZXNjID0gdG9fZnNsX2VkbWFfZGVzYyh2ZGVz Yyk7CisJZnNsX2VkbWFfc2V0X3RjZF9yZWdzKGZzbF9jaGFuLCBmc2xfY2hhbi0+ZWRlc2MtPnRj ZFswXS52dGNkKTsKKwlmc2xfZWRtYV9lbmFibGVfcmVxdWVzdChmc2xfY2hhbik7CisJZnNsX2No YW4tPnN0YXR1cyA9IERNQV9JTl9QUk9HUkVTUzsKKwlmc2xfY2hhbi0+aWRsZSA9IGZhbHNlOwor fQorRVhQT1JUX1NZTUJPTF9HUEwoZnNsX2VkbWFfeGZlcl9kZXNjKTsKKwordm9pZCBmc2xfZWRt YV9pc3N1ZV9wZW5kaW5nKHN0cnVjdCBkbWFfY2hhbiAqY2hhbikKK3sKKwlzdHJ1Y3QgZnNsX2Vk bWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOworCXVuc2lnbmVkIGxv bmcgZmxhZ3M7CisKKwlzcGluX2xvY2tfaXJxc2F2ZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZs YWdzKTsKKworCWlmICh1bmxpa2VseShmc2xfY2hhbi0+cG1fc3RhdGUgIT0gUlVOTklORykpIHsK KwkJc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsK KwkJLyogY2Fubm90IHN1Ym1pdCBkdWUgdG8gc3VzcGVuZCAqLworCQlyZXR1cm47CisJfQorCisJ aWYgKHZjaGFuX2lzc3VlX3BlbmRpbmcoJmZzbF9jaGFuLT52Y2hhbikgJiYgIWZzbF9jaGFuLT5l ZGVzYykKKwkJZnNsX2VkbWFfeGZlcl9kZXNjKGZzbF9jaGFuKTsKKworCXNwaW5fdW5sb2NrX2ly cXJlc3RvcmUoJmZzbF9jaGFuLT52Y2hhbi5sb2NrLCBmbGFncyk7Cit9CitFWFBPUlRfU1lNQk9M X0dQTChmc2xfZWRtYV9pc3N1ZV9wZW5kaW5nKTsKKworaW50IGZzbF9lZG1hX2FsbG9jX2NoYW5f cmVzb3VyY2VzKHN0cnVjdCBkbWFfY2hhbiAqY2hhbikKK3sKKwlzdHJ1Y3QgZnNsX2VkbWFfY2hh biAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOworCisJZnNsX2NoYW4tPnRjZF9w b29sID0gZG1hX3Bvb2xfY3JlYXRlKCJ0Y2RfcG9vbCIsIGNoYW4tPmRldmljZS0+ZGV2LAorCQkJ CXNpemVvZihzdHJ1Y3QgZnNsX2VkbWFfaHdfdGNkKSwKKwkJCQkzMiwgMCk7CisJcmV0dXJuIDA7 Cit9CitFWFBPUlRfU1lNQk9MX0dQTChmc2xfZWRtYV9hbGxvY19jaGFuX3Jlc291cmNlcyk7CisK K3ZvaWQgZnNsX2VkbWFfZnJlZV9jaGFuX3Jlc291cmNlcyhzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4p Cit7CisJc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9jaGFuID0gdG9fZnNsX2VkbWFfY2hhbihj aGFuKTsKKwl1bnNpZ25lZCBsb25nIGZsYWdzOworCUxJU1RfSEVBRChoZWFkKTsKKworCXNwaW5f bG9ja19pcnFzYXZlKCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOworCWZzbF9lZG1hX2Rp c2FibGVfcmVxdWVzdChmc2xfY2hhbik7CisJZnNsX2VkbWFfY2hhbl9tdXgoZnNsX2NoYW4sIDAs IGZhbHNlKTsKKwlmc2xfY2hhbi0+ZWRlc2MgPSBOVUxMOworCXZjaGFuX2dldF9hbGxfZGVzY3Jp cHRvcnMoJmZzbF9jaGFuLT52Y2hhbiwgJmhlYWQpOworCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUo JmZzbF9jaGFuLT52Y2hhbi5sb2NrLCBmbGFncyk7CisKKwl2Y2hhbl9kbWFfZGVzY19mcmVlX2xp c3QoJmZzbF9jaGFuLT52Y2hhbiwgJmhlYWQpOworCWRtYV9wb29sX2Rlc3Ryb3koZnNsX2NoYW4t PnRjZF9wb29sKTsKKwlmc2xfY2hhbi0+dGNkX3Bvb2wgPSBOVUxMOworfQorRVhQT1JUX1NZTUJP TF9HUEwoZnNsX2VkbWFfZnJlZV9jaGFuX3Jlc291cmNlcyk7CisKK3ZvaWQgZnNsX2VkbWFfY2xl YW51cF92Y2hhbihzdHJ1Y3QgZG1hX2RldmljZSAqZG1hZGV2KQoreworCXN0cnVjdCBmc2xfZWRt YV9jaGFuICpjaGFuLCAqX2NoYW47CisKKwlsaXN0X2Zvcl9lYWNoX2VudHJ5X3NhZmUoY2hhbiwg X2NoYW4sCisJCQkJJmRtYWRldi0+Y2hhbm5lbHMsIHZjaGFuLmNoYW4uZGV2aWNlX25vZGUpIHsK KwkJbGlzdF9kZWwoJmNoYW4tPnZjaGFuLmNoYW4uZGV2aWNlX25vZGUpOworCQl0YXNrbGV0X2tp bGwoJmNoYW4tPnZjaGFuLnRhc2spOworCX0KK30KK0VYUE9SVF9TWU1CT0xfR1BMKGZzbF9lZG1h X2NsZWFudXBfdmNoYW4pOworCitNT0RVTEVfTElDRU5TRSgiR1BMIHYyIik7CmRpZmYgLS1naXQg YS9kcml2ZXJzL2RtYS9mc2wtZWRtYS1jb21tb24uaCBiL2RyaXZlcnMvZG1hL2ZzbC1lZG1hLWNv bW1vbi5oCm5ldyBmaWxlIG1vZGUgMTAwNjQ0CmluZGV4IDAwMDAwMDAwMDAwMC4uZjNlYTY4ZTE1 YjIzCi0tLSAvZGV2L251bGwKKysrIGIvZHJpdmVycy9kbWEvZnNsLWVkbWEtY29tbW9uLmgKQEAg LTAsMCArMSwxOTYgQEAKKy8qIFNQRFgtTGljZW5zZS1JZGVudGlmaWVyOiBHUEwtMi4wICovCisv KgorICogQ29weXJpZ2h0IDIwMTMtMjAxNCBGcmVlc2NhbGUgU2VtaWNvbmR1Y3RvciwgSW5jLgor ICogQ29weXJpZ2h0IDIwMTggQW5nZWxvIER1cmVnaGVsbG8gPGFuZ2Vsb0BzeXNhbS5pdD4KKyAq LworI2lmbmRlZiBfRlNMX0VETUFfQ09NTU9OX0hfCisjZGVmaW5lIF9GU0xfRURNQV9DT01NT05f SF8KKworI2luY2x1ZGUgInZpcnQtZG1hLmgiCisKKyNkZWZpbmUgRURNQV9DUgkJCTB4MDAKKyNk ZWZpbmUgRURNQV9FUwkJCTB4MDQKKyNkZWZpbmUgRURNQV9FUlEJCTB4MEMKKyNkZWZpbmUgRURN QV9FRUkJCTB4MTQKKyNkZWZpbmUgRURNQV9TRVJRCQkweDFCCisjZGVmaW5lIEVETUFfQ0VSUQkJ MHgxQQorI2RlZmluZSBFRE1BX1NFRUkJCTB4MTkKKyNkZWZpbmUgRURNQV9DRUVJCQkweDE4Cisj ZGVmaW5lIEVETUFfQ0lOVAkJMHgxRgorI2RlZmluZSBFRE1BX0NFUlIJCTB4MUUKKyNkZWZpbmUg RURNQV9TU1JUCQkweDFECisjZGVmaW5lIEVETUFfQ0RORQkJMHgxQworI2RlZmluZSBFRE1BX0lO VFIJCTB4MjQKKyNkZWZpbmUgRURNQV9FUlIJCTB4MkMKKworI2RlZmluZSBFRE1BX1RDRF9TQURE Uih4KQkoMHgxMDAwICsgMzIgKiAoeCkpCisjZGVmaW5lIEVETUFfVENEX1NPRkYoeCkJKDB4MTAw NCArIDMyICogKHgpKQorI2RlZmluZSBFRE1BX1RDRF9BVFRSKHgpCSgweDEwMDYgKyAzMiAqICh4 KSkKKyNkZWZpbmUgRURNQV9UQ0RfTkJZVEVTKHgpCSgweDEwMDggKyAzMiAqICh4KSkKKyNkZWZp bmUgRURNQV9UQ0RfU0xBU1QoeCkJKDB4MTAwQyArIDMyICogKHgpKQorI2RlZmluZSBFRE1BX1RD RF9EQUREUih4KQkoMHgxMDEwICsgMzIgKiAoeCkpCisjZGVmaW5lIEVETUFfVENEX0RPRkYoeCkJ KDB4MTAxNCArIDMyICogKHgpKQorI2RlZmluZSBFRE1BX1RDRF9DSVRFUl9FTElOSyh4KQkoMHgx MDE2ICsgMzIgKiAoeCkpCisjZGVmaW5lIEVETUFfVENEX0NJVEVSKHgpCSgweDEwMTYgKyAzMiAq ICh4KSkKKyNkZWZpbmUgRURNQV9UQ0RfRExBU1RfU0dBKHgpCSgweDEwMTggKyAzMiAqICh4KSkK KyNkZWZpbmUgRURNQV9UQ0RfQ1NSKHgpCQkoMHgxMDFDICsgMzIgKiAoeCkpCisjZGVmaW5lIEVE TUFfVENEX0JJVEVSX0VMSU5LKHgpCSgweDEwMUUgKyAzMiAqICh4KSkKKyNkZWZpbmUgRURNQV9U Q0RfQklURVIoeCkJKDB4MTAxRSArIDMyICogKHgpKQorCisjZGVmaW5lIEVETUFfQ1JfRURCRwkJ QklUKDEpCisjZGVmaW5lIEVETUFfQ1JfRVJDQQkJQklUKDIpCisjZGVmaW5lIEVETUFfQ1JfRVJH QQkJQklUKDMpCisjZGVmaW5lIEVETUFfQ1JfSE9FCQlCSVQoNCkKKyNkZWZpbmUgRURNQV9DUl9I QUxUCQlCSVQoNSkKKyNkZWZpbmUgRURNQV9DUl9DTE0JCUJJVCg2KQorI2RlZmluZSBFRE1BX0NS X0VNTE0JCUJJVCg3KQorI2RlZmluZSBFRE1BX0NSX0VDWAkJQklUKDE2KQorI2RlZmluZSBFRE1B X0NSX0NYCQlCSVQoMTcpCisKKyNkZWZpbmUgRURNQV9TRUVJX1NFRUkoeCkJKCh4KSAmIDB4MUYp CisjZGVmaW5lIEVETUFfQ0VFSV9DRUVJKHgpCSgoeCkgJiAweDFGKQorI2RlZmluZSBFRE1BX0NJ TlRfQ0lOVCh4KQkoKHgpICYgMHgxRikKKyNkZWZpbmUgRURNQV9DRVJSX0NFUlIoeCkJKCh4KSAm IDB4MUYpCisKKyNkZWZpbmUgRURNQV9UQ0RfQVRUUl9EU0laRSh4KQkJKCgoeCkgJiAweDAwMDcp KQorI2RlZmluZSBFRE1BX1RDRF9BVFRSX0RNT0QoeCkJCSgoKHgpICYgMHgwMDFGKSA8PCAzKQor I2RlZmluZSBFRE1BX1RDRF9BVFRSX1NTSVpFKHgpCQkoKCh4KSAmIDB4MDAwNykgPDwgOCkKKyNk ZWZpbmUgRURNQV9UQ0RfQVRUUl9TTU9EKHgpCQkoKCh4KSAmIDB4MDAxRikgPDwgMTEpCisjZGVm aW5lIEVETUFfVENEX0FUVFJfU1NJWkVfOEJJVAkoMHgwMDAwKQorI2RlZmluZSBFRE1BX1RDRF9B VFRSX1NTSVpFXzE2QklUCSgweDAxMDApCisjZGVmaW5lIEVETUFfVENEX0FUVFJfU1NJWkVfMzJC SVQJKDB4MDIwMCkKKyNkZWZpbmUgRURNQV9UQ0RfQVRUUl9TU0laRV82NEJJVAkoMHgwMzAwKQor I2RlZmluZSBFRE1BX1RDRF9BVFRSX1NTSVpFXzMyQllURQkoMHgwNTAwKQorI2RlZmluZSBFRE1B X1RDRF9BVFRSX0RTSVpFXzhCSVQJKDB4MDAwMCkKKyNkZWZpbmUgRURNQV9UQ0RfQVRUUl9EU0la RV8xNkJJVAkoMHgwMDAxKQorI2RlZmluZSBFRE1BX1RDRF9BVFRSX0RTSVpFXzMyQklUCSgweDAw MDIpCisjZGVmaW5lIEVETUFfVENEX0FUVFJfRFNJWkVfNjRCSVQJKDB4MDAwMykKKyNkZWZpbmUg RURNQV9UQ0RfQVRUUl9EU0laRV8zMkJZVEUJKDB4MDAwNSkKKworI2RlZmluZSBFRE1BX1RDRF9T T0ZGX1NPRkYoeCkJCSh4KQorI2RlZmluZSBFRE1BX1RDRF9OQllURVNfTkJZVEVTKHgpCSh4KQor I2RlZmluZSBFRE1BX1RDRF9TTEFTVF9TTEFTVCh4KQkJKHgpCisjZGVmaW5lIEVETUFfVENEX0RB RERSX0RBRERSKHgpCQkoeCkKKyNkZWZpbmUgRURNQV9UQ0RfQ0lURVJfQ0lURVIoeCkJCSgoeCkg JiAweDdGRkYpCisjZGVmaW5lIEVETUFfVENEX0RPRkZfRE9GRih4KQkJKHgpCisjZGVmaW5lIEVE TUFfVENEX0RMQVNUX1NHQV9ETEFTVF9TR0EoeCkJKHgpCisjZGVmaW5lIEVETUFfVENEX0JJVEVS X0JJVEVSKHgpCQkoKHgpICYgMHg3RkZGKQorCisjZGVmaW5lIEVETUFfVENEX0NTUl9TVEFSVAkJ QklUKDApCisjZGVmaW5lIEVETUFfVENEX0NTUl9JTlRfTUFKT1IJCUJJVCgxKQorI2RlZmluZSBF RE1BX1RDRF9DU1JfSU5UX0hBTEYJCUJJVCgyKQorI2RlZmluZSBFRE1BX1RDRF9DU1JfRF9SRVEJ CUJJVCgzKQorI2RlZmluZSBFRE1BX1RDRF9DU1JfRV9TRwkJQklUKDQpCisjZGVmaW5lIEVETUFf VENEX0NTUl9FX0xJTksJCUJJVCg1KQorI2RlZmluZSBFRE1BX1RDRF9DU1JfQUNUSVZFCQlCSVQo NikKKyNkZWZpbmUgRURNQV9UQ0RfQ1NSX0RPTkUJCUJJVCg3KQorCisjZGVmaW5lIEVETUFNVVhf Q0hDRkdfRElTCQkweDAKKyNkZWZpbmUgRURNQU1VWF9DSENGR19FTkJMCQkweDgwCisjZGVmaW5l IEVETUFNVVhfQ0hDRkdfU09VUkNFKG4pCQkoKG4pICYgMHgzRikKKworI2RlZmluZSBETUFNVVhf TlIJMgorCisjZGVmaW5lIEZTTF9FRE1BX0JVU1dJRFRIUwkoQklUKERNQV9TTEFWRV9CVVNXSURU SF8xX0JZVEUpIHwgXAorCQkJCSBCSVQoRE1BX1NMQVZFX0JVU1dJRFRIXzJfQllURVMpIHwgXAor CQkJCSBCSVQoRE1BX1NMQVZFX0JVU1dJRFRIXzRfQllURVMpIHwgXAorCQkJCSBCSVQoRE1BX1NM QVZFX0JVU1dJRFRIXzhfQllURVMpKQorZW51bSBmc2xfZWRtYV9wbV9zdGF0ZSB7CisJUlVOTklO RyA9IDAsCisJU1VTUEVOREVELAorfTsKKworc3RydWN0IGZzbF9lZG1hX2h3X3RjZCB7CisJX19s ZTMyCXNhZGRyOworCV9fbGUxNglzb2ZmOworCV9fbGUxNglhdHRyOworCV9fbGUzMgluYnl0ZXM7 CisJX19sZTMyCXNsYXN0OworCV9fbGUzMglkYWRkcjsKKwlfX2xlMTYJZG9mZjsKKwlfX2xlMTYJ Y2l0ZXI7CisJX19sZTMyCWRsYXN0X3NnYTsKKwlfX2xlMTYJY3NyOworCV9fbGUxNgliaXRlcjsK K307CisKK3N0cnVjdCBmc2xfZWRtYV9zd190Y2QgeworCWRtYV9hZGRyX3QJCQlwdGNkOworCXN0 cnVjdCBmc2xfZWRtYV9od190Y2QJCSp2dGNkOworfTsKKworc3RydWN0IGZzbF9lZG1hX3NsYXZl X2NvbmZpZyB7CisJZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uCWRpcjsKKwllbnVtIGRtYV9z bGF2ZV9idXN3aWR0aAkJYWRkcl93aWR0aDsKKwl1MzIJCQkJZGV2X2FkZHI7CisJdTMyCQkJCWJ1 cnN0OworCXUzMgkJCQlhdHRyOworfTsKKworc3RydWN0IGZzbF9lZG1hX2NoYW4geworCXN0cnVj dCB2aXJ0X2RtYV9jaGFuCQl2Y2hhbjsKKwllbnVtIGRtYV9zdGF0dXMJCQlzdGF0dXM7CisJZW51 bSBmc2xfZWRtYV9wbV9zdGF0ZQkJcG1fc3RhdGU7CisJYm9vbAkJCQlpZGxlOworCXUzMgkJCQlz bGF2ZV9pZDsKKwlzdHJ1Y3QgZnNsX2VkbWFfZW5naW5lCQkqZWRtYTsKKwlzdHJ1Y3QgZnNsX2Vk bWFfZGVzYwkJKmVkZXNjOworCXN0cnVjdCBmc2xfZWRtYV9zbGF2ZV9jb25maWcJZnNjOworCXN0 cnVjdCBkbWFfcG9vbAkJCSp0Y2RfcG9vbDsKK307CisKK3N0cnVjdCBmc2xfZWRtYV9kZXNjIHsK KwlzdHJ1Y3QgdmlydF9kbWFfZGVzYwkJdmRlc2M7CisJc3RydWN0IGZzbF9lZG1hX2NoYW4JCSpl Y2hhbjsKKwlib29sCQkJCWlzY3ljbGljOworCXVuc2lnbmVkIGludAkJCW5fdGNkczsKKwlzdHJ1 Y3QgZnNsX2VkbWFfc3dfdGNkCQl0Y2RbXTsKK307CisKK3N0cnVjdCBmc2xfZWRtYV9lbmdpbmUg eworCXN0cnVjdCBkbWFfZGV2aWNlCWRtYV9kZXY7CisJdm9pZCBfX2lvbWVtCQkqbWVtYmFzZTsK Kwl2b2lkIF9faW9tZW0JCSptdXhiYXNlW0RNQU1VWF9OUl07CisJc3RydWN0IGNsawkJKm11eGNs a1tETUFNVVhfTlJdOworCXN0cnVjdCBtdXRleAkJZnNsX2VkbWFfbXV0ZXg7CisJdTMyCQkJbl9j aGFuczsKKwlpbnQJCQl0eGlycTsKKwlpbnQJCQllcnJpcnE7CisJYm9vbAkJCWJpZ19lbmRpYW47 CisJc3RydWN0IGZzbF9lZG1hX2NoYW4JY2hhbnNbXTsKK307CisKK3UzMiBlZG1hX3JlYWRsKHN0 cnVjdCBmc2xfZWRtYV9lbmdpbmUgKmVkbWEsIHZvaWQgX19pb21lbSAqYWRkcik7Cit2b2lkIGVk bWFfd3JpdGViKHN0cnVjdCBmc2xfZWRtYV9lbmdpbmUgKmVkbWEsIHU4IHZhbCwgdm9pZCBfX2lv bWVtICphZGRyKTsKK3ZvaWQgZWRtYV93cml0ZXcoc3RydWN0IGZzbF9lZG1hX2VuZ2luZSAqZWRt YSwgdTE2IHZhbCwgdm9pZCBfX2lvbWVtICphZGRyKTsKK3ZvaWQgZWRtYV93cml0ZWwoc3RydWN0 IGZzbF9lZG1hX2VuZ2luZSAqZWRtYSwgdTMyIHZhbCwgdm9pZCBfX2lvbWVtICphZGRyKTsKKwor c3RydWN0IGZzbF9lZG1hX2NoYW4gKnRvX2ZzbF9lZG1hX2NoYW4oc3RydWN0IGRtYV9jaGFuICpj aGFuKTsKK3N0cnVjdCBmc2xfZWRtYV9kZXNjICp0b19mc2xfZWRtYV9kZXNjKHN0cnVjdCB2aXJ0 X2RtYV9kZXNjICp2ZCk7CisKK3ZvaWQgZnNsX2VkbWFfZGlzYWJsZV9yZXF1ZXN0KHN0cnVjdCBm c2xfZWRtYV9jaGFuICpmc2xfY2hhbik7Cit2b2lkIGZzbF9lZG1hX2NoYW5fbXV4KHN0cnVjdCBm c2xfZWRtYV9jaGFuICpmc2xfY2hhbiwKKwkJCXVuc2lnbmVkIGludCBzbG90LCBib29sIGVuYWJs ZSk7Cit2b2lkIGZzbF9lZG1hX2ZyZWVfZGVzYyhzdHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmRlc2Mp OworaW50IGZzbF9lZG1hX3Rlcm1pbmF0ZV9hbGwoc3RydWN0IGRtYV9jaGFuICpjaGFuKTsKK2lu dCBmc2xfZWRtYV9wYXVzZShzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4pOworaW50IGZzbF9lZG1hX3Jl c3VtZShzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4pOworaW50IGZzbF9lZG1hX3NsYXZlX2NvbmZpZyhz dHJ1Y3QgZG1hX2NoYW4gKmNoYW4sCisJCQkJIHN0cnVjdCBkbWFfc2xhdmVfY29uZmlnICpjZmcp OworZW51bSBkbWFfc3RhdHVzIGZzbF9lZG1hX3R4X3N0YXR1cyhzdHJ1Y3QgZG1hX2NoYW4gKmNo YW4sCisJCWRtYV9jb29raWVfdCBjb29raWUsIHN0cnVjdCBkbWFfdHhfc3RhdGUgKnR4c3RhdGUp Oworc3RydWN0IGRtYV9hc3luY190eF9kZXNjcmlwdG9yICpmc2xfZWRtYV9wcmVwX2RtYV9jeWNs aWMoCisJCXN0cnVjdCBkbWFfY2hhbiAqY2hhbiwgZG1hX2FkZHJfdCBkbWFfYWRkciwgc2l6ZV90 IGJ1Zl9sZW4sCisJCXNpemVfdCBwZXJpb2RfbGVuLCBlbnVtIGRtYV90cmFuc2Zlcl9kaXJlY3Rp b24gZGlyZWN0aW9uLAorCQl1bnNpZ25lZCBsb25nIGZsYWdzKTsKK3N0cnVjdCBkbWFfYXN5bmNf dHhfZGVzY3JpcHRvciAqZnNsX2VkbWFfcHJlcF9zbGF2ZV9zZygKKwkJc3RydWN0IGRtYV9jaGFu ICpjaGFuLCBzdHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnbCwKKwkJdW5zaWduZWQgaW50IHNnX2xlbiwg ZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uIGRpcmVjdGlvbiwKKwkJdW5zaWduZWQgbG9uZyBm bGFncywgdm9pZCAqY29udGV4dCk7Cit2b2lkIGZzbF9lZG1hX3hmZXJfZGVzYyhzdHJ1Y3QgZnNs X2VkbWFfY2hhbiAqZnNsX2NoYW4pOwordm9pZCBmc2xfZWRtYV9pc3N1ZV9wZW5kaW5nKHN0cnVj dCBkbWFfY2hhbiAqY2hhbik7CitpbnQgZnNsX2VkbWFfYWxsb2NfY2hhbl9yZXNvdXJjZXMoc3Ry dWN0IGRtYV9jaGFuICpjaGFuKTsKK3ZvaWQgZnNsX2VkbWFfZnJlZV9jaGFuX3Jlc291cmNlcyhz dHJ1Y3QgZG1hX2NoYW4gKmNoYW4pOwordm9pZCBmc2xfZWRtYV9jbGVhbnVwX3ZjaGFuKHN0cnVj dCBkbWFfZGV2aWNlICpkbWFkZXYpOworCisjZW5kaWYgLyogX0ZTTF9FRE1BX0NPTU1PTl9IXyAq LwpkaWZmIC0tZ2l0IGEvZHJpdmVycy9kbWEvZnNsLWVkbWEuYyBiL2RyaXZlcnMvZG1hL2ZzbC1l ZG1hLmMKaW5kZXggYzc1Njg4NjkyODRlLi4zMzdhZDBjMGVhNzggMTAwNjQ0Ci0tLSBhL2RyaXZl cnMvZG1hL2ZzbC1lZG1hLmMKKysrIGIvZHJpdmVycy9kbWEvZnNsLWVkbWEuYwpAQCAtMTMsNjUz ICsxMywxNiBAQAogICogb3B0aW9uKSBhbnkgbGF0ZXIgdmVyc2lvbi4KICAqLwogCi0jaW5jbHVk ZSA8bGludXgvaW5pdC5oPgogI2luY2x1ZGUgPGxpbnV4L21vZHVsZS5oPgogI2luY2x1ZGUgPGxp bnV4L2ludGVycnVwdC5oPgogI2luY2x1ZGUgPGxpbnV4L2Nsay5oPgotI2luY2x1ZGUgPGxpbnV4 L2RtYS1tYXBwaW5nLmg+Ci0jaW5jbHVkZSA8bGludXgvZG1hcG9vbC5oPgotI2luY2x1ZGUgPGxp bnV4L3NsYWIuaD4KLSNpbmNsdWRlIDxsaW51eC9zcGlubG9jay5oPgogI2luY2x1ZGUgPGxpbnV4 L29mLmg+CiAjaW5jbHVkZSA8bGludXgvb2ZfZGV2aWNlLmg+CiAjaW5jbHVkZSA8bGludXgvb2Zf YWRkcmVzcy5oPgogI2luY2x1ZGUgPGxpbnV4L29mX2lycS5oPgogI2luY2x1ZGUgPGxpbnV4L29m X2RtYS5oPgogCi0jaW5jbHVkZSAidmlydC1kbWEuaCIKLQotI2RlZmluZSBFRE1BX0NSCQkJMHgw MAotI2RlZmluZSBFRE1BX0VTCQkJMHgwNAotI2RlZmluZSBFRE1BX0VSUQkJMHgwQwotI2RlZmlu ZSBFRE1BX0VFSQkJMHgxNAotI2RlZmluZSBFRE1BX1NFUlEJCTB4MUIKLSNkZWZpbmUgRURNQV9D RVJRCQkweDFBCi0jZGVmaW5lIEVETUFfU0VFSQkJMHgxOQotI2RlZmluZSBFRE1BX0NFRUkJCTB4 MTgKLSNkZWZpbmUgRURNQV9DSU5UCQkweDFGCi0jZGVmaW5lIEVETUFfQ0VSUgkJMHgxRQotI2Rl ZmluZSBFRE1BX1NTUlQJCTB4MUQKLSNkZWZpbmUgRURNQV9DRE5FCQkweDFDCi0jZGVmaW5lIEVE TUFfSU5UUgkJMHgyNAotI2RlZmluZSBFRE1BX0VSUgkJMHgyQwotCi0jZGVmaW5lIEVETUFfVENE X1NBRERSKHgpCSgweDEwMDAgKyAzMiAqICh4KSkKLSNkZWZpbmUgRURNQV9UQ0RfU09GRih4KQko MHgxMDA0ICsgMzIgKiAoeCkpCi0jZGVmaW5lIEVETUFfVENEX0FUVFIoeCkJKDB4MTAwNiArIDMy ICogKHgpKQotI2RlZmluZSBFRE1BX1RDRF9OQllURVMoeCkJKDB4MTAwOCArIDMyICogKHgpKQot I2RlZmluZSBFRE1BX1RDRF9TTEFTVCh4KQkoMHgxMDBDICsgMzIgKiAoeCkpCi0jZGVmaW5lIEVE TUFfVENEX0RBRERSKHgpCSgweDEwMTAgKyAzMiAqICh4KSkKLSNkZWZpbmUgRURNQV9UQ0RfRE9G Rih4KQkoMHgxMDE0ICsgMzIgKiAoeCkpCi0jZGVmaW5lIEVETUFfVENEX0NJVEVSX0VMSU5LKHgp CSgweDEwMTYgKyAzMiAqICh4KSkKLSNkZWZpbmUgRURNQV9UQ0RfQ0lURVIoeCkJKDB4MTAxNiAr IDMyICogKHgpKQotI2RlZmluZSBFRE1BX1RDRF9ETEFTVF9TR0EoeCkJKDB4MTAxOCArIDMyICog KHgpKQotI2RlZmluZSBFRE1BX1RDRF9DU1IoeCkJCSgweDEwMUMgKyAzMiAqICh4KSkKLSNkZWZp bmUgRURNQV9UQ0RfQklURVJfRUxJTksoeCkJKDB4MTAxRSArIDMyICogKHgpKQotI2RlZmluZSBF RE1BX1RDRF9CSVRFUih4KQkoMHgxMDFFICsgMzIgKiAoeCkpCi0KLSNkZWZpbmUgRURNQV9DUl9F REJHCQlCSVQoMSkKLSNkZWZpbmUgRURNQV9DUl9FUkNBCQlCSVQoMikKLSNkZWZpbmUgRURNQV9D Ul9FUkdBCQlCSVQoMykKLSNkZWZpbmUgRURNQV9DUl9IT0UJCUJJVCg0KQotI2RlZmluZSBFRE1B X0NSX0hBTFQJCUJJVCg1KQotI2RlZmluZSBFRE1BX0NSX0NMTQkJQklUKDYpCi0jZGVmaW5lIEVE TUFfQ1JfRU1MTQkJQklUKDcpCi0jZGVmaW5lIEVETUFfQ1JfRUNYCQlCSVQoMTYpCi0jZGVmaW5l IEVETUFfQ1JfQ1gJCUJJVCgxNykKLQotI2RlZmluZSBFRE1BX1NFRUlfU0VFSSh4KQkoKHgpICYg MHgxRikKLSNkZWZpbmUgRURNQV9DRUVJX0NFRUkoeCkJKCh4KSAmIDB4MUYpCi0jZGVmaW5lIEVE TUFfQ0lOVF9DSU5UKHgpCSgoeCkgJiAweDFGKQotI2RlZmluZSBFRE1BX0NFUlJfQ0VSUih4KQko KHgpICYgMHgxRikKLQotI2RlZmluZSBFRE1BX1RDRF9BVFRSX0RTSVpFKHgpCQkoKCh4KSAmIDB4 MDAwNykpCi0jZGVmaW5lIEVETUFfVENEX0FUVFJfRE1PRCh4KQkJKCgoeCkgJiAweDAwMUYpIDw8 IDMpCi0jZGVmaW5lIEVETUFfVENEX0FUVFJfU1NJWkUoeCkJCSgoKHgpICYgMHgwMDA3KSA8PCA4 KQotI2RlZmluZSBFRE1BX1RDRF9BVFRSX1NNT0QoeCkJCSgoKHgpICYgMHgwMDFGKSA8PCAxMSkK LSNkZWZpbmUgRURNQV9UQ0RfQVRUUl9TU0laRV84QklUCSgweDAwMDApCi0jZGVmaW5lIEVETUFf VENEX0FUVFJfU1NJWkVfMTZCSVQJKDB4MDEwMCkKLSNkZWZpbmUgRURNQV9UQ0RfQVRUUl9TU0la RV8zMkJJVAkoMHgwMjAwKQotI2RlZmluZSBFRE1BX1RDRF9BVFRSX1NTSVpFXzY0QklUCSgweDAz MDApCi0jZGVmaW5lIEVETUFfVENEX0FUVFJfU1NJWkVfMzJCWVRFCSgweDA1MDApCi0jZGVmaW5l IEVETUFfVENEX0FUVFJfRFNJWkVfOEJJVAkoMHgwMDAwKQotI2RlZmluZSBFRE1BX1RDRF9BVFRS X0RTSVpFXzE2QklUCSgweDAwMDEpCi0jZGVmaW5lIEVETUFfVENEX0FUVFJfRFNJWkVfMzJCSVQJ KDB4MDAwMikKLSNkZWZpbmUgRURNQV9UQ0RfQVRUUl9EU0laRV82NEJJVAkoMHgwMDAzKQotI2Rl ZmluZSBFRE1BX1RDRF9BVFRSX0RTSVpFXzMyQllURQkoMHgwMDA1KQotCi0jZGVmaW5lIEVETUFf VENEX1NPRkZfU09GRih4KQkJKHgpCi0jZGVmaW5lIEVETUFfVENEX05CWVRFU19OQllURVMoeCkJ KHgpCi0jZGVmaW5lIEVETUFfVENEX1NMQVNUX1NMQVNUKHgpCQkoeCkKLSNkZWZpbmUgRURNQV9U Q0RfREFERFJfREFERFIoeCkJCSh4KQotI2RlZmluZSBFRE1BX1RDRF9DSVRFUl9DSVRFUih4KQkJ KCh4KSAmIDB4N0ZGRikKLSNkZWZpbmUgRURNQV9UQ0RfRE9GRl9ET0ZGKHgpCQkoeCkKLSNkZWZp bmUgRURNQV9UQ0RfRExBU1RfU0dBX0RMQVNUX1NHQSh4KQkoeCkKLSNkZWZpbmUgRURNQV9UQ0Rf QklURVJfQklURVIoeCkJCSgoeCkgJiAweDdGRkYpCi0KLSNkZWZpbmUgRURNQV9UQ0RfQ1NSX1NU QVJUCQlCSVQoMCkKLSNkZWZpbmUgRURNQV9UQ0RfQ1NSX0lOVF9NQUpPUgkJQklUKDEpCi0jZGVm aW5lIEVETUFfVENEX0NTUl9JTlRfSEFMRgkJQklUKDIpCi0jZGVmaW5lIEVETUFfVENEX0NTUl9E X1JFUQkJQklUKDMpCi0jZGVmaW5lIEVETUFfVENEX0NTUl9FX1NHCQlCSVQoNCkKLSNkZWZpbmUg RURNQV9UQ0RfQ1NSX0VfTElOSwkJQklUKDUpCi0jZGVmaW5lIEVETUFfVENEX0NTUl9BQ1RJVkUJ CUJJVCg2KQotI2RlZmluZSBFRE1BX1RDRF9DU1JfRE9ORQkJQklUKDcpCi0KLSNkZWZpbmUgRURN QU1VWF9DSENGR19ESVMJCTB4MAotI2RlZmluZSBFRE1BTVVYX0NIQ0ZHX0VOQkwJCTB4ODAKLSNk ZWZpbmUgRURNQU1VWF9DSENGR19TT1VSQ0UobikJCSgobikgJiAweDNGKQotCi0jZGVmaW5lIERN QU1VWF9OUgkyCi0KLSNkZWZpbmUgRlNMX0VETUFfQlVTV0lEVEhTCUJJVChETUFfU0xBVkVfQlVT V0lEVEhfMV9CWVRFKSB8IFwKLQkJCQlCSVQoRE1BX1NMQVZFX0JVU1dJRFRIXzJfQllURVMpIHwg XAotCQkJCUJJVChETUFfU0xBVkVfQlVTV0lEVEhfNF9CWVRFUykgfCBcCi0JCQkJQklUKERNQV9T TEFWRV9CVVNXSURUSF84X0JZVEVTKQotZW51bSBmc2xfZWRtYV9wbV9zdGF0ZSB7Ci0JUlVOTklO RyA9IDAsCi0JU1VTUEVOREVELAotfTsKLQotc3RydWN0IGZzbF9lZG1hX2h3X3RjZCB7Ci0JX19s ZTMyCXNhZGRyOwotCV9fbGUxNglzb2ZmOwotCV9fbGUxNglhdHRyOwotCV9fbGUzMgluYnl0ZXM7 Ci0JX19sZTMyCXNsYXN0OwotCV9fbGUzMglkYWRkcjsKLQlfX2xlMTYJZG9mZjsKLQlfX2xlMTYJ Y2l0ZXI7Ci0JX19sZTMyCWRsYXN0X3NnYTsKLQlfX2xlMTYJY3NyOwotCV9fbGUxNgliaXRlcjsK LX07Ci0KLXN0cnVjdCBmc2xfZWRtYV9zd190Y2QgewotCWRtYV9hZGRyX3QJCQlwdGNkOwotCXN0 cnVjdCBmc2xfZWRtYV9od190Y2QJCSp2dGNkOwotfTsKLQotc3RydWN0IGZzbF9lZG1hX3NsYXZl X2NvbmZpZyB7Ci0JZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uCWRpcjsKLQllbnVtIGRtYV9z bGF2ZV9idXN3aWR0aAkJYWRkcl93aWR0aDsKLQl1MzIJCQkJZGV2X2FkZHI7Ci0JdTMyCQkJCWJ1 cnN0OwotCXUzMgkJCQlhdHRyOwotfTsKLQotc3RydWN0IGZzbF9lZG1hX2NoYW4gewotCXN0cnVj dCB2aXJ0X2RtYV9jaGFuCQl2Y2hhbjsKLQllbnVtIGRtYV9zdGF0dXMJCQlzdGF0dXM7Ci0JZW51 bSBmc2xfZWRtYV9wbV9zdGF0ZQkJcG1fc3RhdGU7Ci0JYm9vbAkJCQlpZGxlOwotCXUzMgkJCQlz bGF2ZV9pZDsKLQlzdHJ1Y3QgZnNsX2VkbWFfZW5naW5lCQkqZWRtYTsKLQlzdHJ1Y3QgZnNsX2Vk bWFfZGVzYwkJKmVkZXNjOwotCXN0cnVjdCBmc2xfZWRtYV9zbGF2ZV9jb25maWcJZnNjOwotCXN0 cnVjdCBkbWFfcG9vbAkJCSp0Y2RfcG9vbDsKLX07Ci0KLXN0cnVjdCBmc2xfZWRtYV9kZXNjIHsK LQlzdHJ1Y3QgdmlydF9kbWFfZGVzYwkJdmRlc2M7Ci0Jc3RydWN0IGZzbF9lZG1hX2NoYW4JCSpl Y2hhbjsKLQlib29sCQkJCWlzY3ljbGljOwotCXVuc2lnbmVkIGludAkJCW5fdGNkczsKLQlzdHJ1 Y3QgZnNsX2VkbWFfc3dfdGNkCQl0Y2RbXTsKLX07Ci0KLXN0cnVjdCBmc2xfZWRtYV9lbmdpbmUg ewotCXN0cnVjdCBkbWFfZGV2aWNlCWRtYV9kZXY7Ci0Jdm9pZCBfX2lvbWVtCQkqbWVtYmFzZTsK LQl2b2lkIF9faW9tZW0JCSptdXhiYXNlW0RNQU1VWF9OUl07Ci0Jc3RydWN0IGNsawkJKm11eGNs a1tETUFNVVhfTlJdOwotCXN0cnVjdCBtdXRleAkJZnNsX2VkbWFfbXV0ZXg7Ci0JdTMyCQkJbl9j aGFuczsKLQlpbnQJCQl0eGlycTsKLQlpbnQJCQllcnJpcnE7Ci0JYm9vbAkJCWJpZ19lbmRpYW47 Ci0Jc3RydWN0IGZzbF9lZG1hX2NoYW4JY2hhbnNbXTsKLX07Ci0KLS8qCi0gKiBSL1cgZnVuY3Rp b25zIGZvciBiaWctIG9yIGxpdHRsZS1lbmRpYW4gcmVnaXN0ZXJzOgotICogVGhlIGVETUEgY29u dHJvbGxlcidzIGVuZGlhbiBpcyBpbmRlcGVuZGVudCBvZiB0aGUgQ1BVIGNvcmUncyBlbmRpYW4u Ci0gKiBGb3IgdGhlIGJpZy1lbmRpYW4gSVAgbW9kdWxlLCB0aGUgb2Zmc2V0IGZvciA4LWJpdCBv ciAxNi1iaXQgcmVnaXN0ZXJzCi0gKiBzaG91bGQgYWxzbyBiZSBzd2FwcGVkIG9wcG9zaXRlIHRv IHRoYXQgaW4gbGl0dGxlLWVuZGlhbiBJUC4KLSAqLwotCi1zdGF0aWMgdTMyIGVkbWFfcmVhZGwo c3RydWN0IGZzbF9lZG1hX2VuZ2luZSAqZWRtYSwgdm9pZCBfX2lvbWVtICphZGRyKQotewotCWlm IChlZG1hLT5iaWdfZW5kaWFuKQotCQlyZXR1cm4gaW9yZWFkMzJiZShhZGRyKTsKLQllbHNlCi0J CXJldHVybiBpb3JlYWQzMihhZGRyKTsKLX0KLQotc3RhdGljIHZvaWQgZWRtYV93cml0ZWIoc3Ry dWN0IGZzbF9lZG1hX2VuZ2luZSAqZWRtYSwgdTggdmFsLCB2b2lkIF9faW9tZW0gKmFkZHIpCi17 Ci0JLyogc3dhcCB0aGUgcmVnIG9mZnNldCBmb3IgdGhlc2UgaW4gYmlnLWVuZGlhbiBtb2RlICov Ci0JaWYgKGVkbWEtPmJpZ19lbmRpYW4pCi0JCWlvd3JpdGU4KHZhbCwgKHZvaWQgX19pb21lbSAq KSgodW5zaWduZWQgbG9uZylhZGRyIF4gMHgzKSk7Ci0JZWxzZQotCQlpb3dyaXRlOCh2YWwsIGFk ZHIpOwotfQotCi1zdGF0aWMgdm9pZCBlZG1hX3dyaXRldyhzdHJ1Y3QgZnNsX2VkbWFfZW5naW5l ICplZG1hLCB1MTYgdmFsLCB2b2lkIF9faW9tZW0gKmFkZHIpCi17Ci0JLyogc3dhcCB0aGUgcmVn IG9mZnNldCBmb3IgdGhlc2UgaW4gYmlnLWVuZGlhbiBtb2RlICovCi0JaWYgKGVkbWEtPmJpZ19l bmRpYW4pCi0JCWlvd3JpdGUxNmJlKHZhbCwgKHZvaWQgX19pb21lbSAqKSgodW5zaWduZWQgbG9u ZylhZGRyIF4gMHgyKSk7Ci0JZWxzZQotCQlpb3dyaXRlMTYodmFsLCBhZGRyKTsKLX0KLQotc3Rh dGljIHZvaWQgZWRtYV93cml0ZWwoc3RydWN0IGZzbF9lZG1hX2VuZ2luZSAqZWRtYSwgdTMyIHZh bCwgdm9pZCBfX2lvbWVtICphZGRyKQotewotCWlmIChlZG1hLT5iaWdfZW5kaWFuKQotCQlpb3dy aXRlMzJiZSh2YWwsIGFkZHIpOwotCWVsc2UKLQkJaW93cml0ZTMyKHZhbCwgYWRkcik7Ci19Ci0K LXN0YXRpYyBzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAqdG9fZnNsX2VkbWFfY2hhbihzdHJ1Y3QgZG1h X2NoYW4gKmNoYW4pCi17Ci0JcmV0dXJuIGNvbnRhaW5lcl9vZihjaGFuLCBzdHJ1Y3QgZnNsX2Vk bWFfY2hhbiwgdmNoYW4uY2hhbik7Ci19Ci0KLXN0YXRpYyBzdHJ1Y3QgZnNsX2VkbWFfZGVzYyAq dG9fZnNsX2VkbWFfZGVzYyhzdHJ1Y3QgdmlydF9kbWFfZGVzYyAqdmQpCi17Ci0JcmV0dXJuIGNv bnRhaW5lcl9vZih2ZCwgc3RydWN0IGZzbF9lZG1hX2Rlc2MsIHZkZXNjKTsKLX0KLQotc3RhdGlj IHZvaWQgZnNsX2VkbWFfZW5hYmxlX3JlcXVlc3Qoc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9j aGFuKQotewotCXZvaWQgX19pb21lbSAqYWRkciA9IGZzbF9jaGFuLT5lZG1hLT5tZW1iYXNlOwot CXUzMiBjaCA9IGZzbF9jaGFuLT52Y2hhbi5jaGFuLmNoYW5faWQ7Ci0KLQllZG1hX3dyaXRlYihm c2xfY2hhbi0+ZWRtYSwgRURNQV9TRUVJX1NFRUkoY2gpLCBhZGRyICsgRURNQV9TRUVJKTsKLQll ZG1hX3dyaXRlYihmc2xfY2hhbi0+ZWRtYSwgY2gsIGFkZHIgKyBFRE1BX1NFUlEpOwotfQotCi1z dGF0aWMgdm9pZCBmc2xfZWRtYV9kaXNhYmxlX3JlcXVlc3Qoc3RydWN0IGZzbF9lZG1hX2NoYW4g KmZzbF9jaGFuKQotewotCXZvaWQgX19pb21lbSAqYWRkciA9IGZzbF9jaGFuLT5lZG1hLT5tZW1i YXNlOwotCXUzMiBjaCA9IGZzbF9jaGFuLT52Y2hhbi5jaGFuLmNoYW5faWQ7Ci0KLQllZG1hX3dy aXRlYihmc2xfY2hhbi0+ZWRtYSwgY2gsIGFkZHIgKyBFRE1BX0NFUlEpOwotCWVkbWFfd3JpdGVi KGZzbF9jaGFuLT5lZG1hLCBFRE1BX0NFRUlfQ0VFSShjaCksIGFkZHIgKyBFRE1BX0NFRUkpOwot fQotCi1zdGF0aWMgdm9pZCBmc2xfZWRtYV9jaGFuX211eChzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAq ZnNsX2NoYW4sCi0JCQl1bnNpZ25lZCBpbnQgc2xvdCwgYm9vbCBlbmFibGUpCi17Ci0JdTMyIGNo ID0gZnNsX2NoYW4tPnZjaGFuLmNoYW4uY2hhbl9pZDsKLQl2b2lkIF9faW9tZW0gKm11eGFkZHI7 Ci0JdW5zaWduZWQgY2hhbnNfcGVyX211eCwgY2hfb2ZmOwotCi0JY2hhbnNfcGVyX211eCA9IGZz bF9jaGFuLT5lZG1hLT5uX2NoYW5zIC8gRE1BTVVYX05SOwotCWNoX29mZiA9IGZzbF9jaGFuLT52 Y2hhbi5jaGFuLmNoYW5faWQgJSBjaGFuc19wZXJfbXV4OwotCW11eGFkZHIgPSBmc2xfY2hhbi0+ ZWRtYS0+bXV4YmFzZVtjaCAvIGNoYW5zX3Blcl9tdXhdOwotCXNsb3QgPSBFRE1BTVVYX0NIQ0ZH X1NPVVJDRShzbG90KTsKLQotCWlmIChlbmFibGUpCi0JCWlvd3JpdGU4KEVETUFNVVhfQ0hDRkdf RU5CTCB8IHNsb3QsIG11eGFkZHIgKyBjaF9vZmYpOwotCWVsc2UKLQkJaW93cml0ZTgoRURNQU1V WF9DSENGR19ESVMsIG11eGFkZHIgKyBjaF9vZmYpOwotfQotCi1zdGF0aWMgdW5zaWduZWQgaW50 IGZzbF9lZG1hX2dldF90Y2RfYXR0cihlbnVtIGRtYV9zbGF2ZV9idXN3aWR0aCBhZGRyX3dpZHRo KQotewotCXN3aXRjaCAoYWRkcl93aWR0aCkgewotCWNhc2UgMToKLQkJcmV0dXJuIEVETUFfVENE X0FUVFJfU1NJWkVfOEJJVCB8IEVETUFfVENEX0FUVFJfRFNJWkVfOEJJVDsKLQljYXNlIDI6Ci0J CXJldHVybiBFRE1BX1RDRF9BVFRSX1NTSVpFXzE2QklUIHwgRURNQV9UQ0RfQVRUUl9EU0laRV8x NkJJVDsKLQljYXNlIDQ6Ci0JCXJldHVybiBFRE1BX1RDRF9BVFRSX1NTSVpFXzMyQklUIHwgRURN QV9UQ0RfQVRUUl9EU0laRV8zMkJJVDsKLQljYXNlIDg6Ci0JCXJldHVybiBFRE1BX1RDRF9BVFRS X1NTSVpFXzY0QklUIHwgRURNQV9UQ0RfQVRUUl9EU0laRV82NEJJVDsKLQlkZWZhdWx0OgotCQly ZXR1cm4gRURNQV9UQ0RfQVRUUl9TU0laRV8zMkJJVCB8IEVETUFfVENEX0FUVFJfRFNJWkVfMzJC SVQ7Ci0JfQotfQotCi1zdGF0aWMgdm9pZCBmc2xfZWRtYV9mcmVlX2Rlc2Moc3RydWN0IHZpcnRf ZG1hX2Rlc2MgKnZkZXNjKQotewotCXN0cnVjdCBmc2xfZWRtYV9kZXNjICpmc2xfZGVzYzsKLQlp bnQgaTsKLQotCWZzbF9kZXNjID0gdG9fZnNsX2VkbWFfZGVzYyh2ZGVzYyk7Ci0JZm9yIChpID0g MDsgaSA8IGZzbF9kZXNjLT5uX3RjZHM7IGkrKykKLQkJZG1hX3Bvb2xfZnJlZShmc2xfZGVzYy0+ ZWNoYW4tPnRjZF9wb29sLCBmc2xfZGVzYy0+dGNkW2ldLnZ0Y2QsCi0JCQkgICAgICBmc2xfZGVz Yy0+dGNkW2ldLnB0Y2QpOwotCWtmcmVlKGZzbF9kZXNjKTsKLX0KLQotc3RhdGljIGludCBmc2xf ZWRtYV90ZXJtaW5hdGVfYWxsKHN0cnVjdCBkbWFfY2hhbiAqY2hhbikKLXsKLQlzdHJ1Y3QgZnNs X2VkbWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOwotCXVuc2lnbmVk IGxvbmcgZmxhZ3M7Ci0JTElTVF9IRUFEKGhlYWQpOwotCi0Jc3Bpbl9sb2NrX2lycXNhdmUoJmZz bF9jaGFuLT52Y2hhbi5sb2NrLCBmbGFncyk7Ci0JZnNsX2VkbWFfZGlzYWJsZV9yZXF1ZXN0KGZz bF9jaGFuKTsKLQlmc2xfY2hhbi0+ZWRlc2MgPSBOVUxMOwotCWZzbF9jaGFuLT5pZGxlID0gdHJ1 ZTsKLQl2Y2hhbl9nZXRfYWxsX2Rlc2NyaXB0b3JzKCZmc2xfY2hhbi0+dmNoYW4sICZoZWFkKTsK LQlzcGluX3VubG9ja19pcnFyZXN0b3JlKCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOwot CXZjaGFuX2RtYV9kZXNjX2ZyZWVfbGlzdCgmZnNsX2NoYW4tPnZjaGFuLCAmaGVhZCk7Ci0JcmV0 dXJuIDA7Ci19Ci0KLXN0YXRpYyBpbnQgZnNsX2VkbWFfcGF1c2Uoc3RydWN0IGRtYV9jaGFuICpj aGFuKQotewotCXN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1hX2No YW4oY2hhbik7Ci0JdW5zaWduZWQgbG9uZyBmbGFnczsKLQotCXNwaW5fbG9ja19pcnFzYXZlKCZm c2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOwotCWlmIChmc2xfY2hhbi0+ZWRlc2MpIHsKLQkJ ZnNsX2VkbWFfZGlzYWJsZV9yZXF1ZXN0KGZzbF9jaGFuKTsKLQkJZnNsX2NoYW4tPnN0YXR1cyA9 IERNQV9QQVVTRUQ7Ci0JCWZzbF9jaGFuLT5pZGxlID0gdHJ1ZTsKLQl9Ci0Jc3Bpbl91bmxvY2tf aXJxcmVzdG9yZSgmZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKLQlyZXR1cm4gMDsKLX0K LQotc3RhdGljIGludCBmc2xfZWRtYV9yZXN1bWUoc3RydWN0IGRtYV9jaGFuICpjaGFuKQotewot CXN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1hX2NoYW4oY2hhbik7 Ci0JdW5zaWduZWQgbG9uZyBmbGFnczsKLQotCXNwaW5fbG9ja19pcnFzYXZlKCZmc2xfY2hhbi0+ dmNoYW4ubG9jaywgZmxhZ3MpOwotCWlmIChmc2xfY2hhbi0+ZWRlc2MpIHsKLQkJZnNsX2VkbWFf ZW5hYmxlX3JlcXVlc3QoZnNsX2NoYW4pOwotCQlmc2xfY2hhbi0+c3RhdHVzID0gRE1BX0lOX1BS T0dSRVNTOwotCQlmc2xfY2hhbi0+aWRsZSA9IGZhbHNlOwotCX0KLQlzcGluX3VubG9ja19pcnFy ZXN0b3JlKCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOwotCXJldHVybiAwOwotfQotCi1z dGF0aWMgaW50IGZzbF9lZG1hX3NsYXZlX2NvbmZpZyhzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4sCi0J CQkJIHN0cnVjdCBkbWFfc2xhdmVfY29uZmlnICpjZmcpCi17Ci0Jc3RydWN0IGZzbF9lZG1hX2No YW4gKmZzbF9jaGFuID0gdG9fZnNsX2VkbWFfY2hhbihjaGFuKTsKLQotCWZzbF9jaGFuLT5mc2Mu ZGlyID0gY2ZnLT5kaXJlY3Rpb247Ci0JaWYgKGNmZy0+ZGlyZWN0aW9uID09IERNQV9ERVZfVE9f TUVNKSB7Ci0JCWZzbF9jaGFuLT5mc2MuZGV2X2FkZHIgPSBjZmctPnNyY19hZGRyOwotCQlmc2xf Y2hhbi0+ZnNjLmFkZHJfd2lkdGggPSBjZmctPnNyY19hZGRyX3dpZHRoOwotCQlmc2xfY2hhbi0+ ZnNjLmJ1cnN0ID0gY2ZnLT5zcmNfbWF4YnVyc3Q7Ci0JCWZzbF9jaGFuLT5mc2MuYXR0ciA9IGZz bF9lZG1hX2dldF90Y2RfYXR0cihjZmctPnNyY19hZGRyX3dpZHRoKTsKLQl9IGVsc2UgaWYgKGNm Zy0+ZGlyZWN0aW9uID09IERNQV9NRU1fVE9fREVWKSB7Ci0JCWZzbF9jaGFuLT5mc2MuZGV2X2Fk ZHIgPSBjZmctPmRzdF9hZGRyOwotCQlmc2xfY2hhbi0+ZnNjLmFkZHJfd2lkdGggPSBjZmctPmRz dF9hZGRyX3dpZHRoOwotCQlmc2xfY2hhbi0+ZnNjLmJ1cnN0ID0gY2ZnLT5kc3RfbWF4YnVyc3Q7 Ci0JCWZzbF9jaGFuLT5mc2MuYXR0ciA9IGZzbF9lZG1hX2dldF90Y2RfYXR0cihjZmctPmRzdF9h ZGRyX3dpZHRoKTsKLQl9IGVsc2UgewotCQkJcmV0dXJuIC1FSU5WQUw7Ci0JfQotCXJldHVybiAw OwotfQotCi1zdGF0aWMgc2l6ZV90IGZzbF9lZG1hX2Rlc2NfcmVzaWR1ZShzdHJ1Y3QgZnNsX2Vk bWFfY2hhbiAqZnNsX2NoYW4sCi0JCXN0cnVjdCB2aXJ0X2RtYV9kZXNjICp2ZGVzYywgYm9vbCBp bl9wcm9ncmVzcykKLXsKLQlzdHJ1Y3QgZnNsX2VkbWFfZGVzYyAqZWRlc2MgPSBmc2xfY2hhbi0+ ZWRlc2M7Ci0Jdm9pZCBfX2lvbWVtICphZGRyID0gZnNsX2NoYW4tPmVkbWEtPm1lbWJhc2U7Ci0J dTMyIGNoID0gZnNsX2NoYW4tPnZjaGFuLmNoYW4uY2hhbl9pZDsKLQllbnVtIGRtYV90cmFuc2Zl cl9kaXJlY3Rpb24gZGlyID0gZnNsX2NoYW4tPmZzYy5kaXI7Ci0JZG1hX2FkZHJfdCBjdXJfYWRk ciwgZG1hX2FkZHI7Ci0Jc2l6ZV90IGxlbiwgc2l6ZTsKLQlpbnQgaTsKLQotCS8qIGNhbGN1bGF0 ZSB0aGUgdG90YWwgc2l6ZSBpbiB0aGlzIGRlc2MgKi8KLQlmb3IgKGxlbiA9IGkgPSAwOyBpIDwg ZnNsX2NoYW4tPmVkZXNjLT5uX3RjZHM7IGkrKykKLQkJbGVuICs9IGxlMzJfdG9fY3B1KGVkZXNj LT50Y2RbaV0udnRjZC0+bmJ5dGVzKQotCQkJKiBsZTE2X3RvX2NwdShlZGVzYy0+dGNkW2ldLnZ0 Y2QtPmJpdGVyKTsKLQotCWlmICghaW5fcHJvZ3Jlc3MpCi0JCXJldHVybiBsZW47Ci0KLQlpZiAo ZGlyID09IERNQV9NRU1fVE9fREVWKQotCQljdXJfYWRkciA9IGVkbWFfcmVhZGwoZnNsX2NoYW4t PmVkbWEsIGFkZHIgKyBFRE1BX1RDRF9TQUREUihjaCkpOwotCWVsc2UKLQkJY3VyX2FkZHIgPSBl ZG1hX3JlYWRsKGZzbF9jaGFuLT5lZG1hLCBhZGRyICsgRURNQV9UQ0RfREFERFIoY2gpKTsKLQot CS8qIGZpZ3VyZSBvdXQgdGhlIGZpbmlzaGVkIGFuZCBjYWxjdWxhdGUgdGhlIHJlc2lkdWUgKi8K LQlmb3IgKGkgPSAwOyBpIDwgZnNsX2NoYW4tPmVkZXNjLT5uX3RjZHM7IGkrKykgewotCQlzaXpl ID0gbGUzMl90b19jcHUoZWRlc2MtPnRjZFtpXS52dGNkLT5uYnl0ZXMpCi0JCQkqIGxlMTZfdG9f Y3B1KGVkZXNjLT50Y2RbaV0udnRjZC0+Yml0ZXIpOwotCQlpZiAoZGlyID09IERNQV9NRU1fVE9f REVWKQotCQkJZG1hX2FkZHIgPSBsZTMyX3RvX2NwdShlZGVzYy0+dGNkW2ldLnZ0Y2QtPnNhZGRy KTsKLQkJZWxzZQotCQkJZG1hX2FkZHIgPSBsZTMyX3RvX2NwdShlZGVzYy0+dGNkW2ldLnZ0Y2Qt PmRhZGRyKTsKLQotCQlsZW4gLT0gc2l6ZTsKLQkJaWYgKGN1cl9hZGRyID49IGRtYV9hZGRyICYm IGN1cl9hZGRyIDwgZG1hX2FkZHIgKyBzaXplKSB7Ci0JCQlsZW4gKz0gZG1hX2FkZHIgKyBzaXpl IC0gY3VyX2FkZHI7Ci0JCQlicmVhazsKLQkJfQotCX0KLQotCXJldHVybiBsZW47Ci19Ci0KLXN0 YXRpYyBlbnVtIGRtYV9zdGF0dXMgZnNsX2VkbWFfdHhfc3RhdHVzKHN0cnVjdCBkbWFfY2hhbiAq Y2hhbiwKLQkJZG1hX2Nvb2tpZV90IGNvb2tpZSwgc3RydWN0IGRtYV90eF9zdGF0ZSAqdHhzdGF0 ZSkKLXsKLQlzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFu KGNoYW4pOwotCXN0cnVjdCB2aXJ0X2RtYV9kZXNjICp2ZGVzYzsKLQllbnVtIGRtYV9zdGF0dXMg c3RhdHVzOwotCXVuc2lnbmVkIGxvbmcgZmxhZ3M7Ci0KLQlzdGF0dXMgPSBkbWFfY29va2llX3N0 YXR1cyhjaGFuLCBjb29raWUsIHR4c3RhdGUpOwotCWlmIChzdGF0dXMgPT0gRE1BX0NPTVBMRVRF KQotCQlyZXR1cm4gc3RhdHVzOwotCi0JaWYgKCF0eHN0YXRlKQotCQlyZXR1cm4gZnNsX2NoYW4t PnN0YXR1czsKLQotCXNwaW5fbG9ja19pcnFzYXZlKCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxh Z3MpOwotCXZkZXNjID0gdmNoYW5fZmluZF9kZXNjKCZmc2xfY2hhbi0+dmNoYW4sIGNvb2tpZSk7 Ci0JaWYgKGZzbF9jaGFuLT5lZGVzYyAmJiBjb29raWUgPT0gZnNsX2NoYW4tPmVkZXNjLT52ZGVz Yy50eC5jb29raWUpCi0JCXR4c3RhdGUtPnJlc2lkdWUgPSBmc2xfZWRtYV9kZXNjX3Jlc2lkdWUo ZnNsX2NoYW4sIHZkZXNjLCB0cnVlKTsKLQllbHNlIGlmICh2ZGVzYykKLQkJdHhzdGF0ZS0+cmVz aWR1ZSA9IGZzbF9lZG1hX2Rlc2NfcmVzaWR1ZShmc2xfY2hhbiwgdmRlc2MsIGZhbHNlKTsKLQll bHNlCi0JCXR4c3RhdGUtPnJlc2lkdWUgPSAwOwotCi0Jc3Bpbl91bmxvY2tfaXJxcmVzdG9yZSgm ZnNsX2NoYW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKLQotCXJldHVybiBmc2xfY2hhbi0+c3RhdHVz OwotfQotCi1zdGF0aWMgdm9pZCBmc2xfZWRtYV9zZXRfdGNkX3JlZ3Moc3RydWN0IGZzbF9lZG1h X2NoYW4gKmZzbF9jaGFuLAotCQkJCSAgc3RydWN0IGZzbF9lZG1hX2h3X3RjZCAqdGNkKQotewot CXN0cnVjdCBmc2xfZWRtYV9lbmdpbmUgKmVkbWEgPSBmc2xfY2hhbi0+ZWRtYTsKLQl2b2lkIF9f aW9tZW0gKmFkZHIgPSBmc2xfY2hhbi0+ZWRtYS0+bWVtYmFzZTsKLQl1MzIgY2ggPSBmc2xfY2hh bi0+dmNoYW4uY2hhbi5jaGFuX2lkOwotCi0JLyoKLQkgKiBUQ0QgcGFyYW1ldGVycyBhcmUgc3Rv cmVkIGluIHN0cnVjdCBmc2xfZWRtYV9od190Y2QgaW4gbGl0dGxlCi0JICogZW5kaWFuIGZvcm1h dC4gSG93ZXZlciwgd2UgbmVlZCB0byBsb2FkIHRoZSBUQ0QgcmVnaXN0ZXJzIGluCi0JICogYmln LSBvciBsaXR0bGUtZW5kaWFuIG9iZXlpbmcgdGhlIGVETUEgZW5naW5lIG1vZGVsIGVuZGlhbi4K LQkgKi8KLQllZG1hX3dyaXRldyhlZG1hLCAwLCBhZGRyICsgRURNQV9UQ0RfQ1NSKGNoKSk7Ci0J ZWRtYV93cml0ZWwoZWRtYSwgbGUzMl90b19jcHUodGNkLT5zYWRkciksIGFkZHIgKyBFRE1BX1RD RF9TQUREUihjaCkpOwotCWVkbWFfd3JpdGVsKGVkbWEsIGxlMzJfdG9fY3B1KHRjZC0+ZGFkZHIp LCBhZGRyICsgRURNQV9UQ0RfREFERFIoY2gpKTsKLQotCWVkbWFfd3JpdGV3KGVkbWEsIGxlMTZf dG9fY3B1KHRjZC0+YXR0ciksIGFkZHIgKyBFRE1BX1RDRF9BVFRSKGNoKSk7Ci0JZWRtYV93cml0 ZXcoZWRtYSwgbGUxNl90b19jcHUodGNkLT5zb2ZmKSwgYWRkciArIEVETUFfVENEX1NPRkYoY2gp KTsKLQotCWVkbWFfd3JpdGVsKGVkbWEsIGxlMzJfdG9fY3B1KHRjZC0+bmJ5dGVzKSwgYWRkciAr IEVETUFfVENEX05CWVRFUyhjaCkpOwotCWVkbWFfd3JpdGVsKGVkbWEsIGxlMzJfdG9fY3B1KHRj ZC0+c2xhc3QpLCBhZGRyICsgRURNQV9UQ0RfU0xBU1QoY2gpKTsKLQotCWVkbWFfd3JpdGV3KGVk bWEsIGxlMTZfdG9fY3B1KHRjZC0+Y2l0ZXIpLCBhZGRyICsgRURNQV9UQ0RfQ0lURVIoY2gpKTsK LQllZG1hX3dyaXRldyhlZG1hLCBsZTE2X3RvX2NwdSh0Y2QtPmJpdGVyKSwgYWRkciArIEVETUFf VENEX0JJVEVSKGNoKSk7Ci0JZWRtYV93cml0ZXcoZWRtYSwgbGUxNl90b19jcHUodGNkLT5kb2Zm KSwgYWRkciArIEVETUFfVENEX0RPRkYoY2gpKTsKLQotCWVkbWFfd3JpdGVsKGVkbWEsIGxlMzJf dG9fY3B1KHRjZC0+ZGxhc3Rfc2dhKSwgYWRkciArIEVETUFfVENEX0RMQVNUX1NHQShjaCkpOwot Ci0JZWRtYV93cml0ZXcoZWRtYSwgbGUxNl90b19jcHUodGNkLT5jc3IpLCBhZGRyICsgRURNQV9U Q0RfQ1NSKGNoKSk7Ci19Ci0KLXN0YXRpYyBpbmxpbmUKLXZvaWQgZnNsX2VkbWFfZmlsbF90Y2Qo c3RydWN0IGZzbF9lZG1hX2h3X3RjZCAqdGNkLCB1MzIgc3JjLCB1MzIgZHN0LAotCQkgICAgICAg dTE2IGF0dHIsIHUxNiBzb2ZmLCB1MzIgbmJ5dGVzLCB1MzIgc2xhc3QsIHUxNiBjaXRlciwKLQkJ ICAgICAgIHUxNiBiaXRlciwgdTE2IGRvZmYsIHUzMiBkbGFzdF9zZ2EsIGJvb2wgbWFqb3JfaW50 LAotCQkgICAgICAgYm9vbCBkaXNhYmxlX3JlcSwgYm9vbCBlbmFibGVfc2cpCi17Ci0JdTE2IGNz ciA9IDA7Ci0KLQkvKgotCSAqIGVETUEgaGFyZHdhcmUgU0dzIHJlcXVpcmUgdGhlIFRDRHMgdG8g YmUgc3RvcmVkIGluIGxpdHRsZQotCSAqIGVuZGlhbiBmb3JtYXQgaXJyZXNwZWN0aXZlIG9mIHRo ZSByZWdpc3RlciBlbmRpYW4gbW9kZWwuCi0JICogU28gd2UgcHV0IHRoZSB2YWx1ZSBpbiBsaXR0 bGUgZW5kaWFuIGluIG1lbW9yeSwgd2FpdGluZwotCSAqIGZvciBmc2xfZWRtYV9zZXRfdGNkX3Jl Z3MgZG9pbmcgdGhlIHN3YXAuCi0JICovCi0JdGNkLT5zYWRkciA9IGNwdV90b19sZTMyKHNyYyk7 Ci0JdGNkLT5kYWRkciA9IGNwdV90b19sZTMyKGRzdCk7Ci0KLQl0Y2QtPmF0dHIgPSBjcHVfdG9f bGUxNihhdHRyKTsKLQotCXRjZC0+c29mZiA9IGNwdV90b19sZTE2KEVETUFfVENEX1NPRkZfU09G Rihzb2ZmKSk7Ci0KLQl0Y2QtPm5ieXRlcyA9IGNwdV90b19sZTMyKEVETUFfVENEX05CWVRFU19O QllURVMobmJ5dGVzKSk7Ci0JdGNkLT5zbGFzdCA9IGNwdV90b19sZTMyKEVETUFfVENEX1NMQVNU X1NMQVNUKHNsYXN0KSk7Ci0KLQl0Y2QtPmNpdGVyID0gY3B1X3RvX2xlMTYoRURNQV9UQ0RfQ0lU RVJfQ0lURVIoY2l0ZXIpKTsKLQl0Y2QtPmRvZmYgPSBjcHVfdG9fbGUxNihFRE1BX1RDRF9ET0ZG X0RPRkYoZG9mZikpOwotCi0JdGNkLT5kbGFzdF9zZ2EgPSBjcHVfdG9fbGUzMihFRE1BX1RDRF9E TEFTVF9TR0FfRExBU1RfU0dBKGRsYXN0X3NnYSkpOwotCi0JdGNkLT5iaXRlciA9IGNwdV90b19s ZTE2KEVETUFfVENEX0JJVEVSX0JJVEVSKGJpdGVyKSk7Ci0JaWYgKG1ham9yX2ludCkKLQkJY3Ny IHw9IEVETUFfVENEX0NTUl9JTlRfTUFKT1I7Ci0KLQlpZiAoZGlzYWJsZV9yZXEpCi0JCWNzciB8 PSBFRE1BX1RDRF9DU1JfRF9SRVE7Ci0KLQlpZiAoZW5hYmxlX3NnKQotCQljc3IgfD0gRURNQV9U Q0RfQ1NSX0VfU0c7Ci0KLQl0Y2QtPmNzciA9IGNwdV90b19sZTE2KGNzcik7Ci19Ci0KLXN0YXRp YyBzdHJ1Y3QgZnNsX2VkbWFfZGVzYyAqZnNsX2VkbWFfYWxsb2NfZGVzYyhzdHJ1Y3QgZnNsX2Vk bWFfY2hhbiAqZnNsX2NoYW4sCi0JCWludCBzZ19sZW4pCi17Ci0Jc3RydWN0IGZzbF9lZG1hX2Rl c2MgKmZzbF9kZXNjOwotCWludCBpOwotCi0JZnNsX2Rlc2MgPSBremFsbG9jKHNpemVvZigqZnNs X2Rlc2MpICsgc2l6ZW9mKHN0cnVjdCBmc2xfZWRtYV9zd190Y2QpICogc2dfbGVuLAotCQkJCUdG UF9OT1dBSVQpOwotCWlmICghZnNsX2Rlc2MpCi0JCXJldHVybiBOVUxMOwotCi0JZnNsX2Rlc2Mt PmVjaGFuID0gZnNsX2NoYW47Ci0JZnNsX2Rlc2MtPm5fdGNkcyA9IHNnX2xlbjsKLQlmb3IgKGkg PSAwOyBpIDwgc2dfbGVuOyBpKyspIHsKLQkJZnNsX2Rlc2MtPnRjZFtpXS52dGNkID0gZG1hX3Bv b2xfYWxsb2MoZnNsX2NoYW4tPnRjZF9wb29sLAotCQkJCQlHRlBfTk9XQUlULCAmZnNsX2Rlc2Mt PnRjZFtpXS5wdGNkKTsKLQkJaWYgKCFmc2xfZGVzYy0+dGNkW2ldLnZ0Y2QpCi0JCQlnb3RvIGVy cjsKLQl9Ci0JcmV0dXJuIGZzbF9kZXNjOwotCi1lcnI6Ci0Jd2hpbGUgKC0taSA+PSAwKQotCQlk bWFfcG9vbF9mcmVlKGZzbF9jaGFuLT50Y2RfcG9vbCwgZnNsX2Rlc2MtPnRjZFtpXS52dGNkLAot CQkJCWZzbF9kZXNjLT50Y2RbaV0ucHRjZCk7Ci0Ja2ZyZWUoZnNsX2Rlc2MpOwotCXJldHVybiBO VUxMOwotfQotCi1zdGF0aWMgc3RydWN0IGRtYV9hc3luY190eF9kZXNjcmlwdG9yICpmc2xfZWRt YV9wcmVwX2RtYV9jeWNsaWMoCi0JCXN0cnVjdCBkbWFfY2hhbiAqY2hhbiwgZG1hX2FkZHJfdCBk bWFfYWRkciwgc2l6ZV90IGJ1Zl9sZW4sCi0JCXNpemVfdCBwZXJpb2RfbGVuLCBlbnVtIGRtYV90 cmFuc2Zlcl9kaXJlY3Rpb24gZGlyZWN0aW9uLAotCQl1bnNpZ25lZCBsb25nIGZsYWdzKQotewot CXN0cnVjdCBmc2xfZWRtYV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1hX2NoYW4oY2hhbik7 Ci0Jc3RydWN0IGZzbF9lZG1hX2Rlc2MgKmZzbF9kZXNjOwotCWRtYV9hZGRyX3QgZG1hX2J1Zl9u ZXh0OwotCWludCBzZ19sZW4sIGk7Ci0JdTMyIHNyY19hZGRyLCBkc3RfYWRkciwgbGFzdF9zZywg bmJ5dGVzOwotCXUxNiBzb2ZmLCBkb2ZmLCBpdGVyOwotCi0JaWYgKCFpc19zbGF2ZV9kaXJlY3Rp b24oZnNsX2NoYW4tPmZzYy5kaXIpKQotCQlyZXR1cm4gTlVMTDsKLQotCXNnX2xlbiA9IGJ1Zl9s ZW4gLyBwZXJpb2RfbGVuOwotCWZzbF9kZXNjID0gZnNsX2VkbWFfYWxsb2NfZGVzYyhmc2xfY2hh biwgc2dfbGVuKTsKLQlpZiAoIWZzbF9kZXNjKQotCQlyZXR1cm4gTlVMTDsKLQlmc2xfZGVzYy0+ aXNjeWNsaWMgPSB0cnVlOwotCi0JZG1hX2J1Zl9uZXh0ID0gZG1hX2FkZHI7Ci0JbmJ5dGVzID0g ZnNsX2NoYW4tPmZzYy5hZGRyX3dpZHRoICogZnNsX2NoYW4tPmZzYy5idXJzdDsKLQlpdGVyID0g cGVyaW9kX2xlbiAvIG5ieXRlczsKLQotCWZvciAoaSA9IDA7IGkgPCBzZ19sZW47IGkrKykgewot CQlpZiAoZG1hX2J1Zl9uZXh0ID49IGRtYV9hZGRyICsgYnVmX2xlbikKLQkJCWRtYV9idWZfbmV4 dCA9IGRtYV9hZGRyOwotCi0JCS8qIGdldCBuZXh0IHNnJ3MgcGh5c2ljYWwgYWRkcmVzcyAqLwot CQlsYXN0X3NnID0gZnNsX2Rlc2MtPnRjZFsoaSArIDEpICUgc2dfbGVuXS5wdGNkOwotCi0JCWlm IChmc2xfY2hhbi0+ZnNjLmRpciA9PSBETUFfTUVNX1RPX0RFVikgewotCQkJc3JjX2FkZHIgPSBk bWFfYnVmX25leHQ7Ci0JCQlkc3RfYWRkciA9IGZzbF9jaGFuLT5mc2MuZGV2X2FkZHI7Ci0JCQlz b2ZmID0gZnNsX2NoYW4tPmZzYy5hZGRyX3dpZHRoOwotCQkJZG9mZiA9IDA7Ci0JCX0gZWxzZSB7 Ci0JCQlzcmNfYWRkciA9IGZzbF9jaGFuLT5mc2MuZGV2X2FkZHI7Ci0JCQlkc3RfYWRkciA9IGRt YV9idWZfbmV4dDsKLQkJCXNvZmYgPSAwOwotCQkJZG9mZiA9IGZzbF9jaGFuLT5mc2MuYWRkcl93 aWR0aDsKLQkJfQotCi0JCWZzbF9lZG1hX2ZpbGxfdGNkKGZzbF9kZXNjLT50Y2RbaV0udnRjZCwg c3JjX2FkZHIsIGRzdF9hZGRyLAotCQkJCSAgZnNsX2NoYW4tPmZzYy5hdHRyLCBzb2ZmLCBuYnl0 ZXMsIDAsIGl0ZXIsCi0JCQkJICBpdGVyLCBkb2ZmLCBsYXN0X3NnLCB0cnVlLCBmYWxzZSwgdHJ1 ZSk7Ci0JCWRtYV9idWZfbmV4dCArPSBwZXJpb2RfbGVuOwotCX0KLQotCXJldHVybiB2Y2hhbl90 eF9wcmVwKCZmc2xfY2hhbi0+dmNoYW4sICZmc2xfZGVzYy0+dmRlc2MsIGZsYWdzKTsKLX0KLQot c3RhdGljIHN0cnVjdCBkbWFfYXN5bmNfdHhfZGVzY3JpcHRvciAqZnNsX2VkbWFfcHJlcF9zbGF2 ZV9zZygKLQkJc3RydWN0IGRtYV9jaGFuICpjaGFuLCBzdHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnbCwK LQkJdW5zaWduZWQgaW50IHNnX2xlbiwgZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uIGRpcmVj dGlvbiwKLQkJdW5zaWduZWQgbG9uZyBmbGFncywgdm9pZCAqY29udGV4dCkKLXsKLQlzdHJ1Y3Qg ZnNsX2VkbWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNoYW4pOwotCXN0cnVj dCBmc2xfZWRtYV9kZXNjICpmc2xfZGVzYzsKLQlzdHJ1Y3Qgc2NhdHRlcmxpc3QgKnNnOwotCXUz MiBzcmNfYWRkciwgZHN0X2FkZHIsIGxhc3Rfc2csIG5ieXRlczsKLQl1MTYgc29mZiwgZG9mZiwg aXRlcjsKLQlpbnQgaTsKLQotCWlmICghaXNfc2xhdmVfZGlyZWN0aW9uKGZzbF9jaGFuLT5mc2Mu ZGlyKSkKLQkJcmV0dXJuIE5VTEw7Ci0KLQlmc2xfZGVzYyA9IGZzbF9lZG1hX2FsbG9jX2Rlc2Mo ZnNsX2NoYW4sIHNnX2xlbik7Ci0JaWYgKCFmc2xfZGVzYykKLQkJcmV0dXJuIE5VTEw7Ci0JZnNs X2Rlc2MtPmlzY3ljbGljID0gZmFsc2U7Ci0KLQluYnl0ZXMgPSBmc2xfY2hhbi0+ZnNjLmFkZHJf d2lkdGggKiBmc2xfY2hhbi0+ZnNjLmJ1cnN0OwotCWZvcl9lYWNoX3NnKHNnbCwgc2csIHNnX2xl biwgaSkgewotCQkvKiBnZXQgbmV4dCBzZydzIHBoeXNpY2FsIGFkZHJlc3MgKi8KLQkJbGFzdF9z ZyA9IGZzbF9kZXNjLT50Y2RbKGkgKyAxKSAlIHNnX2xlbl0ucHRjZDsKLQotCQlpZiAoZnNsX2No YW4tPmZzYy5kaXIgPT0gRE1BX01FTV9UT19ERVYpIHsKLQkJCXNyY19hZGRyID0gc2dfZG1hX2Fk ZHJlc3Moc2cpOwotCQkJZHN0X2FkZHIgPSBmc2xfY2hhbi0+ZnNjLmRldl9hZGRyOwotCQkJc29m ZiA9IGZzbF9jaGFuLT5mc2MuYWRkcl93aWR0aDsKLQkJCWRvZmYgPSAwOwotCQl9IGVsc2Ugewot CQkJc3JjX2FkZHIgPSBmc2xfY2hhbi0+ZnNjLmRldl9hZGRyOwotCQkJZHN0X2FkZHIgPSBzZ19k bWFfYWRkcmVzcyhzZyk7Ci0JCQlzb2ZmID0gMDsKLQkJCWRvZmYgPSBmc2xfY2hhbi0+ZnNjLmFk ZHJfd2lkdGg7Ci0JCX0KLQotCQlpdGVyID0gc2dfZG1hX2xlbihzZykgLyBuYnl0ZXM7Ci0JCWlm IChpIDwgc2dfbGVuIC0gMSkgewotCQkJbGFzdF9zZyA9IGZzbF9kZXNjLT50Y2RbKGkgKyAxKV0u cHRjZDsKLQkJCWZzbF9lZG1hX2ZpbGxfdGNkKGZzbF9kZXNjLT50Y2RbaV0udnRjZCwgc3JjX2Fk ZHIsCi0JCQkJCSAgZHN0X2FkZHIsIGZzbF9jaGFuLT5mc2MuYXR0ciwgc29mZiwKLQkJCQkJICBu Ynl0ZXMsIDAsIGl0ZXIsIGl0ZXIsIGRvZmYsIGxhc3Rfc2csCi0JCQkJCSAgZmFsc2UsIGZhbHNl LCB0cnVlKTsKLQkJfSBlbHNlIHsKLQkJCWxhc3Rfc2cgPSAwOwotCQkJZnNsX2VkbWFfZmlsbF90 Y2QoZnNsX2Rlc2MtPnRjZFtpXS52dGNkLCBzcmNfYWRkciwKLQkJCQkJICBkc3RfYWRkciwgZnNs X2NoYW4tPmZzYy5hdHRyLCBzb2ZmLAotCQkJCQkgIG5ieXRlcywgMCwgaXRlciwgaXRlciwgZG9m ZiwgbGFzdF9zZywKLQkJCQkJICB0cnVlLCB0cnVlLCBmYWxzZSk7Ci0JCX0KLQl9Ci0KLQlyZXR1 cm4gdmNoYW5fdHhfcHJlcCgmZnNsX2NoYW4tPnZjaGFuLCAmZnNsX2Rlc2MtPnZkZXNjLCBmbGFn cyk7Ci19Ci0KLXN0YXRpYyB2b2lkIGZzbF9lZG1hX3hmZXJfZGVzYyhzdHJ1Y3QgZnNsX2VkbWFf Y2hhbiAqZnNsX2NoYW4pCi17Ci0Jc3RydWN0IHZpcnRfZG1hX2Rlc2MgKnZkZXNjOwotCi0JdmRl c2MgPSB2Y2hhbl9uZXh0X2Rlc2MoJmZzbF9jaGFuLT52Y2hhbik7Ci0JaWYgKCF2ZGVzYykKLQkJ cmV0dXJuOwotCWZzbF9jaGFuLT5lZGVzYyA9IHRvX2ZzbF9lZG1hX2Rlc2ModmRlc2MpOwotCWZz bF9lZG1hX3NldF90Y2RfcmVncyhmc2xfY2hhbiwgZnNsX2NoYW4tPmVkZXNjLT50Y2RbMF0udnRj ZCk7Ci0JZnNsX2VkbWFfZW5hYmxlX3JlcXVlc3QoZnNsX2NoYW4pOwotCWZzbF9jaGFuLT5zdGF0 dXMgPSBETUFfSU5fUFJPR1JFU1M7Ci0JZnNsX2NoYW4tPmlkbGUgPSBmYWxzZTsKLX0KKyNpbmNs dWRlICJmc2wtZWRtYS1jb21tb24uaCIKIAogc3RhdGljIGlycXJldHVybl90IGZzbF9lZG1hX3R4 X2hhbmRsZXIoaW50IGlycSwgdm9pZCAqZGV2X2lkKQogewpAQCAtNzMwLDI1ICs5Myw2IEBAIHN0 YXRpYyBpcnFyZXR1cm5fdCBmc2xfZWRtYV9pcnFfaGFuZGxlcihpbnQgaXJxLCB2b2lkICpkZXZf aWQpCiAJcmV0dXJuIGZzbF9lZG1hX2Vycl9oYW5kbGVyKGlycSwgZGV2X2lkKTsKIH0KIAotc3Rh dGljIHZvaWQgZnNsX2VkbWFfaXNzdWVfcGVuZGluZyhzdHJ1Y3QgZG1hX2NoYW4gKmNoYW4pCi17 Ci0Jc3RydWN0IGZzbF9lZG1hX2NoYW4gKmZzbF9jaGFuID0gdG9fZnNsX2VkbWFfY2hhbihjaGFu KTsKLQl1bnNpZ25lZCBsb25nIGZsYWdzOwotCi0Jc3Bpbl9sb2NrX2lycXNhdmUoJmZzbF9jaGFu LT52Y2hhbi5sb2NrLCBmbGFncyk7Ci0KLQlpZiAodW5saWtlbHkoZnNsX2NoYW4tPnBtX3N0YXRl ICE9IFJVTk5JTkcpKSB7Ci0JCXNwaW5fdW5sb2NrX2lycXJlc3RvcmUoJmZzbF9jaGFuLT52Y2hh bi5sb2NrLCBmbGFncyk7Ci0JCS8qIGNhbm5vdCBzdWJtaXQgZHVlIHRvIHN1c3BlbmQgKi8KLQkJ cmV0dXJuOwotCX0KLQotCWlmICh2Y2hhbl9pc3N1ZV9wZW5kaW5nKCZmc2xfY2hhbi0+dmNoYW4p ICYmICFmc2xfY2hhbi0+ZWRlc2MpCi0JCWZzbF9lZG1hX3hmZXJfZGVzYyhmc2xfY2hhbik7Ci0K LQlzcGluX3VubG9ja19pcnFyZXN0b3JlKCZmc2xfY2hhbi0+dmNoYW4ubG9jaywgZmxhZ3MpOwot fQotCiBzdGF0aWMgc3RydWN0IGRtYV9jaGFuICpmc2xfZWRtYV94bGF0ZShzdHJ1Y3Qgb2ZfcGhh bmRsZV9hcmdzICpkbWFfc3BlYywKIAkJc3RydWN0IG9mX2RtYSAqb2ZkbWEpCiB7CkBAIC03ODEs MzQgKzEyNSw2IEBAIHN0YXRpYyBzdHJ1Y3QgZG1hX2NoYW4gKmZzbF9lZG1hX3hsYXRlKHN0cnVj dCBvZl9waGFuZGxlX2FyZ3MgKmRtYV9zcGVjLAogCXJldHVybiBOVUxMOwogfQogCi1zdGF0aWMg aW50IGZzbF9lZG1hX2FsbG9jX2NoYW5fcmVzb3VyY2VzKHN0cnVjdCBkbWFfY2hhbiAqY2hhbikK LXsKLQlzdHJ1Y3QgZnNsX2VkbWFfY2hhbiAqZnNsX2NoYW4gPSB0b19mc2xfZWRtYV9jaGFuKGNo YW4pOwotCi0JZnNsX2NoYW4tPnRjZF9wb29sID0gZG1hX3Bvb2xfY3JlYXRlKCJ0Y2RfcG9vbCIs IGNoYW4tPmRldmljZS0+ZGV2LAotCQkJCXNpemVvZihzdHJ1Y3QgZnNsX2VkbWFfaHdfdGNkKSwK LQkJCQkzMiwgMCk7Ci0JcmV0dXJuIDA7Ci19Ci0KLXN0YXRpYyB2b2lkIGZzbF9lZG1hX2ZyZWVf Y2hhbl9yZXNvdXJjZXMoc3RydWN0IGRtYV9jaGFuICpjaGFuKQotewotCXN0cnVjdCBmc2xfZWRt YV9jaGFuICpmc2xfY2hhbiA9IHRvX2ZzbF9lZG1hX2NoYW4oY2hhbik7Ci0JdW5zaWduZWQgbG9u ZyBmbGFnczsKLQlMSVNUX0hFQUQoaGVhZCk7Ci0KLQlzcGluX2xvY2tfaXJxc2F2ZSgmZnNsX2No YW4tPnZjaGFuLmxvY2ssIGZsYWdzKTsKLQlmc2xfZWRtYV9kaXNhYmxlX3JlcXVlc3QoZnNsX2No YW4pOwotCWZzbF9lZG1hX2NoYW5fbXV4KGZzbF9jaGFuLCAwLCBmYWxzZSk7Ci0JZnNsX2NoYW4t PmVkZXNjID0gTlVMTDsKLQl2Y2hhbl9nZXRfYWxsX2Rlc2NyaXB0b3JzKCZmc2xfY2hhbi0+dmNo YW4sICZoZWFkKTsKLQlzcGluX3VubG9ja19pcnFyZXN0b3JlKCZmc2xfY2hhbi0+dmNoYW4ubG9j aywgZmxhZ3MpOwotCi0JdmNoYW5fZG1hX2Rlc2NfZnJlZV9saXN0KCZmc2xfY2hhbi0+dmNoYW4s ICZoZWFkKTsKLQlkbWFfcG9vbF9kZXN0cm95KGZzbF9jaGFuLT50Y2RfcG9vbCk7Ci0JZnNsX2No YW4tPnRjZF9wb29sID0gTlVMTDsKLX0KLQogc3RhdGljIGludAogZnNsX2VkbWFfaXJxX2luaXQo c3RydWN0IHBsYXRmb3JtX2RldmljZSAqcGRldiwgc3RydWN0IGZzbF9lZG1hX2VuZ2luZSAqZnNs X2VkbWEpCiB7CkBAIC05OTUsMTcgKzMxMSw2IEBAIHN0YXRpYyBpbnQgZnNsX2VkbWFfcHJvYmUo c3RydWN0IHBsYXRmb3JtX2RldmljZSAqcGRldikKIAlyZXR1cm4gMDsKIH0KIAotc3RhdGljIHZv aWQgZnNsX2VkbWFfY2xlYW51cF92Y2hhbihzdHJ1Y3QgZG1hX2RldmljZSAqZG1hZGV2KQotewot CXN0cnVjdCBmc2xfZWRtYV9jaGFuICpjaGFuLCAqX2NoYW47Ci0KLQlsaXN0X2Zvcl9lYWNoX2Vu dHJ5X3NhZmUoY2hhbiwgX2NoYW4sCi0JCQkJJmRtYWRldi0+Y2hhbm5lbHMsIHZjaGFuLmNoYW4u ZGV2aWNlX25vZGUpIHsKLQkJbGlzdF9kZWwoJmNoYW4tPnZjaGFuLmNoYW4uZGV2aWNlX25vZGUp OwotCQl0YXNrbGV0X2tpbGwoJmNoYW4tPnZjaGFuLnRhc2spOwotCX0KLX0KLQogc3RhdGljIGlu dCBmc2xfZWRtYV9yZW1vdmUoc3RydWN0IHBsYXRmb3JtX2RldmljZSAqcGRldikKIHsKIAlzdHJ1 Y3QgZGV2aWNlX25vZGUgKm5wID0gcGRldi0+ZGV2Lm9mX25vZGU7Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 From: Angelo Dureghello Subject: [PATCH v8 1/3] dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended) Date: Fri, 3 Aug 2018 21:32:42 +0200 Message-ID: <20180803193244.12084-1-angelo@sysam.it> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=m.gmane.org@lists.infradead.org To: vinod.koul@linaro.org Cc: linux-m68k@vger.kernel.org, Angelo Dureghello , krzk@kernel.org, stefan@agner.ch, dmaengine@vger.kernel.org, linux-arm-kernel@lists.infradead.org List-Id: linux-m68k@vger.kernel.org This patch adds a new fsl-edma-common module to allow new mcf-edma module code to use most of the fsl-edma code. Signed-off-by: Angelo Dureghello --- Changes for v2: - patch splitted into 4 - add mcf-edma as minimal different parts from fsl-edma Changes for v3: none Changes for v4: - patch simplified from 4/4 into 2/2 - collecting all the mcf-edma-related changes Changes for v5: none Changes for v6: - adjusted comment header - fixed bit shift with BIT() - we need to free the interrupts at remove(), so removed all devm_ interrupt related calls Changes for v7: none Changes for v8: - patch rewritten from scratch, splitted into 3, common code isolated, minimal changes from the original Freescale code have been done. The patch has been tested with both Iris + Colibri Vybrid VF50 and stmark2/mcf54415 Coldfire boards. --- drivers/dma/Makefile | 2 +- drivers/dma/fsl-edma-common.c | 576 ++++++++++++++++++++++++++++ drivers/dma/fsl-edma-common.h | 196 ++++++++++ drivers/dma/fsl-edma.c | 697 +--------------------------------- 4 files changed, 774 insertions(+), 697 deletions(-) create mode 100644 drivers/dma/fsl-edma-common.c create mode 100644 drivers/dma/fsl-edma-common.h diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile index 203a99d68315..66022f59fca4 100644 --- a/drivers/dma/Makefile +++ b/drivers/dma/Makefile @@ -31,7 +31,7 @@ obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/ obj-$(CONFIG_DW_DMAC_CORE) += dw/ obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o obj-$(CONFIG_FSL_DMA) += fsldma.o -obj-$(CONFIG_FSL_EDMA) += fsl-edma.o +obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o obj-$(CONFIG_FSL_RAID) += fsl_raid.o obj-$(CONFIG_HSU_DMA) += hsu/ obj-$(CONFIG_IMG_MDC_DMA) += img-mdc-dma.o diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c new file mode 100644 index 000000000000..0ae7094f477a --- /dev/null +++ b/drivers/dma/fsl-edma-common.c @@ -0,0 +1,576 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright (c) 2013-2014 Freescale Semiconductor, Inc +// Copyright (c) 2017 Sysam, Angelo Dureghello + +#include +#include +#include + +#include "fsl-edma-common.h" + +/* + * R/W functions for big- or little-endian registers: + * The eDMA controller's endian is independent of the CPU core's endian. + * For the big-endian IP module, the offset for 8-bit or 16-bit registers + * should also be swapped opposite to that in little-endian IP. + */ +u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr) +{ + if (edma->big_endian) + return ioread32be(addr); + else + return ioread32(addr); +} +EXPORT_SYMBOL_GPL(edma_readl); + +void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); + else + iowrite8(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writeb); + +void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); + else + iowrite16(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writew); + +void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr) +{ + if (edma->big_endian) + iowrite32be(val, addr); + else + iowrite32(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writel); + +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) +{ + return container_of(chan, struct fsl_edma_chan, vchan.chan); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_chan); + +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) +{ + return container_of(vd, struct fsl_edma_desc, vdesc); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_desc); + +static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) +{ + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI); + edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ); +} + +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) +{ + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ); + edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI); +} +EXPORT_SYMBOL_GPL(fsl_edma_disable_request); + +void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, + unsigned int slot, bool enable) +{ + u32 ch = fsl_chan->vchan.chan.chan_id; + void __iomem *muxaddr; + unsigned int chans_per_mux, ch_off; + + chans_per_mux = fsl_chan->edma->n_chans / DMAMUX_NR; + ch_off = fsl_chan->vchan.chan.chan_id % chans_per_mux; + muxaddr = fsl_chan->edma->muxbase[ch / chans_per_mux]; + slot = EDMAMUX_CHCFG_SOURCE(slot); + + if (enable) + iowrite8(EDMAMUX_CHCFG_ENBL | slot, muxaddr + ch_off); + else + iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off); +} +EXPORT_SYMBOL_GPL(fsl_edma_chan_mux); + +static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) +{ + switch (addr_width) { + case 1: + return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; + case 2: + return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; + case 4: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + case 8: + return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; + default: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + } +} + +void fsl_edma_free_desc(struct virt_dma_desc *vdesc) +{ + struct fsl_edma_desc *fsl_desc; + int i; + + fsl_desc = to_fsl_edma_desc(vdesc); + for (i = 0; i < fsl_desc->n_tcds; i++) + dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd, + fsl_desc->tcd[i].ptcd); + kfree(fsl_desc); +} +EXPORT_SYMBOL_GPL(fsl_edma_free_desc); + +int fsl_edma_terminate_all(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_chan->edesc = NULL; + fsl_chan->idle = true; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_terminate_all); + +int fsl_edma_pause(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_disable_request(fsl_chan); + fsl_chan->status = DMA_PAUSED; + fsl_chan->idle = true; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_pause); + +int fsl_edma_resume(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_resume); + +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + + fsl_chan->fsc.dir = cfg->direction; + if (cfg->direction == DMA_DEV_TO_MEM) { + fsl_chan->fsc.dev_addr = cfg->src_addr; + fsl_chan->fsc.addr_width = cfg->src_addr_width; + fsl_chan->fsc.burst = cfg->src_maxburst; + fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width); + } else if (cfg->direction == DMA_MEM_TO_DEV) { + fsl_chan->fsc.dev_addr = cfg->dst_addr; + fsl_chan->fsc.addr_width = cfg->dst_addr_width; + fsl_chan->fsc.burst = cfg->dst_maxburst; + fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width); + } else + return -EINVAL; + + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_slave_config); + +static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, + struct virt_dma_desc *vdesc, bool in_progress) +{ + struct fsl_edma_desc *edesc = fsl_chan->edesc; + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + enum dma_transfer_direction dir = fsl_chan->fsc.dir; + dma_addr_t cur_addr, dma_addr; + size_t len, size; + int i; + + /* calculate the total size in this desc */ + for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) + len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + + if (!in_progress) + return len; + + if (dir == DMA_MEM_TO_DEV) + cur_addr = edma_readl( + fsl_chan->edma, addr + EDMA_TCD_SADDR(ch)); + else + cur_addr = edma_readl( + fsl_chan->edma, addr + EDMA_TCD_DADDR(ch)); + + /* figure out the finished and calculate the residue */ + for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { + size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + if (dir == DMA_MEM_TO_DEV) + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); + else + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); + + len -= size; + if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { + len += dma_addr + size - cur_addr; + break; + } + } + + return len; +} + +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct virt_dma_desc *vdesc; + enum dma_status status; + unsigned long flags; + + status = dma_cookie_status(chan, cookie, txstate); + if (status == DMA_COMPLETE) + return status; + + if (!txstate) + return fsl_chan->status; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); + if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, true); + else if (vdesc) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, false); + else + txstate->residue = 0; + + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + return fsl_chan->status; +} +EXPORT_SYMBOL_GPL(fsl_edma_tx_status); + +static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, + struct fsl_edma_hw_tcd *tcd) +{ + struct fsl_edma_engine *edma = fsl_chan->edma; + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + /* + * TCD parameters are stored in struct fsl_edma_hw_tcd in little + * endian format. However, we need to load the TCD registers in + * big- or little-endian obeying the eDMA engine model endian. + */ + edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch)); + edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch)); + edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch)); + + edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch)); + edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch)); + + edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch)); + edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch)); + + edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch)); + edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch)); + edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch)); + + edma_writel(edma, + le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch)); + + edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch)); +} + +static inline +void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst, + u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer, + u16 biter, u16 doff, u32 dlast_sga, bool major_int, + bool disable_req, bool enable_sg) +{ + u16 csr = 0; + + /* + * eDMA hardware SGs require the TCDs to be stored in little + * endian format irrespective of the register endian model. + * So we put the value in little endian in memory, waiting + * for fsl_edma_set_tcd_regs doing the swap. + */ + tcd->saddr = cpu_to_le32(src); + tcd->daddr = cpu_to_le32(dst); + + tcd->attr = cpu_to_le16(attr); + + tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff)); + + tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes)); + tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast)); + + tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); + tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff)); + + tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga)); + + tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); + if (major_int) + csr |= EDMA_TCD_CSR_INT_MAJOR; + + if (disable_req) + csr |= EDMA_TCD_CSR_D_REQ; + + if (enable_sg) + csr |= EDMA_TCD_CSR_E_SG; + + tcd->csr = cpu_to_le16(csr); +} + +static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, + int sg_len) +{ + struct fsl_edma_desc *fsl_desc; + int i; + + fsl_desc = kzalloc(sizeof(*fsl_desc) + + sizeof(struct fsl_edma_sw_tcd) * + sg_len, GFP_NOWAIT); + if (!fsl_desc) + return NULL; + + fsl_desc->echan = fsl_chan; + fsl_desc->n_tcds = sg_len; + for (i = 0; i < sg_len; i++) { + fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, + GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); + if (!fsl_desc->tcd[i].vtcd) + goto err; + } + return fsl_desc; + +err: + while (--i >= 0) + dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, + fsl_desc->tcd[i].ptcd); + kfree(fsl_desc); + return NULL; +} + +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + dma_addr_t dma_buf_next; + int sg_len, i; + u32 src_addr, dst_addr, last_sg, nbytes; + u16 soff, doff, iter; + + if (!is_slave_direction(fsl_chan->fsc.dir)) + return NULL; + + sg_len = buf_len / period_len; + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + fsl_desc->iscyclic = true; + + dma_buf_next = dma_addr; + nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; + iter = period_len / nbytes; + + for (i = 0; i < sg_len; i++) { + if (dma_buf_next >= dma_addr + buf_len) + dma_buf_next = dma_addr; + + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { + src_addr = dma_buf_next; + dst_addr = fsl_chan->fsc.dev_addr; + soff = fsl_chan->fsc.addr_width; + doff = 0; + } else { + src_addr = fsl_chan->fsc.dev_addr; + dst_addr = dma_buf_next; + soff = 0; + doff = fsl_chan->fsc.addr_width; + } + + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr, + fsl_chan->fsc.attr, soff, nbytes, 0, iter, + iter, doff, last_sg, true, false, true); + dma_buf_next += period_len; + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_dma_cyclic); + +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + struct scatterlist *sg; + u32 src_addr, dst_addr, last_sg, nbytes; + u16 soff, doff, iter; + int i; + + if (!is_slave_direction(fsl_chan->fsc.dir)) + return NULL; + + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + fsl_desc->iscyclic = false; + + nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; + for_each_sg(sgl, sg, sg_len, i) { + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { + src_addr = sg_dma_address(sg); + dst_addr = fsl_chan->fsc.dev_addr; + soff = fsl_chan->fsc.addr_width; + doff = 0; + } else { + src_addr = fsl_chan->fsc.dev_addr; + dst_addr = sg_dma_address(sg); + soff = 0; + doff = fsl_chan->fsc.addr_width; + } + + iter = sg_dma_len(sg) / nbytes; + if (i < sg_len - 1) { + last_sg = fsl_desc->tcd[(i + 1)].ptcd; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, + dst_addr, fsl_chan->fsc.attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + false, false, true); + } else { + last_sg = 0; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, + dst_addr, fsl_chan->fsc.attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + true, true, false); + } + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_slave_sg); + +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) +{ + struct virt_dma_desc *vdesc; + + vdesc = vchan_next_desc(&fsl_chan->vchan); + if (!vdesc) + return; + fsl_chan->edesc = to_fsl_edma_desc(vdesc); + fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; +} +EXPORT_SYMBOL_GPL(fsl_edma_xfer_desc); + +void fsl_edma_issue_pending(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + + if (unlikely(fsl_chan->pm_state != RUNNING)) { + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + /* cannot submit due to suspend */ + return; + } + + if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc) + fsl_edma_xfer_desc(fsl_chan); + + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_issue_pending); + +int fsl_edma_alloc_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + + fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, + sizeof(struct fsl_edma_hw_tcd), + 32, 0); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_alloc_chan_resources); + +void fsl_edma_free_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_edma_chan_mux(fsl_chan, 0, false); + fsl_chan->edesc = NULL; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + dma_pool_destroy(fsl_chan->tcd_pool); + fsl_chan->tcd_pool = NULL; +} +EXPORT_SYMBOL_GPL(fsl_edma_free_chan_resources); + +void fsl_edma_cleanup_vchan(struct dma_device *dmadev) +{ + struct fsl_edma_chan *chan, *_chan; + + list_for_each_entry_safe(chan, _chan, + &dmadev->channels, vchan.chan.device_node) { + list_del(&chan->vchan.chan.device_node); + tasklet_kill(&chan->vchan.task); + } +} +EXPORT_SYMBOL_GPL(fsl_edma_cleanup_vchan); + +MODULE_LICENSE("GPL v2"); diff --git a/drivers/dma/fsl-edma-common.h b/drivers/dma/fsl-edma-common.h new file mode 100644 index 000000000000..f3ea68e15b23 --- /dev/null +++ b/drivers/dma/fsl-edma-common.h @@ -0,0 +1,196 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2013-2014 Freescale Semiconductor, Inc. + * Copyright 2018 Angelo Dureghello + */ +#ifndef _FSL_EDMA_COMMON_H_ +#define _FSL_EDMA_COMMON_H_ + +#include "virt-dma.h" + +#define EDMA_CR 0x00 +#define EDMA_ES 0x04 +#define EDMA_ERQ 0x0C +#define EDMA_EEI 0x14 +#define EDMA_SERQ 0x1B +#define EDMA_CERQ 0x1A +#define EDMA_SEEI 0x19 +#define EDMA_CEEI 0x18 +#define EDMA_CINT 0x1F +#define EDMA_CERR 0x1E +#define EDMA_SSRT 0x1D +#define EDMA_CDNE 0x1C +#define EDMA_INTR 0x24 +#define EDMA_ERR 0x2C + +#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x)) +#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x)) +#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x)) +#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x)) +#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x)) +#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x)) +#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x)) +#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x)) +#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x)) +#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x)) +#define EDMA_TCD_CSR(x) (0x101C + 32 * (x)) +#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x)) +#define EDMA_TCD_BITER(x) (0x101E + 32 * (x)) + +#define EDMA_CR_EDBG BIT(1) +#define EDMA_CR_ERCA BIT(2) +#define EDMA_CR_ERGA BIT(3) +#define EDMA_CR_HOE BIT(4) +#define EDMA_CR_HALT BIT(5) +#define EDMA_CR_CLM BIT(6) +#define EDMA_CR_EMLM BIT(7) +#define EDMA_CR_ECX BIT(16) +#define EDMA_CR_CX BIT(17) + +#define EDMA_SEEI_SEEI(x) ((x) & 0x1F) +#define EDMA_CEEI_CEEI(x) ((x) & 0x1F) +#define EDMA_CINT_CINT(x) ((x) & 0x1F) +#define EDMA_CERR_CERR(x) ((x) & 0x1F) + +#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007)) +#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3) +#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8) +#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11) +#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000) +#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100) +#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200) +#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300) +#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500) +#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000) +#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001) +#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002) +#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003) +#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005) + +#define EDMA_TCD_SOFF_SOFF(x) (x) +#define EDMA_TCD_NBYTES_NBYTES(x) (x) +#define EDMA_TCD_SLAST_SLAST(x) (x) +#define EDMA_TCD_DADDR_DADDR(x) (x) +#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF) +#define EDMA_TCD_DOFF_DOFF(x) (x) +#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x) +#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF) + +#define EDMA_TCD_CSR_START BIT(0) +#define EDMA_TCD_CSR_INT_MAJOR BIT(1) +#define EDMA_TCD_CSR_INT_HALF BIT(2) +#define EDMA_TCD_CSR_D_REQ BIT(3) +#define EDMA_TCD_CSR_E_SG BIT(4) +#define EDMA_TCD_CSR_E_LINK BIT(5) +#define EDMA_TCD_CSR_ACTIVE BIT(6) +#define EDMA_TCD_CSR_DONE BIT(7) + +#define EDMAMUX_CHCFG_DIS 0x0 +#define EDMAMUX_CHCFG_ENBL 0x80 +#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F) + +#define DMAMUX_NR 2 + +#define FSL_EDMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ + BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_8_BYTES)) +enum fsl_edma_pm_state { + RUNNING = 0, + SUSPENDED, +}; + +struct fsl_edma_hw_tcd { + __le32 saddr; + __le16 soff; + __le16 attr; + __le32 nbytes; + __le32 slast; + __le32 daddr; + __le16 doff; + __le16 citer; + __le32 dlast_sga; + __le16 csr; + __le16 biter; +}; + +struct fsl_edma_sw_tcd { + dma_addr_t ptcd; + struct fsl_edma_hw_tcd *vtcd; +}; + +struct fsl_edma_slave_config { + enum dma_transfer_direction dir; + enum dma_slave_buswidth addr_width; + u32 dev_addr; + u32 burst; + u32 attr; +}; + +struct fsl_edma_chan { + struct virt_dma_chan vchan; + enum dma_status status; + enum fsl_edma_pm_state pm_state; + bool idle; + u32 slave_id; + struct fsl_edma_engine *edma; + struct fsl_edma_desc *edesc; + struct fsl_edma_slave_config fsc; + struct dma_pool *tcd_pool; +}; + +struct fsl_edma_desc { + struct virt_dma_desc vdesc; + struct fsl_edma_chan *echan; + bool iscyclic; + unsigned int n_tcds; + struct fsl_edma_sw_tcd tcd[]; +}; + +struct fsl_edma_engine { + struct dma_device dma_dev; + void __iomem *membase; + void __iomem *muxbase[DMAMUX_NR]; + struct clk *muxclk[DMAMUX_NR]; + struct mutex fsl_edma_mutex; + u32 n_chans; + int txirq; + int errirq; + bool big_endian; + struct fsl_edma_chan chans[]; +}; + +u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr); +void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr); +void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr); +void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr); + +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan); +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd); + +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan); +void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, + unsigned int slot, bool enable); +void fsl_edma_free_desc(struct virt_dma_desc *vdesc); +int fsl_edma_terminate_all(struct dma_chan *chan); +int fsl_edma_pause(struct dma_chan *chan); +int fsl_edma_resume(struct dma_chan *chan); +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg); +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate); +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags); +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context); +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan); +void fsl_edma_issue_pending(struct dma_chan *chan); +int fsl_edma_alloc_chan_resources(struct dma_chan *chan); +void fsl_edma_free_chan_resources(struct dma_chan *chan); +void fsl_edma_cleanup_vchan(struct dma_device *dmadev); + +#endif /* _FSL_EDMA_COMMON_H_ */ diff --git a/drivers/dma/fsl-edma.c b/drivers/dma/fsl-edma.c index c7568869284e..337ad0c0ea78 100644 --- a/drivers/dma/fsl-edma.c +++ b/drivers/dma/fsl-edma.c @@ -13,653 +13,16 @@ * option) any later version. */ -#include #include #include #include -#include -#include -#include -#include #include #include #include #include #include -#include "virt-dma.h" - -#define EDMA_CR 0x00 -#define EDMA_ES 0x04 -#define EDMA_ERQ 0x0C -#define EDMA_EEI 0x14 -#define EDMA_SERQ 0x1B -#define EDMA_CERQ 0x1A -#define EDMA_SEEI 0x19 -#define EDMA_CEEI 0x18 -#define EDMA_CINT 0x1F -#define EDMA_CERR 0x1E -#define EDMA_SSRT 0x1D -#define EDMA_CDNE 0x1C -#define EDMA_INTR 0x24 -#define EDMA_ERR 0x2C - -#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x)) -#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x)) -#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x)) -#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x)) -#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x)) -#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x)) -#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x)) -#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x)) -#define EDMA_TCD_CSR(x) (0x101C + 32 * (x)) -#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x)) -#define EDMA_TCD_BITER(x) (0x101E + 32 * (x)) - -#define EDMA_CR_EDBG BIT(1) -#define EDMA_CR_ERCA BIT(2) -#define EDMA_CR_ERGA BIT(3) -#define EDMA_CR_HOE BIT(4) -#define EDMA_CR_HALT BIT(5) -#define EDMA_CR_CLM BIT(6) -#define EDMA_CR_EMLM BIT(7) -#define EDMA_CR_ECX BIT(16) -#define EDMA_CR_CX BIT(17) - -#define EDMA_SEEI_SEEI(x) ((x) & 0x1F) -#define EDMA_CEEI_CEEI(x) ((x) & 0x1F) -#define EDMA_CINT_CINT(x) ((x) & 0x1F) -#define EDMA_CERR_CERR(x) ((x) & 0x1F) - -#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007)) -#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3) -#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8) -#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11) -#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100) -#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200) -#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300) -#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500) -#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001) -#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002) -#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003) -#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005) - -#define EDMA_TCD_SOFF_SOFF(x) (x) -#define EDMA_TCD_NBYTES_NBYTES(x) (x) -#define EDMA_TCD_SLAST_SLAST(x) (x) -#define EDMA_TCD_DADDR_DADDR(x) (x) -#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF) -#define EDMA_TCD_DOFF_DOFF(x) (x) -#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x) -#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF) - -#define EDMA_TCD_CSR_START BIT(0) -#define EDMA_TCD_CSR_INT_MAJOR BIT(1) -#define EDMA_TCD_CSR_INT_HALF BIT(2) -#define EDMA_TCD_CSR_D_REQ BIT(3) -#define EDMA_TCD_CSR_E_SG BIT(4) -#define EDMA_TCD_CSR_E_LINK BIT(5) -#define EDMA_TCD_CSR_ACTIVE BIT(6) -#define EDMA_TCD_CSR_DONE BIT(7) - -#define EDMAMUX_CHCFG_DIS 0x0 -#define EDMAMUX_CHCFG_ENBL 0x80 -#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F) - -#define DMAMUX_NR 2 - -#define FSL_EDMA_BUSWIDTHS BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ - BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_8_BYTES) -enum fsl_edma_pm_state { - RUNNING = 0, - SUSPENDED, -}; - -struct fsl_edma_hw_tcd { - __le32 saddr; - __le16 soff; - __le16 attr; - __le32 nbytes; - __le32 slast; - __le32 daddr; - __le16 doff; - __le16 citer; - __le32 dlast_sga; - __le16 csr; - __le16 biter; -}; - -struct fsl_edma_sw_tcd { - dma_addr_t ptcd; - struct fsl_edma_hw_tcd *vtcd; -}; - -struct fsl_edma_slave_config { - enum dma_transfer_direction dir; - enum dma_slave_buswidth addr_width; - u32 dev_addr; - u32 burst; - u32 attr; -}; - -struct fsl_edma_chan { - struct virt_dma_chan vchan; - enum dma_status status; - enum fsl_edma_pm_state pm_state; - bool idle; - u32 slave_id; - struct fsl_edma_engine *edma; - struct fsl_edma_desc *edesc; - struct fsl_edma_slave_config fsc; - struct dma_pool *tcd_pool; -}; - -struct fsl_edma_desc { - struct virt_dma_desc vdesc; - struct fsl_edma_chan *echan; - bool iscyclic; - unsigned int n_tcds; - struct fsl_edma_sw_tcd tcd[]; -}; - -struct fsl_edma_engine { - struct dma_device dma_dev; - void __iomem *membase; - void __iomem *muxbase[DMAMUX_NR]; - struct clk *muxclk[DMAMUX_NR]; - struct mutex fsl_edma_mutex; - u32 n_chans; - int txirq; - int errirq; - bool big_endian; - struct fsl_edma_chan chans[]; -}; - -/* - * R/W functions for big- or little-endian registers: - * The eDMA controller's endian is independent of the CPU core's endian. - * For the big-endian IP module, the offset for 8-bit or 16-bit registers - * should also be swapped opposite to that in little-endian IP. - */ - -static u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr) -{ - if (edma->big_endian) - return ioread32be(addr); - else - return ioread32(addr); -} - -static void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); - else - iowrite8(val, addr); -} - -static void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); - else - iowrite16(val, addr); -} - -static void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr) -{ - if (edma->big_endian) - iowrite32be(val, addr); - else - iowrite32(val, addr); -} - -static struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) -{ - return container_of(chan, struct fsl_edma_chan, vchan.chan); -} - -static struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) -{ - return container_of(vd, struct fsl_edma_desc, vdesc); -} - -static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI); - edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ); -} - -static void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ); - edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI); -} - -static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, - unsigned int slot, bool enable) -{ - u32 ch = fsl_chan->vchan.chan.chan_id; - void __iomem *muxaddr; - unsigned chans_per_mux, ch_off; - - chans_per_mux = fsl_chan->edma->n_chans / DMAMUX_NR; - ch_off = fsl_chan->vchan.chan.chan_id % chans_per_mux; - muxaddr = fsl_chan->edma->muxbase[ch / chans_per_mux]; - slot = EDMAMUX_CHCFG_SOURCE(slot); - - if (enable) - iowrite8(EDMAMUX_CHCFG_ENBL | slot, muxaddr + ch_off); - else - iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off); -} - -static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) -{ - switch (addr_width) { - case 1: - return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; - case 2: - return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; - case 4: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - case 8: - return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; - default: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - } -} - -static void fsl_edma_free_desc(struct virt_dma_desc *vdesc) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = to_fsl_edma_desc(vdesc); - for (i = 0; i < fsl_desc->n_tcds; i++) - dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); -} - -static int fsl_edma_terminate_all(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_chan->edesc = NULL; - fsl_chan->idle = true; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - return 0; -} - -static int fsl_edma_pause(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_disable_request(fsl_chan); - fsl_chan->status = DMA_PAUSED; - fsl_chan->idle = true; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_resume(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_slave_config(struct dma_chan *chan, - struct dma_slave_config *cfg) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->fsc.dir = cfg->direction; - if (cfg->direction == DMA_DEV_TO_MEM) { - fsl_chan->fsc.dev_addr = cfg->src_addr; - fsl_chan->fsc.addr_width = cfg->src_addr_width; - fsl_chan->fsc.burst = cfg->src_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width); - } else if (cfg->direction == DMA_MEM_TO_DEV) { - fsl_chan->fsc.dev_addr = cfg->dst_addr; - fsl_chan->fsc.addr_width = cfg->dst_addr_width; - fsl_chan->fsc.burst = cfg->dst_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width); - } else { - return -EINVAL; - } - return 0; -} - -static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, - struct virt_dma_desc *vdesc, bool in_progress) -{ - struct fsl_edma_desc *edesc = fsl_chan->edesc; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - enum dma_transfer_direction dir = fsl_chan->fsc.dir; - dma_addr_t cur_addr, dma_addr; - size_t len, size; - int i; - - /* calculate the total size in this desc */ - for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) - len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - - if (!in_progress) - return len; - - if (dir == DMA_MEM_TO_DEV) - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_SADDR(ch)); - else - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_DADDR(ch)); - - /* figure out the finished and calculate the residue */ - for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { - size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - if (dir == DMA_MEM_TO_DEV) - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); - else - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); - - len -= size; - if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { - len += dma_addr + size - cur_addr; - break; - } - } - - return len; -} - -static enum dma_status fsl_edma_tx_status(struct dma_chan *chan, - dma_cookie_t cookie, struct dma_tx_state *txstate) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct virt_dma_desc *vdesc; - enum dma_status status; - unsigned long flags; - - status = dma_cookie_status(chan, cookie, txstate); - if (status == DMA_COMPLETE) - return status; - - if (!txstate) - return fsl_chan->status; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); - if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, true); - else if (vdesc) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, false); - else - txstate->residue = 0; - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - return fsl_chan->status; -} - -static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, - struct fsl_edma_hw_tcd *tcd) -{ - struct fsl_edma_engine *edma = fsl_chan->edma; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - /* - * TCD parameters are stored in struct fsl_edma_hw_tcd in little - * endian format. However, we need to load the TCD registers in - * big- or little-endian obeying the eDMA engine model endian. - */ - edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch)); - edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch)); - edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch)); - - edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch)); - edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch)); - edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch)); - - edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch)); - - edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch)); -} - -static inline -void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst, - u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer, - u16 biter, u16 doff, u32 dlast_sga, bool major_int, - bool disable_req, bool enable_sg) -{ - u16 csr = 0; - - /* - * eDMA hardware SGs require the TCDs to be stored in little - * endian format irrespective of the register endian model. - * So we put the value in little endian in memory, waiting - * for fsl_edma_set_tcd_regs doing the swap. - */ - tcd->saddr = cpu_to_le32(src); - tcd->daddr = cpu_to_le32(dst); - - tcd->attr = cpu_to_le16(attr); - - tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff)); - - tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes)); - tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast)); - - tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); - tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff)); - - tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga)); - - tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); - if (major_int) - csr |= EDMA_TCD_CSR_INT_MAJOR; - - if (disable_req) - csr |= EDMA_TCD_CSR_D_REQ; - - if (enable_sg) - csr |= EDMA_TCD_CSR_E_SG; - - tcd->csr = cpu_to_le16(csr); -} - -static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, - int sg_len) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = kzalloc(sizeof(*fsl_desc) + sizeof(struct fsl_edma_sw_tcd) * sg_len, - GFP_NOWAIT); - if (!fsl_desc) - return NULL; - - fsl_desc->echan = fsl_chan; - fsl_desc->n_tcds = sg_len; - for (i = 0; i < sg_len; i++) { - fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, - GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); - if (!fsl_desc->tcd[i].vtcd) - goto err; - } - return fsl_desc; - -err: - while (--i >= 0) - dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); - return NULL; -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( - struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, - size_t period_len, enum dma_transfer_direction direction, - unsigned long flags) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - dma_addr_t dma_buf_next; - int sg_len, i; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - sg_len = buf_len / period_len; - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = true; - - dma_buf_next = dma_addr; - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - iter = period_len / nbytes; - - for (i = 0; i < sg_len; i++) { - if (dma_buf_next >= dma_addr + buf_len) - dma_buf_next = dma_addr; - - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = dma_buf_next; - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = dma_buf_next; - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr, - fsl_chan->fsc.attr, soff, nbytes, 0, iter, - iter, doff, last_sg, true, false, true); - dma_buf_next += period_len; - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( - struct dma_chan *chan, struct scatterlist *sgl, - unsigned int sg_len, enum dma_transfer_direction direction, - unsigned long flags, void *context) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - struct scatterlist *sg; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - int i; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = false; - - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - for_each_sg(sgl, sg, sg_len, i) { - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = sg_dma_address(sg); - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = sg_dma_address(sg); - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - iter = sg_dma_len(sg) / nbytes; - if (i < sg_len - 1) { - last_sg = fsl_desc->tcd[(i + 1)].ptcd; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - false, false, true); - } else { - last_sg = 0; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - true, true, false); - } - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) -{ - struct virt_dma_desc *vdesc; - - vdesc = vchan_next_desc(&fsl_chan->vchan); - if (!vdesc) - return; - fsl_chan->edesc = to_fsl_edma_desc(vdesc); - fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; -} +#include "fsl-edma-common.h" static irqreturn_t fsl_edma_tx_handler(int irq, void *dev_id) { @@ -730,25 +93,6 @@ static irqreturn_t fsl_edma_irq_handler(int irq, void *dev_id) return fsl_edma_err_handler(irq, dev_id); } -static void fsl_edma_issue_pending(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - - if (unlikely(fsl_chan->pm_state != RUNNING)) { - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - /* cannot submit due to suspend */ - return; - } - - if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc) - fsl_edma_xfer_desc(fsl_chan); - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); -} - static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, struct of_dma *ofdma) { @@ -781,34 +125,6 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, return NULL; } -static int fsl_edma_alloc_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, - sizeof(struct fsl_edma_hw_tcd), - 32, 0); - return 0; -} - -static void fsl_edma_free_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_edma_chan_mux(fsl_chan, 0, false); - fsl_chan->edesc = NULL; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - dma_pool_destroy(fsl_chan->tcd_pool); - fsl_chan->tcd_pool = NULL; -} - static int fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma) { @@ -995,17 +311,6 @@ static int fsl_edma_probe(struct platform_device *pdev) return 0; } -static void fsl_edma_cleanup_vchan(struct dma_device *dmadev) -{ - struct fsl_edma_chan *chan, *_chan; - - list_for_each_entry_safe(chan, _chan, - &dmadev->channels, vchan.chan.device_node) { - list_del(&chan->vchan.chan.device_node); - tasklet_kill(&chan->vchan.task); - } -} - static int fsl_edma_remove(struct platform_device *pdev) { struct device_node *np = pdev->dev.of_node; -- 2.18.0 From mboxrd@z Thu Jan 1 00:00:00 1970 From: angelo@sysam.it (Angelo Dureghello) Date: Fri, 3 Aug 2018 21:32:42 +0200 Subject: [PATCH v8 1/3] dmaengine: fsl-edma: extract common fsl-edma code (no changes in behavior intended) Message-ID: <20180803193244.12084-1-angelo@sysam.it> To: linux-arm-kernel@lists.infradead.org List-Id: linux-arm-kernel.lists.infradead.org This patch adds a new fsl-edma-common module to allow new mcf-edma module code to use most of the fsl-edma code. Signed-off-by: Angelo Dureghello --- Changes for v2: - patch splitted into 4 - add mcf-edma as minimal different parts from fsl-edma Changes for v3: none Changes for v4: - patch simplified from 4/4 into 2/2 - collecting all the mcf-edma-related changes Changes for v5: none Changes for v6: - adjusted comment header - fixed bit shift with BIT() - we need to free the interrupts at remove(), so removed all devm_ interrupt related calls Changes for v7: none Changes for v8: - patch rewritten from scratch, splitted into 3, common code isolated, minimal changes from the original Freescale code have been done. The patch has been tested with both Iris + Colibri Vybrid VF50 and stmark2/mcf54415 Coldfire boards. --- drivers/dma/Makefile | 2 +- drivers/dma/fsl-edma-common.c | 576 ++++++++++++++++++++++++++++ drivers/dma/fsl-edma-common.h | 196 ++++++++++ drivers/dma/fsl-edma.c | 697 +--------------------------------- 4 files changed, 774 insertions(+), 697 deletions(-) create mode 100644 drivers/dma/fsl-edma-common.c create mode 100644 drivers/dma/fsl-edma-common.h diff --git a/drivers/dma/Makefile b/drivers/dma/Makefile index 203a99d68315..66022f59fca4 100644 --- a/drivers/dma/Makefile +++ b/drivers/dma/Makefile @@ -31,7 +31,7 @@ obj-$(CONFIG_DW_AXI_DMAC) += dw-axi-dmac/ obj-$(CONFIG_DW_DMAC_CORE) += dw/ obj-$(CONFIG_EP93XX_DMA) += ep93xx_dma.o obj-$(CONFIG_FSL_DMA) += fsldma.o -obj-$(CONFIG_FSL_EDMA) += fsl-edma.o +obj-$(CONFIG_FSL_EDMA) += fsl-edma.o fsl-edma-common.o obj-$(CONFIG_FSL_RAID) += fsl_raid.o obj-$(CONFIG_HSU_DMA) += hsu/ obj-$(CONFIG_IMG_MDC_DMA) += img-mdc-dma.o diff --git a/drivers/dma/fsl-edma-common.c b/drivers/dma/fsl-edma-common.c new file mode 100644 index 000000000000..0ae7094f477a --- /dev/null +++ b/drivers/dma/fsl-edma-common.c @@ -0,0 +1,576 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// Copyright (c) 2013-2014 Freescale Semiconductor, Inc +// Copyright (c) 2017 Sysam, Angelo Dureghello + +#include +#include +#include + +#include "fsl-edma-common.h" + +/* + * R/W functions for big- or little-endian registers: + * The eDMA controller's endian is independent of the CPU core's endian. + * For the big-endian IP module, the offset for 8-bit or 16-bit registers + * should also be swapped opposite to that in little-endian IP. + */ +u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr) +{ + if (edma->big_endian) + return ioread32be(addr); + else + return ioread32(addr); +} +EXPORT_SYMBOL_GPL(edma_readl); + +void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); + else + iowrite8(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writeb); + +void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr) +{ + /* swap the reg offset for these in big-endian mode */ + if (edma->big_endian) + iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); + else + iowrite16(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writew); + +void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr) +{ + if (edma->big_endian) + iowrite32be(val, addr); + else + iowrite32(val, addr); +} +EXPORT_SYMBOL_GPL(edma_writel); + +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) +{ + return container_of(chan, struct fsl_edma_chan, vchan.chan); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_chan); + +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) +{ + return container_of(vd, struct fsl_edma_desc, vdesc); +} +EXPORT_SYMBOL_GPL(to_fsl_edma_desc); + +static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) +{ + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI); + edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ); +} + +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) +{ + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ); + edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI); +} +EXPORT_SYMBOL_GPL(fsl_edma_disable_request); + +void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, + unsigned int slot, bool enable) +{ + u32 ch = fsl_chan->vchan.chan.chan_id; + void __iomem *muxaddr; + unsigned int chans_per_mux, ch_off; + + chans_per_mux = fsl_chan->edma->n_chans / DMAMUX_NR; + ch_off = fsl_chan->vchan.chan.chan_id % chans_per_mux; + muxaddr = fsl_chan->edma->muxbase[ch / chans_per_mux]; + slot = EDMAMUX_CHCFG_SOURCE(slot); + + if (enable) + iowrite8(EDMAMUX_CHCFG_ENBL | slot, muxaddr + ch_off); + else + iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off); +} +EXPORT_SYMBOL_GPL(fsl_edma_chan_mux); + +static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) +{ + switch (addr_width) { + case 1: + return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; + case 2: + return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; + case 4: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + case 8: + return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; + default: + return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; + } +} + +void fsl_edma_free_desc(struct virt_dma_desc *vdesc) +{ + struct fsl_edma_desc *fsl_desc; + int i; + + fsl_desc = to_fsl_edma_desc(vdesc); + for (i = 0; i < fsl_desc->n_tcds; i++) + dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd, + fsl_desc->tcd[i].ptcd); + kfree(fsl_desc); +} +EXPORT_SYMBOL_GPL(fsl_edma_free_desc); + +int fsl_edma_terminate_all(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_chan->edesc = NULL; + fsl_chan->idle = true; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_terminate_all); + +int fsl_edma_pause(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_disable_request(fsl_chan); + fsl_chan->status = DMA_PAUSED; + fsl_chan->idle = true; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_pause); + +int fsl_edma_resume(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + if (fsl_chan->edesc) { + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; + } + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_resume); + +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + + fsl_chan->fsc.dir = cfg->direction; + if (cfg->direction == DMA_DEV_TO_MEM) { + fsl_chan->fsc.dev_addr = cfg->src_addr; + fsl_chan->fsc.addr_width = cfg->src_addr_width; + fsl_chan->fsc.burst = cfg->src_maxburst; + fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width); + } else if (cfg->direction == DMA_MEM_TO_DEV) { + fsl_chan->fsc.dev_addr = cfg->dst_addr; + fsl_chan->fsc.addr_width = cfg->dst_addr_width; + fsl_chan->fsc.burst = cfg->dst_maxburst; + fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width); + } else + return -EINVAL; + + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_slave_config); + +static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, + struct virt_dma_desc *vdesc, bool in_progress) +{ + struct fsl_edma_desc *edesc = fsl_chan->edesc; + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + enum dma_transfer_direction dir = fsl_chan->fsc.dir; + dma_addr_t cur_addr, dma_addr; + size_t len, size; + int i; + + /* calculate the total size in this desc */ + for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) + len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + + if (!in_progress) + return len; + + if (dir == DMA_MEM_TO_DEV) + cur_addr = edma_readl( + fsl_chan->edma, addr + EDMA_TCD_SADDR(ch)); + else + cur_addr = edma_readl( + fsl_chan->edma, addr + EDMA_TCD_DADDR(ch)); + + /* figure out the finished and calculate the residue */ + for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { + size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) + * le16_to_cpu(edesc->tcd[i].vtcd->biter); + if (dir == DMA_MEM_TO_DEV) + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); + else + dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); + + len -= size; + if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { + len += dma_addr + size - cur_addr; + break; + } + } + + return len; +} + +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct virt_dma_desc *vdesc; + enum dma_status status; + unsigned long flags; + + status = dma_cookie_status(chan, cookie, txstate); + if (status == DMA_COMPLETE) + return status; + + if (!txstate) + return fsl_chan->status; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); + if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, true); + else if (vdesc) + txstate->residue = + fsl_edma_desc_residue(fsl_chan, vdesc, false); + else + txstate->residue = 0; + + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + return fsl_chan->status; +} +EXPORT_SYMBOL_GPL(fsl_edma_tx_status); + +static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, + struct fsl_edma_hw_tcd *tcd) +{ + struct fsl_edma_engine *edma = fsl_chan->edma; + void __iomem *addr = fsl_chan->edma->membase; + u32 ch = fsl_chan->vchan.chan.chan_id; + + /* + * TCD parameters are stored in struct fsl_edma_hw_tcd in little + * endian format. However, we need to load the TCD registers in + * big- or little-endian obeying the eDMA engine model endian. + */ + edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch)); + edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch)); + edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch)); + + edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch)); + edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch)); + + edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch)); + edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch)); + + edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch)); + edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch)); + edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch)); + + edma_writel(edma, + le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch)); + + edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch)); +} + +static inline +void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst, + u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer, + u16 biter, u16 doff, u32 dlast_sga, bool major_int, + bool disable_req, bool enable_sg) +{ + u16 csr = 0; + + /* + * eDMA hardware SGs require the TCDs to be stored in little + * endian format irrespective of the register endian model. + * So we put the value in little endian in memory, waiting + * for fsl_edma_set_tcd_regs doing the swap. + */ + tcd->saddr = cpu_to_le32(src); + tcd->daddr = cpu_to_le32(dst); + + tcd->attr = cpu_to_le16(attr); + + tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff)); + + tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes)); + tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast)); + + tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); + tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff)); + + tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga)); + + tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); + if (major_int) + csr |= EDMA_TCD_CSR_INT_MAJOR; + + if (disable_req) + csr |= EDMA_TCD_CSR_D_REQ; + + if (enable_sg) + csr |= EDMA_TCD_CSR_E_SG; + + tcd->csr = cpu_to_le16(csr); +} + +static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, + int sg_len) +{ + struct fsl_edma_desc *fsl_desc; + int i; + + fsl_desc = kzalloc(sizeof(*fsl_desc) + + sizeof(struct fsl_edma_sw_tcd) * + sg_len, GFP_NOWAIT); + if (!fsl_desc) + return NULL; + + fsl_desc->echan = fsl_chan; + fsl_desc->n_tcds = sg_len; + for (i = 0; i < sg_len; i++) { + fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, + GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); + if (!fsl_desc->tcd[i].vtcd) + goto err; + } + return fsl_desc; + +err: + while (--i >= 0) + dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, + fsl_desc->tcd[i].ptcd); + kfree(fsl_desc); + return NULL; +} + +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + dma_addr_t dma_buf_next; + int sg_len, i; + u32 src_addr, dst_addr, last_sg, nbytes; + u16 soff, doff, iter; + + if (!is_slave_direction(fsl_chan->fsc.dir)) + return NULL; + + sg_len = buf_len / period_len; + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + fsl_desc->iscyclic = true; + + dma_buf_next = dma_addr; + nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; + iter = period_len / nbytes; + + for (i = 0; i < sg_len; i++) { + if (dma_buf_next >= dma_addr + buf_len) + dma_buf_next = dma_addr; + + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { + src_addr = dma_buf_next; + dst_addr = fsl_chan->fsc.dev_addr; + soff = fsl_chan->fsc.addr_width; + doff = 0; + } else { + src_addr = fsl_chan->fsc.dev_addr; + dst_addr = dma_buf_next; + soff = 0; + doff = fsl_chan->fsc.addr_width; + } + + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr, + fsl_chan->fsc.attr, soff, nbytes, 0, iter, + iter, doff, last_sg, true, false, true); + dma_buf_next += period_len; + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_dma_cyclic); + +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + struct fsl_edma_desc *fsl_desc; + struct scatterlist *sg; + u32 src_addr, dst_addr, last_sg, nbytes; + u16 soff, doff, iter; + int i; + + if (!is_slave_direction(fsl_chan->fsc.dir)) + return NULL; + + fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); + if (!fsl_desc) + return NULL; + fsl_desc->iscyclic = false; + + nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; + for_each_sg(sgl, sg, sg_len, i) { + /* get next sg's physical address */ + last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; + + if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { + src_addr = sg_dma_address(sg); + dst_addr = fsl_chan->fsc.dev_addr; + soff = fsl_chan->fsc.addr_width; + doff = 0; + } else { + src_addr = fsl_chan->fsc.dev_addr; + dst_addr = sg_dma_address(sg); + soff = 0; + doff = fsl_chan->fsc.addr_width; + } + + iter = sg_dma_len(sg) / nbytes; + if (i < sg_len - 1) { + last_sg = fsl_desc->tcd[(i + 1)].ptcd; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, + dst_addr, fsl_chan->fsc.attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + false, false, true); + } else { + last_sg = 0; + fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, + dst_addr, fsl_chan->fsc.attr, soff, + nbytes, 0, iter, iter, doff, last_sg, + true, true, false); + } + } + + return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_prep_slave_sg); + +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) +{ + struct virt_dma_desc *vdesc; + + vdesc = vchan_next_desc(&fsl_chan->vchan); + if (!vdesc) + return; + fsl_chan->edesc = to_fsl_edma_desc(vdesc); + fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); + fsl_edma_enable_request(fsl_chan); + fsl_chan->status = DMA_IN_PROGRESS; + fsl_chan->idle = false; +} +EXPORT_SYMBOL_GPL(fsl_edma_xfer_desc); + +void fsl_edma_issue_pending(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + + if (unlikely(fsl_chan->pm_state != RUNNING)) { + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + /* cannot submit due to suspend */ + return; + } + + if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc) + fsl_edma_xfer_desc(fsl_chan); + + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); +} +EXPORT_SYMBOL_GPL(fsl_edma_issue_pending); + +int fsl_edma_alloc_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + + fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, + sizeof(struct fsl_edma_hw_tcd), + 32, 0); + return 0; +} +EXPORT_SYMBOL_GPL(fsl_edma_alloc_chan_resources); + +void fsl_edma_free_chan_resources(struct dma_chan *chan) +{ + struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&fsl_chan->vchan.lock, flags); + fsl_edma_disable_request(fsl_chan); + fsl_edma_chan_mux(fsl_chan, 0, false); + fsl_chan->edesc = NULL; + vchan_get_all_descriptors(&fsl_chan->vchan, &head); + spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); + + vchan_dma_desc_free_list(&fsl_chan->vchan, &head); + dma_pool_destroy(fsl_chan->tcd_pool); + fsl_chan->tcd_pool = NULL; +} +EXPORT_SYMBOL_GPL(fsl_edma_free_chan_resources); + +void fsl_edma_cleanup_vchan(struct dma_device *dmadev) +{ + struct fsl_edma_chan *chan, *_chan; + + list_for_each_entry_safe(chan, _chan, + &dmadev->channels, vchan.chan.device_node) { + list_del(&chan->vchan.chan.device_node); + tasklet_kill(&chan->vchan.task); + } +} +EXPORT_SYMBOL_GPL(fsl_edma_cleanup_vchan); + +MODULE_LICENSE("GPL v2"); diff --git a/drivers/dma/fsl-edma-common.h b/drivers/dma/fsl-edma-common.h new file mode 100644 index 000000000000..f3ea68e15b23 --- /dev/null +++ b/drivers/dma/fsl-edma-common.h @@ -0,0 +1,196 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* + * Copyright 2013-2014 Freescale Semiconductor, Inc. + * Copyright 2018 Angelo Dureghello + */ +#ifndef _FSL_EDMA_COMMON_H_ +#define _FSL_EDMA_COMMON_H_ + +#include "virt-dma.h" + +#define EDMA_CR 0x00 +#define EDMA_ES 0x04 +#define EDMA_ERQ 0x0C +#define EDMA_EEI 0x14 +#define EDMA_SERQ 0x1B +#define EDMA_CERQ 0x1A +#define EDMA_SEEI 0x19 +#define EDMA_CEEI 0x18 +#define EDMA_CINT 0x1F +#define EDMA_CERR 0x1E +#define EDMA_SSRT 0x1D +#define EDMA_CDNE 0x1C +#define EDMA_INTR 0x24 +#define EDMA_ERR 0x2C + +#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x)) +#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x)) +#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x)) +#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x)) +#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x)) +#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x)) +#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x)) +#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x)) +#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x)) +#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x)) +#define EDMA_TCD_CSR(x) (0x101C + 32 * (x)) +#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x)) +#define EDMA_TCD_BITER(x) (0x101E + 32 * (x)) + +#define EDMA_CR_EDBG BIT(1) +#define EDMA_CR_ERCA BIT(2) +#define EDMA_CR_ERGA BIT(3) +#define EDMA_CR_HOE BIT(4) +#define EDMA_CR_HALT BIT(5) +#define EDMA_CR_CLM BIT(6) +#define EDMA_CR_EMLM BIT(7) +#define EDMA_CR_ECX BIT(16) +#define EDMA_CR_CX BIT(17) + +#define EDMA_SEEI_SEEI(x) ((x) & 0x1F) +#define EDMA_CEEI_CEEI(x) ((x) & 0x1F) +#define EDMA_CINT_CINT(x) ((x) & 0x1F) +#define EDMA_CERR_CERR(x) ((x) & 0x1F) + +#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007)) +#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3) +#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8) +#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11) +#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000) +#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100) +#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200) +#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300) +#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500) +#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000) +#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001) +#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002) +#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003) +#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005) + +#define EDMA_TCD_SOFF_SOFF(x) (x) +#define EDMA_TCD_NBYTES_NBYTES(x) (x) +#define EDMA_TCD_SLAST_SLAST(x) (x) +#define EDMA_TCD_DADDR_DADDR(x) (x) +#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF) +#define EDMA_TCD_DOFF_DOFF(x) (x) +#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x) +#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF) + +#define EDMA_TCD_CSR_START BIT(0) +#define EDMA_TCD_CSR_INT_MAJOR BIT(1) +#define EDMA_TCD_CSR_INT_HALF BIT(2) +#define EDMA_TCD_CSR_D_REQ BIT(3) +#define EDMA_TCD_CSR_E_SG BIT(4) +#define EDMA_TCD_CSR_E_LINK BIT(5) +#define EDMA_TCD_CSR_ACTIVE BIT(6) +#define EDMA_TCD_CSR_DONE BIT(7) + +#define EDMAMUX_CHCFG_DIS 0x0 +#define EDMAMUX_CHCFG_ENBL 0x80 +#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F) + +#define DMAMUX_NR 2 + +#define FSL_EDMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ + BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ + BIT(DMA_SLAVE_BUSWIDTH_8_BYTES)) +enum fsl_edma_pm_state { + RUNNING = 0, + SUSPENDED, +}; + +struct fsl_edma_hw_tcd { + __le32 saddr; + __le16 soff; + __le16 attr; + __le32 nbytes; + __le32 slast; + __le32 daddr; + __le16 doff; + __le16 citer; + __le32 dlast_sga; + __le16 csr; + __le16 biter; +}; + +struct fsl_edma_sw_tcd { + dma_addr_t ptcd; + struct fsl_edma_hw_tcd *vtcd; +}; + +struct fsl_edma_slave_config { + enum dma_transfer_direction dir; + enum dma_slave_buswidth addr_width; + u32 dev_addr; + u32 burst; + u32 attr; +}; + +struct fsl_edma_chan { + struct virt_dma_chan vchan; + enum dma_status status; + enum fsl_edma_pm_state pm_state; + bool idle; + u32 slave_id; + struct fsl_edma_engine *edma; + struct fsl_edma_desc *edesc; + struct fsl_edma_slave_config fsc; + struct dma_pool *tcd_pool; +}; + +struct fsl_edma_desc { + struct virt_dma_desc vdesc; + struct fsl_edma_chan *echan; + bool iscyclic; + unsigned int n_tcds; + struct fsl_edma_sw_tcd tcd[]; +}; + +struct fsl_edma_engine { + struct dma_device dma_dev; + void __iomem *membase; + void __iomem *muxbase[DMAMUX_NR]; + struct clk *muxclk[DMAMUX_NR]; + struct mutex fsl_edma_mutex; + u32 n_chans; + int txirq; + int errirq; + bool big_endian; + struct fsl_edma_chan chans[]; +}; + +u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr); +void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr); +void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr); +void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr); + +struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan); +struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd); + +void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan); +void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, + unsigned int slot, bool enable); +void fsl_edma_free_desc(struct virt_dma_desc *vdesc); +int fsl_edma_terminate_all(struct dma_chan *chan); +int fsl_edma_pause(struct dma_chan *chan); +int fsl_edma_resume(struct dma_chan *chan); +int fsl_edma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg); +enum dma_status fsl_edma_tx_status(struct dma_chan *chan, + dma_cookie_t cookie, struct dma_tx_state *txstate); +struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( + struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, + size_t period_len, enum dma_transfer_direction direction, + unsigned long flags); +struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( + struct dma_chan *chan, struct scatterlist *sgl, + unsigned int sg_len, enum dma_transfer_direction direction, + unsigned long flags, void *context); +void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan); +void fsl_edma_issue_pending(struct dma_chan *chan); +int fsl_edma_alloc_chan_resources(struct dma_chan *chan); +void fsl_edma_free_chan_resources(struct dma_chan *chan); +void fsl_edma_cleanup_vchan(struct dma_device *dmadev); + +#endif /* _FSL_EDMA_COMMON_H_ */ diff --git a/drivers/dma/fsl-edma.c b/drivers/dma/fsl-edma.c index c7568869284e..337ad0c0ea78 100644 --- a/drivers/dma/fsl-edma.c +++ b/drivers/dma/fsl-edma.c @@ -13,653 +13,16 @@ * option) any later version. */ -#include #include #include #include -#include -#include -#include -#include #include #include #include #include #include -#include "virt-dma.h" - -#define EDMA_CR 0x00 -#define EDMA_ES 0x04 -#define EDMA_ERQ 0x0C -#define EDMA_EEI 0x14 -#define EDMA_SERQ 0x1B -#define EDMA_CERQ 0x1A -#define EDMA_SEEI 0x19 -#define EDMA_CEEI 0x18 -#define EDMA_CINT 0x1F -#define EDMA_CERR 0x1E -#define EDMA_SSRT 0x1D -#define EDMA_CDNE 0x1C -#define EDMA_INTR 0x24 -#define EDMA_ERR 0x2C - -#define EDMA_TCD_SADDR(x) (0x1000 + 32 * (x)) -#define EDMA_TCD_SOFF(x) (0x1004 + 32 * (x)) -#define EDMA_TCD_ATTR(x) (0x1006 + 32 * (x)) -#define EDMA_TCD_NBYTES(x) (0x1008 + 32 * (x)) -#define EDMA_TCD_SLAST(x) (0x100C + 32 * (x)) -#define EDMA_TCD_DADDR(x) (0x1010 + 32 * (x)) -#define EDMA_TCD_DOFF(x) (0x1014 + 32 * (x)) -#define EDMA_TCD_CITER_ELINK(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_CITER(x) (0x1016 + 32 * (x)) -#define EDMA_TCD_DLAST_SGA(x) (0x1018 + 32 * (x)) -#define EDMA_TCD_CSR(x) (0x101C + 32 * (x)) -#define EDMA_TCD_BITER_ELINK(x) (0x101E + 32 * (x)) -#define EDMA_TCD_BITER(x) (0x101E + 32 * (x)) - -#define EDMA_CR_EDBG BIT(1) -#define EDMA_CR_ERCA BIT(2) -#define EDMA_CR_ERGA BIT(3) -#define EDMA_CR_HOE BIT(4) -#define EDMA_CR_HALT BIT(5) -#define EDMA_CR_CLM BIT(6) -#define EDMA_CR_EMLM BIT(7) -#define EDMA_CR_ECX BIT(16) -#define EDMA_CR_CX BIT(17) - -#define EDMA_SEEI_SEEI(x) ((x) & 0x1F) -#define EDMA_CEEI_CEEI(x) ((x) & 0x1F) -#define EDMA_CINT_CINT(x) ((x) & 0x1F) -#define EDMA_CERR_CERR(x) ((x) & 0x1F) - -#define EDMA_TCD_ATTR_DSIZE(x) (((x) & 0x0007)) -#define EDMA_TCD_ATTR_DMOD(x) (((x) & 0x001F) << 3) -#define EDMA_TCD_ATTR_SSIZE(x) (((x) & 0x0007) << 8) -#define EDMA_TCD_ATTR_SMOD(x) (((x) & 0x001F) << 11) -#define EDMA_TCD_ATTR_SSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_SSIZE_16BIT (0x0100) -#define EDMA_TCD_ATTR_SSIZE_32BIT (0x0200) -#define EDMA_TCD_ATTR_SSIZE_64BIT (0x0300) -#define EDMA_TCD_ATTR_SSIZE_32BYTE (0x0500) -#define EDMA_TCD_ATTR_DSIZE_8BIT (0x0000) -#define EDMA_TCD_ATTR_DSIZE_16BIT (0x0001) -#define EDMA_TCD_ATTR_DSIZE_32BIT (0x0002) -#define EDMA_TCD_ATTR_DSIZE_64BIT (0x0003) -#define EDMA_TCD_ATTR_DSIZE_32BYTE (0x0005) - -#define EDMA_TCD_SOFF_SOFF(x) (x) -#define EDMA_TCD_NBYTES_NBYTES(x) (x) -#define EDMA_TCD_SLAST_SLAST(x) (x) -#define EDMA_TCD_DADDR_DADDR(x) (x) -#define EDMA_TCD_CITER_CITER(x) ((x) & 0x7FFF) -#define EDMA_TCD_DOFF_DOFF(x) (x) -#define EDMA_TCD_DLAST_SGA_DLAST_SGA(x) (x) -#define EDMA_TCD_BITER_BITER(x) ((x) & 0x7FFF) - -#define EDMA_TCD_CSR_START BIT(0) -#define EDMA_TCD_CSR_INT_MAJOR BIT(1) -#define EDMA_TCD_CSR_INT_HALF BIT(2) -#define EDMA_TCD_CSR_D_REQ BIT(3) -#define EDMA_TCD_CSR_E_SG BIT(4) -#define EDMA_TCD_CSR_E_LINK BIT(5) -#define EDMA_TCD_CSR_ACTIVE BIT(6) -#define EDMA_TCD_CSR_DONE BIT(7) - -#define EDMAMUX_CHCFG_DIS 0x0 -#define EDMAMUX_CHCFG_ENBL 0x80 -#define EDMAMUX_CHCFG_SOURCE(n) ((n) & 0x3F) - -#define DMAMUX_NR 2 - -#define FSL_EDMA_BUSWIDTHS BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \ - BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_4_BYTES) | \ - BIT(DMA_SLAVE_BUSWIDTH_8_BYTES) -enum fsl_edma_pm_state { - RUNNING = 0, - SUSPENDED, -}; - -struct fsl_edma_hw_tcd { - __le32 saddr; - __le16 soff; - __le16 attr; - __le32 nbytes; - __le32 slast; - __le32 daddr; - __le16 doff; - __le16 citer; - __le32 dlast_sga; - __le16 csr; - __le16 biter; -}; - -struct fsl_edma_sw_tcd { - dma_addr_t ptcd; - struct fsl_edma_hw_tcd *vtcd; -}; - -struct fsl_edma_slave_config { - enum dma_transfer_direction dir; - enum dma_slave_buswidth addr_width; - u32 dev_addr; - u32 burst; - u32 attr; -}; - -struct fsl_edma_chan { - struct virt_dma_chan vchan; - enum dma_status status; - enum fsl_edma_pm_state pm_state; - bool idle; - u32 slave_id; - struct fsl_edma_engine *edma; - struct fsl_edma_desc *edesc; - struct fsl_edma_slave_config fsc; - struct dma_pool *tcd_pool; -}; - -struct fsl_edma_desc { - struct virt_dma_desc vdesc; - struct fsl_edma_chan *echan; - bool iscyclic; - unsigned int n_tcds; - struct fsl_edma_sw_tcd tcd[]; -}; - -struct fsl_edma_engine { - struct dma_device dma_dev; - void __iomem *membase; - void __iomem *muxbase[DMAMUX_NR]; - struct clk *muxclk[DMAMUX_NR]; - struct mutex fsl_edma_mutex; - u32 n_chans; - int txirq; - int errirq; - bool big_endian; - struct fsl_edma_chan chans[]; -}; - -/* - * R/W functions for big- or little-endian registers: - * The eDMA controller's endian is independent of the CPU core's endian. - * For the big-endian IP module, the offset for 8-bit or 16-bit registers - * should also be swapped opposite to that in little-endian IP. - */ - -static u32 edma_readl(struct fsl_edma_engine *edma, void __iomem *addr) -{ - if (edma->big_endian) - return ioread32be(addr); - else - return ioread32(addr); -} - -static void edma_writeb(struct fsl_edma_engine *edma, u8 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite8(val, (void __iomem *)((unsigned long)addr ^ 0x3)); - else - iowrite8(val, addr); -} - -static void edma_writew(struct fsl_edma_engine *edma, u16 val, void __iomem *addr) -{ - /* swap the reg offset for these in big-endian mode */ - if (edma->big_endian) - iowrite16be(val, (void __iomem *)((unsigned long)addr ^ 0x2)); - else - iowrite16(val, addr); -} - -static void edma_writel(struct fsl_edma_engine *edma, u32 val, void __iomem *addr) -{ - if (edma->big_endian) - iowrite32be(val, addr); - else - iowrite32(val, addr); -} - -static struct fsl_edma_chan *to_fsl_edma_chan(struct dma_chan *chan) -{ - return container_of(chan, struct fsl_edma_chan, vchan.chan); -} - -static struct fsl_edma_desc *to_fsl_edma_desc(struct virt_dma_desc *vd) -{ - return container_of(vd, struct fsl_edma_desc, vdesc); -} - -static void fsl_edma_enable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, EDMA_SEEI_SEEI(ch), addr + EDMA_SEEI); - edma_writeb(fsl_chan->edma, ch, addr + EDMA_SERQ); -} - -static void fsl_edma_disable_request(struct fsl_edma_chan *fsl_chan) -{ - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - edma_writeb(fsl_chan->edma, ch, addr + EDMA_CERQ); - edma_writeb(fsl_chan->edma, EDMA_CEEI_CEEI(ch), addr + EDMA_CEEI); -} - -static void fsl_edma_chan_mux(struct fsl_edma_chan *fsl_chan, - unsigned int slot, bool enable) -{ - u32 ch = fsl_chan->vchan.chan.chan_id; - void __iomem *muxaddr; - unsigned chans_per_mux, ch_off; - - chans_per_mux = fsl_chan->edma->n_chans / DMAMUX_NR; - ch_off = fsl_chan->vchan.chan.chan_id % chans_per_mux; - muxaddr = fsl_chan->edma->muxbase[ch / chans_per_mux]; - slot = EDMAMUX_CHCFG_SOURCE(slot); - - if (enable) - iowrite8(EDMAMUX_CHCFG_ENBL | slot, muxaddr + ch_off); - else - iowrite8(EDMAMUX_CHCFG_DIS, muxaddr + ch_off); -} - -static unsigned int fsl_edma_get_tcd_attr(enum dma_slave_buswidth addr_width) -{ - switch (addr_width) { - case 1: - return EDMA_TCD_ATTR_SSIZE_8BIT | EDMA_TCD_ATTR_DSIZE_8BIT; - case 2: - return EDMA_TCD_ATTR_SSIZE_16BIT | EDMA_TCD_ATTR_DSIZE_16BIT; - case 4: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - case 8: - return EDMA_TCD_ATTR_SSIZE_64BIT | EDMA_TCD_ATTR_DSIZE_64BIT; - default: - return EDMA_TCD_ATTR_SSIZE_32BIT | EDMA_TCD_ATTR_DSIZE_32BIT; - } -} - -static void fsl_edma_free_desc(struct virt_dma_desc *vdesc) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = to_fsl_edma_desc(vdesc); - for (i = 0; i < fsl_desc->n_tcds; i++) - dma_pool_free(fsl_desc->echan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); -} - -static int fsl_edma_terminate_all(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_chan->edesc = NULL; - fsl_chan->idle = true; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - return 0; -} - -static int fsl_edma_pause(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_disable_request(fsl_chan); - fsl_chan->status = DMA_PAUSED; - fsl_chan->idle = true; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_resume(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - if (fsl_chan->edesc) { - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; - } - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - return 0; -} - -static int fsl_edma_slave_config(struct dma_chan *chan, - struct dma_slave_config *cfg) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->fsc.dir = cfg->direction; - if (cfg->direction == DMA_DEV_TO_MEM) { - fsl_chan->fsc.dev_addr = cfg->src_addr; - fsl_chan->fsc.addr_width = cfg->src_addr_width; - fsl_chan->fsc.burst = cfg->src_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->src_addr_width); - } else if (cfg->direction == DMA_MEM_TO_DEV) { - fsl_chan->fsc.dev_addr = cfg->dst_addr; - fsl_chan->fsc.addr_width = cfg->dst_addr_width; - fsl_chan->fsc.burst = cfg->dst_maxburst; - fsl_chan->fsc.attr = fsl_edma_get_tcd_attr(cfg->dst_addr_width); - } else { - return -EINVAL; - } - return 0; -} - -static size_t fsl_edma_desc_residue(struct fsl_edma_chan *fsl_chan, - struct virt_dma_desc *vdesc, bool in_progress) -{ - struct fsl_edma_desc *edesc = fsl_chan->edesc; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - enum dma_transfer_direction dir = fsl_chan->fsc.dir; - dma_addr_t cur_addr, dma_addr; - size_t len, size; - int i; - - /* calculate the total size in this desc */ - for (len = i = 0; i < fsl_chan->edesc->n_tcds; i++) - len += le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - - if (!in_progress) - return len; - - if (dir == DMA_MEM_TO_DEV) - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_SADDR(ch)); - else - cur_addr = edma_readl(fsl_chan->edma, addr + EDMA_TCD_DADDR(ch)); - - /* figure out the finished and calculate the residue */ - for (i = 0; i < fsl_chan->edesc->n_tcds; i++) { - size = le32_to_cpu(edesc->tcd[i].vtcd->nbytes) - * le16_to_cpu(edesc->tcd[i].vtcd->biter); - if (dir == DMA_MEM_TO_DEV) - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->saddr); - else - dma_addr = le32_to_cpu(edesc->tcd[i].vtcd->daddr); - - len -= size; - if (cur_addr >= dma_addr && cur_addr < dma_addr + size) { - len += dma_addr + size - cur_addr; - break; - } - } - - return len; -} - -static enum dma_status fsl_edma_tx_status(struct dma_chan *chan, - dma_cookie_t cookie, struct dma_tx_state *txstate) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct virt_dma_desc *vdesc; - enum dma_status status; - unsigned long flags; - - status = dma_cookie_status(chan, cookie, txstate); - if (status == DMA_COMPLETE) - return status; - - if (!txstate) - return fsl_chan->status; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - vdesc = vchan_find_desc(&fsl_chan->vchan, cookie); - if (fsl_chan->edesc && cookie == fsl_chan->edesc->vdesc.tx.cookie) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, true); - else if (vdesc) - txstate->residue = fsl_edma_desc_residue(fsl_chan, vdesc, false); - else - txstate->residue = 0; - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - return fsl_chan->status; -} - -static void fsl_edma_set_tcd_regs(struct fsl_edma_chan *fsl_chan, - struct fsl_edma_hw_tcd *tcd) -{ - struct fsl_edma_engine *edma = fsl_chan->edma; - void __iomem *addr = fsl_chan->edma->membase; - u32 ch = fsl_chan->vchan.chan.chan_id; - - /* - * TCD parameters are stored in struct fsl_edma_hw_tcd in little - * endian format. However, we need to load the TCD registers in - * big- or little-endian obeying the eDMA engine model endian. - */ - edma_writew(edma, 0, addr + EDMA_TCD_CSR(ch)); - edma_writel(edma, le32_to_cpu(tcd->saddr), addr + EDMA_TCD_SADDR(ch)); - edma_writel(edma, le32_to_cpu(tcd->daddr), addr + EDMA_TCD_DADDR(ch)); - - edma_writew(edma, le16_to_cpu(tcd->attr), addr + EDMA_TCD_ATTR(ch)); - edma_writew(edma, le16_to_cpu(tcd->soff), addr + EDMA_TCD_SOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->nbytes), addr + EDMA_TCD_NBYTES(ch)); - edma_writel(edma, le32_to_cpu(tcd->slast), addr + EDMA_TCD_SLAST(ch)); - - edma_writew(edma, le16_to_cpu(tcd->citer), addr + EDMA_TCD_CITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->biter), addr + EDMA_TCD_BITER(ch)); - edma_writew(edma, le16_to_cpu(tcd->doff), addr + EDMA_TCD_DOFF(ch)); - - edma_writel(edma, le32_to_cpu(tcd->dlast_sga), addr + EDMA_TCD_DLAST_SGA(ch)); - - edma_writew(edma, le16_to_cpu(tcd->csr), addr + EDMA_TCD_CSR(ch)); -} - -static inline -void fsl_edma_fill_tcd(struct fsl_edma_hw_tcd *tcd, u32 src, u32 dst, - u16 attr, u16 soff, u32 nbytes, u32 slast, u16 citer, - u16 biter, u16 doff, u32 dlast_sga, bool major_int, - bool disable_req, bool enable_sg) -{ - u16 csr = 0; - - /* - * eDMA hardware SGs require the TCDs to be stored in little - * endian format irrespective of the register endian model. - * So we put the value in little endian in memory, waiting - * for fsl_edma_set_tcd_regs doing the swap. - */ - tcd->saddr = cpu_to_le32(src); - tcd->daddr = cpu_to_le32(dst); - - tcd->attr = cpu_to_le16(attr); - - tcd->soff = cpu_to_le16(EDMA_TCD_SOFF_SOFF(soff)); - - tcd->nbytes = cpu_to_le32(EDMA_TCD_NBYTES_NBYTES(nbytes)); - tcd->slast = cpu_to_le32(EDMA_TCD_SLAST_SLAST(slast)); - - tcd->citer = cpu_to_le16(EDMA_TCD_CITER_CITER(citer)); - tcd->doff = cpu_to_le16(EDMA_TCD_DOFF_DOFF(doff)); - - tcd->dlast_sga = cpu_to_le32(EDMA_TCD_DLAST_SGA_DLAST_SGA(dlast_sga)); - - tcd->biter = cpu_to_le16(EDMA_TCD_BITER_BITER(biter)); - if (major_int) - csr |= EDMA_TCD_CSR_INT_MAJOR; - - if (disable_req) - csr |= EDMA_TCD_CSR_D_REQ; - - if (enable_sg) - csr |= EDMA_TCD_CSR_E_SG; - - tcd->csr = cpu_to_le16(csr); -} - -static struct fsl_edma_desc *fsl_edma_alloc_desc(struct fsl_edma_chan *fsl_chan, - int sg_len) -{ - struct fsl_edma_desc *fsl_desc; - int i; - - fsl_desc = kzalloc(sizeof(*fsl_desc) + sizeof(struct fsl_edma_sw_tcd) * sg_len, - GFP_NOWAIT); - if (!fsl_desc) - return NULL; - - fsl_desc->echan = fsl_chan; - fsl_desc->n_tcds = sg_len; - for (i = 0; i < sg_len; i++) { - fsl_desc->tcd[i].vtcd = dma_pool_alloc(fsl_chan->tcd_pool, - GFP_NOWAIT, &fsl_desc->tcd[i].ptcd); - if (!fsl_desc->tcd[i].vtcd) - goto err; - } - return fsl_desc; - -err: - while (--i >= 0) - dma_pool_free(fsl_chan->tcd_pool, fsl_desc->tcd[i].vtcd, - fsl_desc->tcd[i].ptcd); - kfree(fsl_desc); - return NULL; -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_dma_cyclic( - struct dma_chan *chan, dma_addr_t dma_addr, size_t buf_len, - size_t period_len, enum dma_transfer_direction direction, - unsigned long flags) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - dma_addr_t dma_buf_next; - int sg_len, i; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - sg_len = buf_len / period_len; - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = true; - - dma_buf_next = dma_addr; - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - iter = period_len / nbytes; - - for (i = 0; i < sg_len; i++) { - if (dma_buf_next >= dma_addr + buf_len) - dma_buf_next = dma_addr; - - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = dma_buf_next; - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = dma_buf_next; - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, dst_addr, - fsl_chan->fsc.attr, soff, nbytes, 0, iter, - iter, doff, last_sg, true, false, true); - dma_buf_next += period_len; - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static struct dma_async_tx_descriptor *fsl_edma_prep_slave_sg( - struct dma_chan *chan, struct scatterlist *sgl, - unsigned int sg_len, enum dma_transfer_direction direction, - unsigned long flags, void *context) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - struct fsl_edma_desc *fsl_desc; - struct scatterlist *sg; - u32 src_addr, dst_addr, last_sg, nbytes; - u16 soff, doff, iter; - int i; - - if (!is_slave_direction(fsl_chan->fsc.dir)) - return NULL; - - fsl_desc = fsl_edma_alloc_desc(fsl_chan, sg_len); - if (!fsl_desc) - return NULL; - fsl_desc->iscyclic = false; - - nbytes = fsl_chan->fsc.addr_width * fsl_chan->fsc.burst; - for_each_sg(sgl, sg, sg_len, i) { - /* get next sg's physical address */ - last_sg = fsl_desc->tcd[(i + 1) % sg_len].ptcd; - - if (fsl_chan->fsc.dir == DMA_MEM_TO_DEV) { - src_addr = sg_dma_address(sg); - dst_addr = fsl_chan->fsc.dev_addr; - soff = fsl_chan->fsc.addr_width; - doff = 0; - } else { - src_addr = fsl_chan->fsc.dev_addr; - dst_addr = sg_dma_address(sg); - soff = 0; - doff = fsl_chan->fsc.addr_width; - } - - iter = sg_dma_len(sg) / nbytes; - if (i < sg_len - 1) { - last_sg = fsl_desc->tcd[(i + 1)].ptcd; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - false, false, true); - } else { - last_sg = 0; - fsl_edma_fill_tcd(fsl_desc->tcd[i].vtcd, src_addr, - dst_addr, fsl_chan->fsc.attr, soff, - nbytes, 0, iter, iter, doff, last_sg, - true, true, false); - } - } - - return vchan_tx_prep(&fsl_chan->vchan, &fsl_desc->vdesc, flags); -} - -static void fsl_edma_xfer_desc(struct fsl_edma_chan *fsl_chan) -{ - struct virt_dma_desc *vdesc; - - vdesc = vchan_next_desc(&fsl_chan->vchan); - if (!vdesc) - return; - fsl_chan->edesc = to_fsl_edma_desc(vdesc); - fsl_edma_set_tcd_regs(fsl_chan, fsl_chan->edesc->tcd[0].vtcd); - fsl_edma_enable_request(fsl_chan); - fsl_chan->status = DMA_IN_PROGRESS; - fsl_chan->idle = false; -} +#include "fsl-edma-common.h" static irqreturn_t fsl_edma_tx_handler(int irq, void *dev_id) { @@ -730,25 +93,6 @@ static irqreturn_t fsl_edma_irq_handler(int irq, void *dev_id) return fsl_edma_err_handler(irq, dev_id); } -static void fsl_edma_issue_pending(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - - if (unlikely(fsl_chan->pm_state != RUNNING)) { - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - /* cannot submit due to suspend */ - return; - } - - if (vchan_issue_pending(&fsl_chan->vchan) && !fsl_chan->edesc) - fsl_edma_xfer_desc(fsl_chan); - - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); -} - static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, struct of_dma *ofdma) { @@ -781,34 +125,6 @@ static struct dma_chan *fsl_edma_xlate(struct of_phandle_args *dma_spec, return NULL; } -static int fsl_edma_alloc_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - - fsl_chan->tcd_pool = dma_pool_create("tcd_pool", chan->device->dev, - sizeof(struct fsl_edma_hw_tcd), - 32, 0); - return 0; -} - -static void fsl_edma_free_chan_resources(struct dma_chan *chan) -{ - struct fsl_edma_chan *fsl_chan = to_fsl_edma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&fsl_chan->vchan.lock, flags); - fsl_edma_disable_request(fsl_chan); - fsl_edma_chan_mux(fsl_chan, 0, false); - fsl_chan->edesc = NULL; - vchan_get_all_descriptors(&fsl_chan->vchan, &head); - spin_unlock_irqrestore(&fsl_chan->vchan.lock, flags); - - vchan_dma_desc_free_list(&fsl_chan->vchan, &head); - dma_pool_destroy(fsl_chan->tcd_pool); - fsl_chan->tcd_pool = NULL; -} - static int fsl_edma_irq_init(struct platform_device *pdev, struct fsl_edma_engine *fsl_edma) { @@ -995,17 +311,6 @@ static int fsl_edma_probe(struct platform_device *pdev) return 0; } -static void fsl_edma_cleanup_vchan(struct dma_device *dmadev) -{ - struct fsl_edma_chan *chan, *_chan; - - list_for_each_entry_safe(chan, _chan, - &dmadev->channels, vchan.chan.device_node) { - list_del(&chan->vchan.chan.device_node); - tasklet_kill(&chan->vchan.task); - } -} - static int fsl_edma_remove(struct platform_device *pdev) { struct device_node *np = pdev->dev.of_node; -- 2.18.0