From mboxrd@z Thu Jan 1 00:00:00 1970 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: base64 Subject: [v4] dmaengine: pl330: flush before wait, and add dev burst support. From: Frank Mori Hess Message-Id: <2863317.1ZoR41yQkl@bear> Date: Tue, 13 Mar 2018 14:34:26 -0400 To: dmaengine@vger.kernel.org Cc: Vinod Koul , Dan Williams , linux-kernel@vger.kernel.org List-ID: RG8gRE1BRkxVU0hQIF9iZWZvcmVfIHRoZSBmaXJzdCBETUFXRlAgdG8gZW5zdXJlIGNvbnRyb2xs ZXIKYW5kIHBlcmlwaGVyYWwgYXJlIGluIGFncmVlbWVudCBhYm91dCBkbWEgcmVxdWVzdCBzdGF0 ZSBiZWZvcmUgZmlyc3QKdHJhbnNmZXIuICBBZGQgc3VwcG9ydCBmb3IgYnVyc3QgdHJhbnNmZXJz IHRvL2Zyb20gcGVyaXBoZXJhbHMuIEluIHRoZSBuZXcKc2NoZW1lLCB0aGUgY29udHJvbGxlciBk b2VzIGFzIG1hbnkgYnVyc3QgdHJhbnNmZXJzIGFzIGl0IGNhbiB0aGVuCnRyYW5zZmVycyB0aGUg cmVtYWluaW5nIGRyZWdzIHdpdGggZWl0aGVyIHNpbmdsZSB0cmFuc2ZlcnMgZm9yCnBlcmlwaGVy YWxzLCBvciB3aXRoIGEgcmVkdWNlZCBzaXplIGJ1cnN0IGZvciBtZW1vcnktdG8tbWVtb3J5IHRy YW5zZmVycy4KClNpZ25lZC1vZmYtYnk6IEZyYW5rIE1vcmkgSGVzcyA8Zm1oNmpqQGdtYWlsLmNv bT4KVGVzdGVkLWJ5OiBGcmFuayBNb3JpIEhlc3MgPGZtaDZqakBnbWFpbC5jb20+Ci0tLQoKSSB0 ZXN0ZWQgZG1hIHRyYW5zZmVycyB0byBwZXJpcGhlcmFscyB3aXRoIHYzIHBhdGNoIGFuZCBkZXNp Z253YXJlIHNlcmlhbCAKcG9ydCAoZHJpdmVycy90dHkvc2VyaWFsLzgyNTAvODI1MF9kdy5jKSBh bmQgYSBHUElCIGludGVyZmFjZQooaHR0cHM6Ly9naXRodWIuY29tL2ZtaGVzcy9mbWhfZ3BpYl9j b3JlKS4gIEkgdGVzdGVkIG1lbW9yeS10by1tZW1vcnkKdHJhbnNmZXJzIHVzaW5nIHRoZSBkbWF0 ZXN0IG1vZHVsZS4KCnYzIG9mIHRoaXMgcGF0Y2ggc2hvdWxkIGJlIHRoZSBzYW1lIGFzIHYyIGV4 Y2VwdCB3aXRoIGNoZWNrcGF0Y2gucGwKd2FybmluZ3MgYW5kIGVycm9ycyBjbGVhbmVkIHVwLgoK djQgYWRkcmVzc2VzIGNvc21ldGljIGNvbXBsYWludHMgYWJvdXQgdjMsIHNob3VsZCBiZSBmdW5j dGlvbmFsbHkgdW5jaGFuZ2VkLgoKIGRyaXZlcnMvZG1hL3BsMzMwLmMgfCAyMDkgKysrKysrKysr KysrKysrKysrKysrKysrKysrKysrKysrKysrKysrLS0tLS0tLS0tLS0tLQogMSBmaWxlIGNoYW5n ZWQsIDE1OSBpbnNlcnRpb25zKCspLCA1MCBkZWxldGlvbnMoLSkKCmRpZmYgLS1naXQgYS9kcml2 ZXJzL2RtYS9wbDMzMC5jIGIvZHJpdmVycy9kbWEvcGwzMzAuYwppbmRleCBkNzMyN2ZkNWY0NDUu LjgxOWE1NzhlMzE3ZiAxMDA2NDQKLS0tIGEvZHJpdmVycy9kbWEvcGwzMzAuYworKysgYi9kcml2 ZXJzL2RtYS9wbDMzMC5jCkBAIC0yNyw2ICsyNyw3IEBACiAjaW5jbHVkZSA8bGludXgvb2ZfZG1h Lmg+CiAjaW5jbHVkZSA8bGludXgvZXJyLmg+CiAjaW5jbHVkZSA8bGludXgvcG1fcnVudGltZS5o PgorI2luY2x1ZGUgPGxpbnV4L2J1Zy5oPgogCiAjaW5jbHVkZSAiZG1hZW5naW5lLmgiCiAjZGVm aW5lIFBMMzMwX01BWF9DSEFOCQk4CkBAIC0xMDk0LDUxICsxMDk1LDk2IEBAIHN0YXRpYyBpbmxp bmUgaW50IF9sZHN0X21lbXRvbWVtKHVuc2lnbmVkIGRyeV9ydW4sIHU4IGJ1ZltdLAogCXJldHVy biBvZmY7CiB9CiAKLXN0YXRpYyBpbmxpbmUgaW50IF9sZHN0X2RldnRvbWVtKHN0cnVjdCBwbDMz MF9kbWFjICpwbDMzMCwgdW5zaWduZWQgZHJ5X3J1biwKLQkJCQkgdTggYnVmW10sIGNvbnN0IHN0 cnVjdCBfeGZlcl9zcGVjICpweHMsCi0JCQkJIGludCBjeWMpCitzdGF0aWMgdTMyIF9lbWl0X2xv YWQodW5zaWduZWQgaW50IGRyeV9ydW4sIHU4IGJ1ZltdLAorCWVudW0gcGwzMzBfY29uZCBjb25k LCBlbnVtIGRtYV90cmFuc2Zlcl9kaXJlY3Rpb24gZGlyZWN0aW9uLAorCXU4IHBlcmkpCiB7CiAJ aW50IG9mZiA9IDA7Ci0JZW51bSBwbDMzMF9jb25kIGNvbmQ7CiAKLQlpZiAocGwzMzAtPnF1aXJr cyAmIFBMMzMwX1FVSVJLX0JST0tFTl9OT19GTFVTSFApCi0JCWNvbmQgPSBCVVJTVDsKLQllbHNl Ci0JCWNvbmQgPSBTSU5HTEU7CisJc3dpdGNoIChkaXJlY3Rpb24pIHsKKwljYXNlIERNQV9NRU1f VE9fTUVNOgorCQkvKiBmYWxsIHRocm91Z2ggKi8KKwljYXNlIERNQV9NRU1fVE9fREVWOgorCQlv ZmYgKz0gX2VtaXRfTEQoZHJ5X3J1biwgJmJ1ZltvZmZdLCBjb25kKTsKKwkJYnJlYWs7CiAKLQl3 aGlsZSAoY3ljLS0pIHsKLQkJb2ZmICs9IF9lbWl0X1dGUChkcnlfcnVuLCAmYnVmW29mZl0sIGNv bmQsIHB4cy0+ZGVzYy0+cGVyaSk7Ci0JCW9mZiArPSBfZW1pdF9MRFAoZHJ5X3J1biwgJmJ1Zltv ZmZdLCBjb25kLCBweHMtPmRlc2MtPnBlcmkpOwotCQlvZmYgKz0gX2VtaXRfU1QoZHJ5X3J1biwg JmJ1ZltvZmZdLCBBTFdBWVMpOworCWNhc2UgRE1BX0RFVl9UT19NRU06CisJCWlmIChjb25kID09 IEFMV0FZUykgeworCQkJb2ZmICs9IF9lbWl0X0xEUChkcnlfcnVuLCAmYnVmW29mZl0sIFNJTkdM RSwKKwkJCQlwZXJpKTsKKwkJCW9mZiArPSBfZW1pdF9MRFAoZHJ5X3J1biwgJmJ1ZltvZmZdLCBC VVJTVCwKKwkJCQlwZXJpKTsKKwkJfSBlbHNlIHsKKwkJCW9mZiArPSBfZW1pdF9MRFAoZHJ5X3J1 biwgJmJ1ZltvZmZdLCBjb25kLAorCQkJCXBlcmkpOworCQl9CisJCWJyZWFrOwogCi0JCWlmICgh KHBsMzMwLT5xdWlya3MgJiBQTDMzMF9RVUlSS19CUk9LRU5fTk9fRkxVU0hQKSkKLQkJCW9mZiAr PSBfZW1pdF9GTFVTSFAoZHJ5X3J1biwgJmJ1ZltvZmZdLAotCQkJCQkgICAgcHhzLT5kZXNjLT5w ZXJpKTsKKwlkZWZhdWx0OgorCQkvKiB0aGlzIGNvZGUgc2hvdWxkIGJlIHVucmVhY2hhYmxlICov CisJCVdBUk5fT04oMSk7CisJCWJyZWFrOwogCX0KIAogCXJldHVybiBvZmY7CiB9CiAKLXN0YXRp YyBpbmxpbmUgaW50IF9sZHN0X21lbXRvZGV2KHN0cnVjdCBwbDMzMF9kbWFjICpwbDMzMCwKK3N0 YXRpYyBpbmxpbmUgdTMyIF9lbWl0X3N0b3JlKHVuc2lnbmVkIGludCBkcnlfcnVuLCB1OCBidWZb XSwKKwllbnVtIHBsMzMwX2NvbmQgY29uZCwgZW51bSBkbWFfdHJhbnNmZXJfZGlyZWN0aW9uIGRp cmVjdGlvbiwKKwl1OCBwZXJpKQoreworCWludCBvZmYgPSAwOworCisJc3dpdGNoIChkaXJlY3Rp b24pIHsKKwljYXNlIERNQV9NRU1fVE9fTUVNOgorCQkvKiBmYWxsIHRocm91Z2ggKi8KKwljYXNl IERNQV9ERVZfVE9fTUVNOgorCQlvZmYgKz0gX2VtaXRfU1QoZHJ5X3J1biwgJmJ1ZltvZmZdLCBj b25kKTsKKwkJYnJlYWs7CisKKwljYXNlIERNQV9NRU1fVE9fREVWOgorCQlpZiAoY29uZCA9PSBB TFdBWVMpIHsKKwkJCW9mZiArPSBfZW1pdF9TVFAoZHJ5X3J1biwgJmJ1ZltvZmZdLCBTSU5HTEUs CisJCQkJcGVyaSk7CisJCQlvZmYgKz0gX2VtaXRfU1RQKGRyeV9ydW4sICZidWZbb2ZmXSwgQlVS U1QsCisJCQkJcGVyaSk7CisJCX0gZWxzZSB7CisJCQlvZmYgKz0gX2VtaXRfU1RQKGRyeV9ydW4s ICZidWZbb2ZmXSwgY29uZCwKKwkJCQlwZXJpKTsKKwkJfQorCQlicmVhazsKKworCWRlZmF1bHQ6 CisJCS8qIHRoaXMgY29kZSBzaG91bGQgYmUgdW5yZWFjaGFibGUgKi8KKwkJV0FSTl9PTigxKTsK KwkJYnJlYWs7CisJfQorCisJcmV0dXJuIG9mZjsKK30KKworc3RhdGljIGlubGluZSBpbnQgX2xk c3RfcGVyaXBoZXJhbChzdHJ1Y3QgcGwzMzBfZG1hYyAqcGwzMzAsCiAJCQkJIHVuc2lnbmVkIGRy eV9ydW4sIHU4IGJ1ZltdLAotCQkJCSBjb25zdCBzdHJ1Y3QgX3hmZXJfc3BlYyAqcHhzLCBpbnQg Y3ljKQorCQkJCSBjb25zdCBzdHJ1Y3QgX3hmZXJfc3BlYyAqcHhzLCBpbnQgY3ljLAorCQkJCSBl bnVtIHBsMzMwX2NvbmQgY29uZCkKIHsKIAlpbnQgb2ZmID0gMDsKLQllbnVtIHBsMzMwX2NvbmQg Y29uZDsKIAogCWlmIChwbDMzMC0+cXVpcmtzICYgUEwzMzBfUVVJUktfQlJPS0VOX05PX0ZMVVNI UCkKIAkJY29uZCA9IEJVUlNUOwotCWVsc2UKLQkJY29uZCA9IFNJTkdMRTsKIAorCS8qCisJICog ZG8gRkxVU0hQIGF0IGJlZ2lubmluZyB0byBjbGVhciBhbnkgc3RhbGUgZG1hIHJlcXVlc3RzIGJl Zm9yZSB0aGUKKwkgKiBmaXJzdCBXRlAuCisJICovCisJaWYgKCEocGwzMzAtPnF1aXJrcyAmIFBM MzMwX1FVSVJLX0JST0tFTl9OT19GTFVTSFApKQorCQlvZmYgKz0gX2VtaXRfRkxVU0hQKGRyeV9y dW4sICZidWZbb2ZmXSwgcHhzLT5kZXNjLT5wZXJpKTsKIAl3aGlsZSAoY3ljLS0pIHsKIAkJb2Zm ICs9IF9lbWl0X1dGUChkcnlfcnVuLCAmYnVmW29mZl0sIGNvbmQsIHB4cy0+ZGVzYy0+cGVyaSk7 Ci0JCW9mZiArPSBfZW1pdF9MRChkcnlfcnVuLCAmYnVmW29mZl0sIEFMV0FZUyk7Ci0JCW9mZiAr PSBfZW1pdF9TVFAoZHJ5X3J1biwgJmJ1ZltvZmZdLCBjb25kLCBweHMtPmRlc2MtPnBlcmkpOwot Ci0JCWlmICghKHBsMzMwLT5xdWlya3MgJiBQTDMzMF9RVUlSS19CUk9LRU5fTk9fRkxVU0hQKSkK LQkJCW9mZiArPSBfZW1pdF9GTFVTSFAoZHJ5X3J1biwgJmJ1ZltvZmZdLAotCQkJCQkgICAgcHhz LT5kZXNjLT5wZXJpKTsKKwkJb2ZmICs9IF9lbWl0X2xvYWQoZHJ5X3J1biwgJmJ1ZltvZmZdLCBj b25kLCBweHMtPmRlc2MtPnJxdHlwZSwKKwkJCXB4cy0+ZGVzYy0+cGVyaSk7CisJCW9mZiArPSBf ZW1pdF9zdG9yZShkcnlfcnVuLCAmYnVmW29mZl0sIGNvbmQsIHB4cy0+ZGVzYy0+cnF0eXBlLAor CQkJcHhzLT5kZXNjLT5wZXJpKTsKIAl9CiAKIAlyZXR1cm4gb2ZmOwpAQCAtMTE0OCwxOSArMTE5 NCw2NSBAQCBzdGF0aWMgaW50IF9idXJzdHMoc3RydWN0IHBsMzMwX2RtYWMgKnBsMzMwLCB1bnNp Z25lZCBkcnlfcnVuLCB1OCBidWZbXSwKIAkJY29uc3Qgc3RydWN0IF94ZmVyX3NwZWMgKnB4cywg aW50IGN5YykKIHsKIAlpbnQgb2ZmID0gMDsKKwllbnVtIHBsMzMwX2NvbmQgY29uZCA9IEJSU1Rf TEVOKHB4cy0+Y2NyKSA+IDEgPyBCVVJTVCA6IFNJTkdMRTsKIAogCXN3aXRjaCAocHhzLT5kZXNj LT5ycXR5cGUpIHsKIAljYXNlIERNQV9NRU1fVE9fREVWOgotCQlvZmYgKz0gX2xkc3RfbWVtdG9k ZXYocGwzMzAsIGRyeV9ydW4sICZidWZbb2ZmXSwgcHhzLCBjeWMpOwotCQlicmVhazsKKwkJLyog ZmFsbCB0aHJvdWdoICovCiAJY2FzZSBETUFfREVWX1RPX01FTToKLQkJb2ZmICs9IF9sZHN0X2Rl dnRvbWVtKHBsMzMwLCBkcnlfcnVuLCAmYnVmW29mZl0sIHB4cywgY3ljKTsKKwkJb2ZmICs9IF9s ZHN0X3BlcmlwaGVyYWwocGwzMzAsIGRyeV9ydW4sICZidWZbb2ZmXSwgcHhzLCBjeWMsCisJCQlj b25kKTsKIAkJYnJlYWs7CisKIAljYXNlIERNQV9NRU1fVE9fTUVNOgogCQlvZmYgKz0gX2xkc3Rf bWVtdG9tZW0oZHJ5X3J1biwgJmJ1ZltvZmZdLCBweHMsIGN5Yyk7CiAJCWJyZWFrOworCisJZGVm YXVsdDoKKwkJLyogdGhpcyBjb2RlIHNob3VsZCBiZSB1bnJlYWNoYWJsZSAqLworCQlXQVJOX09O KDEpOworCQlicmVhazsKKwl9CisKKwlyZXR1cm4gb2ZmOworfQorCisvKgorICogdHJhbnNmZXIg ZHJlZ3Mgd2l0aCBzaW5nbGUgdHJhbnNmZXJzIHRvIHBlcmlwaGVyYWwsIG9yIGEgcmVkdWNlZCBz aXplIGJ1cnN0CisgKiBmb3IgbWVtLXRvLW1lbS4KKyAqLworc3RhdGljIGludCBfZHJlZ3Moc3Ry dWN0IHBsMzMwX2RtYWMgKnBsMzMwLCB1bnNpZ25lZCBpbnQgZHJ5X3J1biwgdTggYnVmW10sCisJ CWNvbnN0IHN0cnVjdCBfeGZlcl9zcGVjICpweHMsIGludCB0cmFuc2Zlcl9sZW5ndGgpCit7CisJ aW50IG9mZiA9IDA7CisJaW50IGRyZWdzX2NjcjsKKworCWlmICh0cmFuc2Zlcl9sZW5ndGggPT0g MCkKKwkJcmV0dXJuIG9mZjsKKworCXN3aXRjaCAocHhzLT5kZXNjLT5ycXR5cGUpIHsKKwljYXNl IERNQV9NRU1fVE9fREVWOgorCQkvKiBmYWxsIHRocm91Z2ggKi8KKwljYXNlIERNQV9ERVZfVE9f TUVNOgorCQlvZmYgKz0gX2xkc3RfcGVyaXBoZXJhbChwbDMzMCwgZHJ5X3J1biwgJmJ1ZltvZmZd LCBweHMsCisJCQl0cmFuc2Zlcl9sZW5ndGgsIFNJTkdMRSk7CisJCWJyZWFrOworCisJY2FzZSBE TUFfTUVNX1RPX01FTToKKwkJZHJlZ3NfY2NyID0gcHhzLT5jY3I7CisJCWRyZWdzX2NjciAmPSB+ KCgweGYgPDwgQ0NfU1JDQlJTVExFTl9TSEZUKSB8CisJCQkoMHhmIDw8IENDX0RTVEJSU1RMRU5f U0hGVCkpOworCQlkcmVnc19jY3IgfD0gKCgodHJhbnNmZXJfbGVuZ3RoIC0gMSkgJiAweGYpIDw8 CisJCQlDQ19TUkNCUlNUTEVOX1NIRlQpOworCQlkcmVnc19jY3IgfD0gKCgodHJhbnNmZXJfbGVu Z3RoIC0gMSkgJiAweGYpIDw8CisJCQlDQ19EU1RCUlNUTEVOX1NIRlQpOworCQlvZmYgKz0gX2Vt aXRfTU9WKGRyeV9ydW4sICZidWZbb2ZmXSwgQ0NSLCBkcmVnc19jY3IpOworCQlvZmYgKz0gX2xk c3RfbWVtdG9tZW0oZHJ5X3J1biwgJmJ1ZltvZmZdLCBweHMsIDEpOworCQlicmVhazsKKwogCWRl ZmF1bHQ6Ci0JCW9mZiArPSAweDQwMDAwMDAwOyAvKiBTY2FyZSBvZmYgdGhlIENsaWVudCAqLwor CQkvKiB0aGlzIGNvZGUgc2hvdWxkIGJlIHVucmVhY2hhYmxlICovCisJCVdBUk5fT04oMSk7CiAJ CWJyZWFrOwogCX0KIApAQCAtMTI1Niw2ICsxMzQ4LDggQEAgc3RhdGljIGlubGluZSBpbnQgX3Nl dHVwX2xvb3BzKHN0cnVjdCBwbDMzMF9kbWFjICpwbDMzMCwKIAlzdHJ1Y3QgcGwzMzBfeGZlciAq eCA9ICZweHMtPmRlc2MtPnB4OwogCXUzMiBjY3IgPSBweHMtPmNjcjsKIAl1bnNpZ25lZCBsb25n IGMsIGJ1cnN0cyA9IEJZVEVfVE9fQlVSU1QoeC0+Ynl0ZXMsIGNjcik7CisJaW50IG51bV9kcmVn cyA9ICh4LT5ieXRlcyAtIEJVUlNUX1RPX0JZVEUoYnVyc3RzLCBjY3IpKSAvCisJCUJSU1RfU0la RShjY3IpOwogCWludCBvZmYgPSAwOwogCiAJd2hpbGUgKGJ1cnN0cykgewpAQCAtMTI2Myw2ICsx MzU3LDcgQEAgc3RhdGljIGlubGluZSBpbnQgX3NldHVwX2xvb3BzKHN0cnVjdCBwbDMzMF9kbWFj ICpwbDMzMCwKIAkJb2ZmICs9IF9sb29wKHBsMzMwLCBkcnlfcnVuLCAmYnVmW29mZl0sICZjLCBw eHMpOwogCQlidXJzdHMgLT0gYzsKIAl9CisJb2ZmICs9IF9kcmVncyhwbDMzMCwgZHJ5X3J1biwg JmJ1ZltvZmZdLCBweHMsIG51bV9kcmVncyk7CiAKIAlyZXR1cm4gb2ZmOwogfQpAQCAtMTI5NCw3 ICsxMzg5LDYgQEAgc3RhdGljIGludCBfc2V0dXBfcmVxKHN0cnVjdCBwbDMzMF9kbWFjICpwbDMz MCwgdW5zaWduZWQgZHJ5X3J1biwKIAkJICAgICAgc3RydWN0IF94ZmVyX3NwZWMgKnB4cykKIHsK IAlzdHJ1Y3QgX3BsMzMwX3JlcSAqcmVxID0gJnRocmQtPnJlcVtpbmRleF07Ci0Jc3RydWN0IHBs MzMwX3hmZXIgKng7CiAJdTggKmJ1ZiA9IHJlcS0+bWNfY3B1OwogCWludCBvZmYgPSAwOwogCkBA IC0xMzAzLDExICsxMzk3LDYgQEAgc3RhdGljIGludCBfc2V0dXBfcmVxKHN0cnVjdCBwbDMzMF9k bWFjICpwbDMzMCwgdW5zaWduZWQgZHJ5X3J1biwKIAkvKiBETUFNT1YgQ0NSLCBjY3IgKi8KIAlv ZmYgKz0gX2VtaXRfTU9WKGRyeV9ydW4sICZidWZbb2ZmXSwgQ0NSLCBweHMtPmNjcik7CiAKLQl4 ID0gJnB4cy0+ZGVzYy0+cHg7Ci0JLyogRXJyb3IgaWYgeGZlciBsZW5ndGggaXMgbm90IGFsaWdu ZWQgYXQgYnVyc3Qgc2l6ZSAqLwotCWlmICh4LT5ieXRlcyAlIChCUlNUX1NJWkUocHhzLT5jY3Ip ICogQlJTVF9MRU4ocHhzLT5jY3IpKSkKLQkJcmV0dXJuIC1FSU5WQUw7Ci0KIAlvZmYgKz0gX3Nl dHVwX3hmZXIocGwzMzAsIGRyeV9ydW4sICZidWZbb2ZmXSwgcHhzKTsKIAogCS8qIERNQVNFViBw ZXJpcGhlcmFsL2V2ZW50ICovCkBAIC0xMzY1LDYgKzE0NTQsMjAgQEAgc3RhdGljIGludCBwbDMz MF9zdWJtaXRfcmVxKHN0cnVjdCBwbDMzMF90aHJlYWQgKnRocmQsCiAJdTMyIGNjcjsKIAlpbnQg cmV0ID0gMDsKIAorCXN3aXRjaCAoZGVzYy0+cnF0eXBlKSB7CisJY2FzZSBETUFfTUVNX1RPX0RF VjoKKwkJYnJlYWs7CisKKwljYXNlIERNQV9ERVZfVE9fTUVNOgorCQlicmVhazsKKworCWNhc2Ug RE1BX01FTV9UT19NRU06CisJCWJyZWFrOworCisJZGVmYXVsdDoKKwkJcmV0dXJuIC1FTk9UU1VQ UDsKKwl9CisKIAlpZiAocGwzMzAtPnN0YXRlID09IERZSU5HCiAJCXx8IHBsMzMwLT5kbWFjX3Ri ZC5yZXNldF9jaGFuICYgKDEgPDwgdGhyZC0+aWQpKSB7CiAJCWRldl9pbmZvKHRocmQtPmRtYWMt PmRkbWEuZGV2LCAiJXM6JWRcbiIsCkBAIC0yMTA0LDYgKzIyMDcsMTggQEAgc3RhdGljIGJvb2wg cGwzMzBfcHJlcF9zbGF2ZV9maWZvKHN0cnVjdCBkbWFfcGwzMzBfY2hhbiAqcGNoLAogCXJldHVy biB0cnVlOwogfQogCitzdGF0aWMgaW50IGZpeHVwX2J1cnN0X2xlbihpbnQgbWF4X2J1cnN0X2xl biwgaW50IHF1aXJrcykKK3sKKwlpZiAocXVpcmtzICYgUEwzMzBfUVVJUktfQlJPS0VOX05PX0ZM VVNIUCkKKwkJcmV0dXJuIDE7CisJZWxzZSBpZiAobWF4X2J1cnN0X2xlbiA+IFBMMzMwX01BWF9C VVJTVCkKKwkJcmV0dXJuIFBMMzMwX01BWF9CVVJTVDsKKwllbHNlIGlmIChtYXhfYnVyc3RfbGVu IDwgMSkKKwkJcmV0dXJuIDE7CisJZWxzZQorCQlyZXR1cm4gbWF4X2J1cnN0X2xlbjsKK30KKwog c3RhdGljIGludCBwbDMzMF9jb25maWcoc3RydWN0IGRtYV9jaGFuICpjaGFuLAogCQkJc3RydWN0 IGRtYV9zbGF2ZV9jb25maWcgKnNsYXZlX2NvbmZpZykKIHsKQEAgLTIxMTUsMTUgKzIyMzAsMTUg QEAgc3RhdGljIGludCBwbDMzMF9jb25maWcoc3RydWN0IGRtYV9jaGFuICpjaGFuLAogCQkJcGNo LT5maWZvX2FkZHIgPSBzbGF2ZV9jb25maWctPmRzdF9hZGRyOwogCQlpZiAoc2xhdmVfY29uZmln LT5kc3RfYWRkcl93aWR0aCkKIAkJCXBjaC0+YnVyc3Rfc3ogPSBfX2ZmcyhzbGF2ZV9jb25maWct PmRzdF9hZGRyX3dpZHRoKTsKLQkJaWYgKHNsYXZlX2NvbmZpZy0+ZHN0X21heGJ1cnN0KQotCQkJ cGNoLT5idXJzdF9sZW4gPSBzbGF2ZV9jb25maWctPmRzdF9tYXhidXJzdDsKKwkJcGNoLT5idXJz dF9sZW4gPSBmaXh1cF9idXJzdF9sZW4oc2xhdmVfY29uZmlnLT5kc3RfbWF4YnVyc3QsCisJCQlw Y2gtPmRtYWMtPnF1aXJrcyk7CiAJfSBlbHNlIGlmIChzbGF2ZV9jb25maWctPmRpcmVjdGlvbiA9 PSBETUFfREVWX1RPX01FTSkgewogCQlpZiAoc2xhdmVfY29uZmlnLT5zcmNfYWRkcikKIAkJCXBj aC0+Zmlmb19hZGRyID0gc2xhdmVfY29uZmlnLT5zcmNfYWRkcjsKIAkJaWYgKHNsYXZlX2NvbmZp Zy0+c3JjX2FkZHJfd2lkdGgpCiAJCQlwY2gtPmJ1cnN0X3N6ID0gX19mZnMoc2xhdmVfY29uZmln LT5zcmNfYWRkcl93aWR0aCk7Ci0JCWlmIChzbGF2ZV9jb25maWctPnNyY19tYXhidXJzdCkKLQkJ CXBjaC0+YnVyc3RfbGVuID0gc2xhdmVfY29uZmlnLT5zcmNfbWF4YnVyc3Q7CisJCXBjaC0+YnVy c3RfbGVuID0gZml4dXBfYnVyc3RfbGVuKHNsYXZlX2NvbmZpZy0+c3JjX21heGJ1cnN0LAorCQkJ cGNoLT5kbWFjLT5xdWlya3MpOwogCX0KIAogCXJldHVybiAwOwpAQCAtMjUxNywxNCArMjYzMiw4 IEBAIHN0YXRpYyBpbmxpbmUgaW50IGdldF9idXJzdF9sZW4oc3RydWN0IGRtYV9wbDMzMF9kZXNj ICpkZXNjLCBzaXplX3QgbGVuKQogCWJ1cnN0X2xlbiA+Pj0gZGVzYy0+cnFjZmcuYnJzdF9zaXpl OwogCiAJLyogc3JjL2RzdF9idXJzdF9sZW4gY2FuJ3QgYmUgbW9yZSB0aGFuIDE2ICovCi0JaWYg KGJ1cnN0X2xlbiA+IDE2KQotCQlidXJzdF9sZW4gPSAxNjsKLQotCXdoaWxlIChidXJzdF9sZW4g PiAxKSB7Ci0JCWlmICghKGxlbiAlIChidXJzdF9sZW4gPDwgZGVzYy0+cnFjZmcuYnJzdF9zaXpl KSkpCi0JCQlicmVhazsKLQkJYnVyc3RfbGVuLS07Ci0JfQorCWlmIChidXJzdF9sZW4gPiBQTDMz MF9NQVhfQlVSU1QpCisJCWJ1cnN0X2xlbiA9IFBMMzMwX01BWF9CVVJTVDsKIAogCXJldHVybiBi dXJzdF9sZW47CiB9CkBAIC0yNTk2LDcgKzI3MDUsNyBAQCBzdGF0aWMgc3RydWN0IGRtYV9hc3lu Y190eF9kZXNjcmlwdG9yICpwbDMzMF9wcmVwX2RtYV9jeWNsaWMoCiAKIAkJZGVzYy0+cnF0eXBl ID0gZGlyZWN0aW9uOwogCQlkZXNjLT5ycWNmZy5icnN0X3NpemUgPSBwY2gtPmJ1cnN0X3N6Owot CQlkZXNjLT5ycWNmZy5icnN0X2xlbiA9IDE7CisJCWRlc2MtPnJxY2ZnLmJyc3RfbGVuID0gcGNo LT5idXJzdF9sZW47CiAJCWRlc2MtPmJ5dGVzX3JlcXVlc3RlZCA9IHBlcmlvZF9sZW47CiAJCWZp bGxfcHgoJmRlc2MtPnB4LCBkc3QsIHNyYywgcGVyaW9kX2xlbik7CiAKQEAgLTI3NDEsNyArMjg1 MCw3IEBAIHBsMzMwX3ByZXBfc2xhdmVfc2coc3RydWN0IGRtYV9jaGFuICpjaGFuLCBzdHJ1Y3Qg c2NhdHRlcmxpc3QgKnNnbCwKIAkJfQogCiAJCWRlc2MtPnJxY2ZnLmJyc3Rfc2l6ZSA9IHBjaC0+ YnVyc3Rfc3o7Ci0JCWRlc2MtPnJxY2ZnLmJyc3RfbGVuID0gMTsKKwkJZGVzYy0+cnFjZmcuYnJz dF9sZW4gPSBwY2gtPmJ1cnN0X2xlbjsKIAkJZGVzYy0+cnF0eXBlID0gZGlyZWN0aW9uOwogCQlk ZXNjLT5ieXRlc19yZXF1ZXN0ZWQgPSBzZ19kbWFfbGVuKHNnKTsKIAl9Cg== From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932522AbeCMSeg (ORCPT ); Tue, 13 Mar 2018 14:34:36 -0400 Received: from mail-qt0-f194.google.com ([209.85.216.194]:35502 "EHLO mail-qt0-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751529AbeCMSed (ORCPT ); Tue, 13 Mar 2018 14:34:33 -0400 X-Google-Smtp-Source: AG47ELtz5oxoViyN+aG1rq8dvp2HqhFiA3aUe4BPEVNu3ncQrpVbyYtBzqerTEDHDzRIhdIEyAHgPA== From: Frank Mori Hess To: dmaengine@vger.kernel.org Cc: Vinod Koul , Dan Williams , linux-kernel@vger.kernel.org Subject: [PATCH v4] dmaengine: pl330: flush before wait, and add dev burst support. Date: Tue, 13 Mar 2018 14:34:26 -0400 Message-ID: <2863317.1ZoR41yQkl@bear> User-Agent: KMail/5.2.3 (Linux/4.9.0-6-amd64; KDE/5.28.0; x86_64; ; ) In-Reply-To: <20180311150151.GB15443@localhost> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Do DMAFLUSHP _before_ the first DMAWFP to ensure controller and peripheral are in agreement about dma request state before first transfer. Add support for burst transfers to/from peripherals. In the new scheme, the controller does as many burst transfers as it can then transfers the remaining dregs with either single transfers for peripherals, or with a reduced size burst for memory-to-memory transfers. Signed-off-by: Frank Mori Hess Tested-by: Frank Mori Hess --- I tested dma transfers to peripherals with v3 patch and designware serial port (drivers/tty/serial/8250/8250_dw.c) and a GPIB interface (https://github.com/fmhess/fmh_gpib_core). I tested memory-to-memory transfers using the dmatest module. v3 of this patch should be the same as v2 except with checkpatch.pl warnings and errors cleaned up. v4 addresses cosmetic complaints about v3, should be functionally unchanged. drivers/dma/pl330.c | 209 +++++++++++++++++++++++++++++++++++++++------------- 1 file changed, 159 insertions(+), 50 deletions(-) diff --git a/drivers/dma/pl330.c b/drivers/dma/pl330.c index d7327fd5f445..819a578e317f 100644 --- a/drivers/dma/pl330.c +++ b/drivers/dma/pl330.c @@ -27,6 +27,7 @@ #include #include #include +#include #include "dmaengine.h" #define PL330_MAX_CHAN 8 @@ -1094,51 +1095,96 @@ static inline int _ldst_memtomem(unsigned dry_run, u8 buf[], return off; } -static inline int _ldst_devtomem(struct pl330_dmac *pl330, unsigned dry_run, - u8 buf[], const struct _xfer_spec *pxs, - int cyc) +static u32 _emit_load(unsigned int dry_run, u8 buf[], + enum pl330_cond cond, enum dma_transfer_direction direction, + u8 peri) { int off = 0; - enum pl330_cond cond; - if (pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP) - cond = BURST; - else - cond = SINGLE; + switch (direction) { + case DMA_MEM_TO_MEM: + /* fall through */ + case DMA_MEM_TO_DEV: + off += _emit_LD(dry_run, &buf[off], cond); + break; - while (cyc--) { - off += _emit_WFP(dry_run, &buf[off], cond, pxs->desc->peri); - off += _emit_LDP(dry_run, &buf[off], cond, pxs->desc->peri); - off += _emit_ST(dry_run, &buf[off], ALWAYS); + case DMA_DEV_TO_MEM: + if (cond == ALWAYS) { + off += _emit_LDP(dry_run, &buf[off], SINGLE, + peri); + off += _emit_LDP(dry_run, &buf[off], BURST, + peri); + } else { + off += _emit_LDP(dry_run, &buf[off], cond, + peri); + } + break; - if (!(pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP)) - off += _emit_FLUSHP(dry_run, &buf[off], - pxs->desc->peri); + default: + /* this code should be unreachable */ + WARN_ON(1); + break; } return off; } -static inline int _ldst_memtodev(struct pl330_dmac *pl330, +static inline u32 _emit_store(unsigned int dry_run, u8 buf[], + enum pl330_cond cond, enum dma_transfer_direction direction, + u8 peri) +{ + int off = 0; + + switch (direction) { + case DMA_MEM_TO_MEM: + /* fall through */ + case DMA_DEV_TO_MEM: + off += _emit_ST(dry_run, &buf[off], cond); + break; + + case DMA_MEM_TO_DEV: + if (cond == ALWAYS) { + off += _emit_STP(dry_run, &buf[off], SINGLE, + peri); + off += _emit_STP(dry_run, &buf[off], BURST, + peri); + } else { + off += _emit_STP(dry_run, &buf[off], cond, + peri); + } + break; + + default: + /* this code should be unreachable */ + WARN_ON(1); + break; + } + + return off; +} + +static inline int _ldst_peripheral(struct pl330_dmac *pl330, unsigned dry_run, u8 buf[], - const struct _xfer_spec *pxs, int cyc) + const struct _xfer_spec *pxs, int cyc, + enum pl330_cond cond) { int off = 0; - enum pl330_cond cond; if (pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP) cond = BURST; - else - cond = SINGLE; + /* + * do FLUSHP at beginning to clear any stale dma requests before the + * first WFP. + */ + if (!(pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP)) + off += _emit_FLUSHP(dry_run, &buf[off], pxs->desc->peri); while (cyc--) { off += _emit_WFP(dry_run, &buf[off], cond, pxs->desc->peri); - off += _emit_LD(dry_run, &buf[off], ALWAYS); - off += _emit_STP(dry_run, &buf[off], cond, pxs->desc->peri); - - if (!(pl330->quirks & PL330_QUIRK_BROKEN_NO_FLUSHP)) - off += _emit_FLUSHP(dry_run, &buf[off], - pxs->desc->peri); + off += _emit_load(dry_run, &buf[off], cond, pxs->desc->rqtype, + pxs->desc->peri); + off += _emit_store(dry_run, &buf[off], cond, pxs->desc->rqtype, + pxs->desc->peri); } return off; @@ -1148,19 +1194,65 @@ static int _bursts(struct pl330_dmac *pl330, unsigned dry_run, u8 buf[], const struct _xfer_spec *pxs, int cyc) { int off = 0; + enum pl330_cond cond = BRST_LEN(pxs->ccr) > 1 ? BURST : SINGLE; switch (pxs->desc->rqtype) { case DMA_MEM_TO_DEV: - off += _ldst_memtodev(pl330, dry_run, &buf[off], pxs, cyc); - break; + /* fall through */ case DMA_DEV_TO_MEM: - off += _ldst_devtomem(pl330, dry_run, &buf[off], pxs, cyc); + off += _ldst_peripheral(pl330, dry_run, &buf[off], pxs, cyc, + cond); break; + case DMA_MEM_TO_MEM: off += _ldst_memtomem(dry_run, &buf[off], pxs, cyc); break; + + default: + /* this code should be unreachable */ + WARN_ON(1); + break; + } + + return off; +} + +/* + * transfer dregs with single transfers to peripheral, or a reduced size burst + * for mem-to-mem. + */ +static int _dregs(struct pl330_dmac *pl330, unsigned int dry_run, u8 buf[], + const struct _xfer_spec *pxs, int transfer_length) +{ + int off = 0; + int dregs_ccr; + + if (transfer_length == 0) + return off; + + switch (pxs->desc->rqtype) { + case DMA_MEM_TO_DEV: + /* fall through */ + case DMA_DEV_TO_MEM: + off += _ldst_peripheral(pl330, dry_run, &buf[off], pxs, + transfer_length, SINGLE); + break; + + case DMA_MEM_TO_MEM: + dregs_ccr = pxs->ccr; + dregs_ccr &= ~((0xf << CC_SRCBRSTLEN_SHFT) | + (0xf << CC_DSTBRSTLEN_SHFT)); + dregs_ccr |= (((transfer_length - 1) & 0xf) << + CC_SRCBRSTLEN_SHFT); + dregs_ccr |= (((transfer_length - 1) & 0xf) << + CC_DSTBRSTLEN_SHFT); + off += _emit_MOV(dry_run, &buf[off], CCR, dregs_ccr); + off += _ldst_memtomem(dry_run, &buf[off], pxs, 1); + break; + default: - off += 0x40000000; /* Scare off the Client */ + /* this code should be unreachable */ + WARN_ON(1); break; } @@ -1256,6 +1348,8 @@ static inline int _setup_loops(struct pl330_dmac *pl330, struct pl330_xfer *x = &pxs->desc->px; u32 ccr = pxs->ccr; unsigned long c, bursts = BYTE_TO_BURST(x->bytes, ccr); + int num_dregs = (x->bytes - BURST_TO_BYTE(bursts, ccr)) / + BRST_SIZE(ccr); int off = 0; while (bursts) { @@ -1263,6 +1357,7 @@ static inline int _setup_loops(struct pl330_dmac *pl330, off += _loop(pl330, dry_run, &buf[off], &c, pxs); bursts -= c; } + off += _dregs(pl330, dry_run, &buf[off], pxs, num_dregs); return off; } @@ -1294,7 +1389,6 @@ static int _setup_req(struct pl330_dmac *pl330, unsigned dry_run, struct _xfer_spec *pxs) { struct _pl330_req *req = &thrd->req[index]; - struct pl330_xfer *x; u8 *buf = req->mc_cpu; int off = 0; @@ -1303,11 +1397,6 @@ static int _setup_req(struct pl330_dmac *pl330, unsigned dry_run, /* DMAMOV CCR, ccr */ off += _emit_MOV(dry_run, &buf[off], CCR, pxs->ccr); - x = &pxs->desc->px; - /* Error if xfer length is not aligned at burst size */ - if (x->bytes % (BRST_SIZE(pxs->ccr) * BRST_LEN(pxs->ccr))) - return -EINVAL; - off += _setup_xfer(pl330, dry_run, &buf[off], pxs); /* DMASEV peripheral/event */ @@ -1365,6 +1454,20 @@ static int pl330_submit_req(struct pl330_thread *thrd, u32 ccr; int ret = 0; + switch (desc->rqtype) { + case DMA_MEM_TO_DEV: + break; + + case DMA_DEV_TO_MEM: + break; + + case DMA_MEM_TO_MEM: + break; + + default: + return -ENOTSUPP; + } + if (pl330->state == DYING || pl330->dmac_tbd.reset_chan & (1 << thrd->id)) { dev_info(thrd->dmac->ddma.dev, "%s:%d\n", @@ -2104,6 +2207,18 @@ static bool pl330_prep_slave_fifo(struct dma_pl330_chan *pch, return true; } +static int fixup_burst_len(int max_burst_len, int quirks) +{ + if (quirks & PL330_QUIRK_BROKEN_NO_FLUSHP) + return 1; + else if (max_burst_len > PL330_MAX_BURST) + return PL330_MAX_BURST; + else if (max_burst_len < 1) + return 1; + else + return max_burst_len; +} + static int pl330_config(struct dma_chan *chan, struct dma_slave_config *slave_config) { @@ -2115,15 +2230,15 @@ static int pl330_config(struct dma_chan *chan, pch->fifo_addr = slave_config->dst_addr; if (slave_config->dst_addr_width) pch->burst_sz = __ffs(slave_config->dst_addr_width); - if (slave_config->dst_maxburst) - pch->burst_len = slave_config->dst_maxburst; + pch->burst_len = fixup_burst_len(slave_config->dst_maxburst, + pch->dmac->quirks); } else if (slave_config->direction == DMA_DEV_TO_MEM) { if (slave_config->src_addr) pch->fifo_addr = slave_config->src_addr; if (slave_config->src_addr_width) pch->burst_sz = __ffs(slave_config->src_addr_width); - if (slave_config->src_maxburst) - pch->burst_len = slave_config->src_maxburst; + pch->burst_len = fixup_burst_len(slave_config->src_maxburst, + pch->dmac->quirks); } return 0; @@ -2517,14 +2632,8 @@ static inline int get_burst_len(struct dma_pl330_desc *desc, size_t len) burst_len >>= desc->rqcfg.brst_size; /* src/dst_burst_len can't be more than 16 */ - if (burst_len > 16) - burst_len = 16; - - while (burst_len > 1) { - if (!(len % (burst_len << desc->rqcfg.brst_size))) - break; - burst_len--; - } + if (burst_len > PL330_MAX_BURST) + burst_len = PL330_MAX_BURST; return burst_len; } @@ -2596,7 +2705,7 @@ static struct dma_async_tx_descriptor *pl330_prep_dma_cyclic( desc->rqtype = direction; desc->rqcfg.brst_size = pch->burst_sz; - desc->rqcfg.brst_len = 1; + desc->rqcfg.brst_len = pch->burst_len; desc->bytes_requested = period_len; fill_px(&desc->px, dst, src, period_len); @@ -2741,7 +2850,7 @@ pl330_prep_slave_sg(struct dma_chan *chan, struct scatterlist *sgl, } desc->rqcfg.brst_size = pch->burst_sz; - desc->rqcfg.brst_len = 1; + desc->rqcfg.brst_len = pch->burst_len; desc->rqtype = direction; desc->bytes_requested = sg_dma_len(sg); } -- 2.11.0