From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mx0b-0002e601.pphosted.com (mx0b-0002e601.pphosted.com [148.163.154.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 19E3A42BC39; Fri, 24 Jul 2026 10:22:26 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=fail smtp.client-ip=148.163.154.28 ARC-Seal:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784888550; cv=fail; b=Y7nzuYMIG6rnP5sgmMMHV+KYgob9KB1zDp3SIfoZPstjkMJFBac48UonnllEvDdPAOfisKXu0nfWu+l1bzFxDLtFcSIGCHBVZVpGyTAGfx6uodK9J79rsw33rIuuF91kW2Bt1R1sn0p7GVmtRjCwXUWqP4+k6jOzNI4ygKfk6MQ= ARC-Message-Signature:i=2; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1784888550; c=relaxed/simple; bh=sFwvcjjoMGSRQsXWKnuyW515g66pv1YKYEOiSQftFwo=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=hOkklQbplYxU8WVBaskZyvvAyKEAyL+3U/wlWlg+kJ8Kwzupb5CbS+xzfR7tvm5W1PwNh6s93LWdsbTpyelYAG0johtrXn/WCkkUZXt4oOG05hCAxjF7nlyQdNqwmoqdLe0kj4TZr83DqwIejJ+2FxbFg7W80M9ZriH4u1RoQUk= ARC-Authentication-Results:i=2; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com; spf=pass smtp.mailfrom=ti.com; dkim=pass (2048-bit key) header.d=ti.com header.i=@ti.com header.b=TAyFiw3P; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b=qnsrgQsF; arc=fail smtp.client-ip=148.163.154.28 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=ti.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=ti.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=ti.com header.i=@ti.com header.b="TAyFiw3P"; dkim=pass (1024-bit key) header.d=ti.com header.i=@ti.com header.b="qnsrgQsF" Received: from pps.filterd (m0374955.ppops.net [127.0.0.1]) by mx0b-0002e601.pphosted.com (8.18.1.11/8.18.1.11) with ESMTP id 66OAFTTq1093779; Fri, 24 Jul 2026 05:22:17 -0500 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; h=cc :content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to; s= proofpoint-05-2026; bh=zrebkiMGBZNbtJYxl/QbWwqr4JB1gEgZ5kZ3ZpUs4 i8=; b=TAyFiw3Pvpnw8UefXbNxrXnGVANh4UoVcuHrQQxqK/QdGlhEn2Qevb2lJ DENnMkJLAvjHj/sPLQw2eKOkm5y290EfSWi79EHgYuymUEoLZ6ZOF0gkNNoKRXdl nTCrdc0ikGtMlTnP97ZLzITyvhLUy+IOEJBN0g/A3EOio61h31FWvfn2v4HYrT9o L0NwEKK+L0nuEV5WM4uEdN597ljT4oza3xPnj1ASzJwuQS3o1ImjSxcXlBzH8K7C GTPt8CqGVhtW/N8o0uPCNo/SDLmG4Y5U4I2VgZr6+Vpl0EH3fmz1BIzHz6oBlNRv 2nyb/ClYmhl4PxpJH0IGotvWfkqzg== Received: from sn4pr0501cu005.outbound.protection.outlook.com (mail-southcentralusazon11011039.outbound.protection.outlook.com [40.93.194.39]) by mx0b-0002e601.pphosted.com (PPS) with ESMTPS id 4fkyp7j3xh-1 (version=TLSv1.3 cipher=TLS_AES_256_GCM_SHA384 bits=256 verify=NOT); Fri, 24 Jul 2026 05:22:16 -0500 (CDT) ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=kywPLaFwYNvj40i4WcD2ToXiZDmp2ZaxjBQOwqFtNpKjI0HmxdzEfkD2CQNuEYXxnLBWAZcMuRIlF0An8pAcgBErBr9NPSmaiQ+eXjr6DYoT6KUleKo944fn9BmUGSVHYnENvmaCCASxO6yyfVmrsiyww0sM6iMx61/oNwHSFdyZjrhjq5SEFWansSsd8cUtl64YHd2RKoGFHpN0zUMypItP0m70uOkLJfOdUtGc3NpzjuzGpXKnpk4eJjaT8DeQfTv++LOGPGERs5s+A12WLwbNHvAeXQp7C147sPczDhkAtu7J00H9EBkcc5u1vI6oALPYPBS9r8+/bvzsFfgzzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=zrebkiMGBZNbtJYxl/QbWwqr4JB1gEgZ5kZ3ZpUs4i8=; b=DXcSdEPJjR0DDh8/2xVvMJXNy52+sDZqJu18Ce/Iv3DCMPbYr0gwgPrNCIMsCWooqcLXSJBMIzvuKH2VAEIYpnTwrE8ZenSgjFTkQDQadV0A8TPeG4OyMlUeg23mSRD5OKRUrXSGnWTnMYgLmVwHYUpOYqTYlTPTr8ngpOT+UHGTZzjpDgdpzJ7x77DyWePfuSHSCoKvPQa+WnLxpFso11E48DrIK1bR1rBe52DsvMv4iV4UioPZjEacNuV6LZnaKz91qLPe2W4sYVNQfII3t1XKlS+Ss44i8pLlDVZfVCXstI1AwDfQzHQwmV7TXBVhB6nR8y7X9VTaTIZKf1De0Q== ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 198.47.21.194) smtp.rcpttodomain=nxp.com smtp.mailfrom=ti.com; dmarc=pass (p=quarantine sp=none pct=100) action=none header.from=ti.com; dkim=none (message not signed); arc=none (0) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ti.com; s=selector1; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=zrebkiMGBZNbtJYxl/QbWwqr4JB1gEgZ5kZ3ZpUs4i8=; b=qnsrgQsF3HxOPXs2hPClTWEVwKL33HK7Z0BLZ9HlQqPDq6XIyVBGE6cF9dPIEHhMqGJXjZgkyoPAI8RjYzej8xVwhaWirXCXN6OXk0lArhguND7eB3jdIG+V7Of+hIS8l85UyMJ91LYeLKqMhkhFxIWzhejXyZUuJ3jwXljXxFQ= Received: from MW4P222CA0019.NAMP222.PROD.OUTLOOK.COM (2603:10b6:303:114::24) by CYXPR10MB7976.namprd10.prod.outlook.com (2603:10b6:930:df::22) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.245.11; Fri, 24 Jul 2026 10:22:11 +0000 Received: from MWH0EPF000C6194.namprd02.prod.outlook.com (2603:10b6:303:114:cafe::58) by MW4P222CA0019.outlook.office365.com (2603:10b6:303:114::24) with Microsoft SMTP Server (version=TLS1_3, cipher=TLS_AES_256_GCM_SHA384) id 15.21.245.11 via Frontend Transport; Fri, 24 Jul 2026 10:22:11 +0000 X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 198.47.21.194) smtp.mailfrom=ti.com; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=ti.com; Received-SPF: Pass (protection.outlook.com: domain of ti.com designates 198.47.21.194 as permitted sender) receiver=protection.outlook.com; client-ip=198.47.21.194; helo=flwvzet200.ext.ti.com; pr=C Received: from flwvzet200.ext.ti.com (198.47.21.194) by MWH0EPF000C6194.mail.protection.outlook.com (10.167.249.104) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.21.270.5 via Frontend Transport; Fri, 24 Jul 2026 10:22:10 +0000 Received: from DFLE204.ent.ti.com (10.64.6.62) by flwvzet200.ext.ti.com (10.248.192.31) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Fri, 24 Jul 2026 05:22:08 -0500 Received: from DFLE208.ent.ti.com (10.64.6.66) by DFLE204.ent.ti.com (10.64.6.62) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37; Fri, 24 Jul 2026 05:22:08 -0500 Received: from lelvem-mr06.itg.ti.com (10.180.75.8) by DFLE208.ent.ti.com (10.64.6.66) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.2562.37 via Frontend Transport; Fri, 24 Jul 2026 05:22:08 -0500 Received: from uda0498651.dhcp.ti.com (uda0498651.dhcp.ti.com [10.24.51.254]) by lelvem-mr06.itg.ti.com (8.18.1/8.18.1) with ESMTP id 66OALMFN3017088; Fri, 24 Jul 2026 05:22:04 -0500 From: Sai Sree Kartheek Adivi To: , , , , , , , , , , , , , CC: , Subject: [PATCH v7 08/19] dmaengine: ti: k3-udma: move udma utility functions to k3-udma-common.c Date: Fri, 24 Jul 2026 15:50:41 +0530 Message-ID: <20260724102115.2854-9-s-adivi@ti.com> X-Mailer: git-send-email 2.54.0 In-Reply-To: <20260724102115.2854-1-s-adivi@ti.com> References: <20260724102115.2854-1-s-adivi@ti.com> Precedence: bulk X-Mailing-List: devicetree@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-C2ProcessedOrg: 333ef613-75bf-4e12-a4b1-8e3623f5dcea X-EOPAttributedMessage: 0 X-MS-PublicTrafficType: Email X-MS-TrafficTypeDiagnostic: MWH0EPF000C6194:EE_|CYXPR10MB7976:EE_ X-MS-Office365-Filtering-Correlation-Id: cc25f8c5-a1e1-40f4-666e-08dee96d6be7 X-MS-Exchange-SenderADCheck: 1 X-MS-Exchange-AntiSpam-Relay: 0 X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700016|23010399003|7416014|376014|82310400026|1800799024|921020|6133799003|56012099006|10067099003|5023799004|18002099003|22082099003|3023799007; X-Microsoft-Antispam-Message-Info: B++kmizM1jYxmfRaXYH3GOdvNsjLEoZ6APBTW+hV+zdlW9c6XGORJ74gc0JTtwiTYJL1Mw4F1BsfW0kdC1laWfqblm3OJjV6KJ7l2ZsH7BalZQxGJVE84hY2rE5TcNG7J4jJMDDMDl/NrpqxXfxqLsF8eolOjJDJeU0TNLn9Jr2cbQbPUPpeMAtVJLM/J402zsB3EOs6T8NHPsdNSPoClnjfqk1BJRD301he3rGVnv2LVmOKE5ZfLNToMtChgGN/hUydYIIt5T6UAWag1wbePO9rKJLJn6FzUU8Il+W4uZUnHsLEGG/H/B52FWg9Nk5tw2HvwfXdSD3onr19RbiqIrL+BmtvwBsLOengQBTe3tSPyVIR4I3DXScIYgZjOn2vCoI9eUvyTbfwGC6/xuHXQujMYELjIcOS8zVNRKYn06C+6bMP+V8oFU3xZ62WbjY75iy6JflWuRdoI0GT6lSBtHhZW3MtpiYH3ryj/JObsyxrTLqGhHzKF3T/NcbPSshlcRK8+2dDtYDs5ZQ03gEZpBiW+//3pCKKb2nWnZtT0w+aUBD9PD035t+JdkCTLePCU6748EahW27jaTwKuRuHu13hsZ1L5Ns6p4oPgTstRKg3wiqaw/dE4wFe5UciZVdB7ln2Hkir5VM1Uf/b7rxeltRWugn1x7yWsPNtXQefGkytuzexoL0YpY0rGyHkyFDaHT4g+WABMgQfknv2BKBH82sB0crbFe6YVCCVjIaobMBmFZgZDcatWFW73APyuSn0 X-Forefront-Antispam-Report: CIP:198.47.21.194;CTRY:US;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:flwvzet200.ext.ti.com;PTR:ErrorRetry;CAT:NONE;SFS:(13230040)(36860700016)(23010399003)(7416014)(376014)(82310400026)(1800799024)(921020)(6133799003)(56012099006)(10067099003)(5023799004)(18002099003)(22082099003)(3023799007);DIR:OUT;SFP:1101; X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1 X-MS-Exchange-AntiSpam-MessageData-0: fm2/t8F6PnAxgmaeJe7tDBCU7skIKF8i36pxPwaQZASGWPu+W8WHLuGDoBGceFniOPeAVWVTfJ5EvO+96vN7YfQ8RKuSR5jkW1c4KvgJ34WYh1En1KgfdQQ2RqmiCybSPBZMQXPXdw5/OxpCUE+a+2l4wqlo5vHA2EiQJBq1qDF8Hi4APu1NKCydI6q9sJpm3+LloFoCCTjj1Y7nJFLmDfK3uwrlt6LPdhPHFATOKscrR43anvzbDGKvuyVj3nJajmWsCozTWj5JBjTgxIxoUa3LmwoDnv0UXMmiIuqW1/TOMHIQw6lf2xwaYTkTlS91QFezJj4I9r2L2RZtZt1MjmvYB0rWMgNi5e2P27rJV8QyQtwgpbxVOjLY3gZd1LXHoZWO1tUxLV4+0Pde9aLm9kPJyCxPxDppsR5+kzCVxbuFjqT+SCt7R64LCMHG6HwJ X-Exchange-RoutingPolicyChecked: jK7oDnensGMd4d8HL4Pw+YQ97husIY09hnuEvfDFtyGbVuvTe++tr2xzbE17NzerdfiVJSbI98hJ26fQvZX1yDccnJxZvB8skN+2vcruu+c/QE/9YSYAmQf8JFzEOHFmRX+QvUXEjzW0Nf/R4Hn54D/To4xmifwHRBM1xHhVLMJrmkP+5dx9PXsGO+K/qCoCmapIn/Twgzgncf/3beZVYOFZH0y+dMp1OEbgGe1fPRKaTl4rPUuJLJPBDApPljeUpNmwhDW1dQYCxBhJMO+QyZ4R8teH/X51dQ04aufE7xaM7MAnLB98YO0ISXFvN2HgQslDExejELzdOqfCecOT/g== X-OriginatorOrg: ti.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Jul 2026 10:22:10.2423 (UTC) X-MS-Exchange-CrossTenant-Network-Message-Id: cc25f8c5-a1e1-40f4-666e-08dee96d6be7 X-MS-Exchange-CrossTenant-Id: e5b49634-450b-4709-8abb-1e2b19b982b7 X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e5b49634-450b-4709-8abb-1e2b19b982b7;Ip=[198.47.21.194];Helo=[flwvzet200.ext.ti.com] X-MS-Exchange-CrossTenant-AuthSource: MWH0EPF000C6194.namprd02.prod.outlook.com X-MS-Exchange-CrossTenant-AuthAs: Anonymous X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYXPR10MB7976 X-Authority-Analysis: v=2.4 cv=M4p97Sws c=1 sm=1 tr=0 ts=6a633cd9 cx=c_pps a=ChtxJ3izA4DGUCEzIvXZbg==:117 a=iwqwCZQqcuTv3JOpYdM7/Q==:17 a=6eWqkTHjU83fiwn7nKZWdM+Sl24=:19 a=RAioF0-LDSMA:10 a=V5UXEbMT0ywA:10 a=VkNPw1HP01LnGYTKEx00:22 a=Z8NIEmU8O1QQgoT56wFK:22 a=fPAWb5peG099m5CrUpKH:22 a=sozttTNsAAAA:8 a=bI_GjgVLxrynCmwSAzAA:9 X-Proofpoint-Spam-Info: AW1haW4tMjYwNzI0MDA5NCBTYWx0ZWRfX8mFKRnDdFHhK eii1hDuQ8eZT+erIpnqr/6geWszapEZCu1nW+E8GffefHtHhQZF6I9SO4QXs0Zdz8FkJZ5AsKRU 8mxQZv+0WWkXMTCGpW270GXCQRGsKts= X-Proofpoint-ORIG-GUID: rjc1aLqpWskSRTAgPQEVduSMzNaUlZnz X-Proofpoint-GUID: rjc1aLqpWskSRTAgPQEVduSMzNaUlZnz X-Proofpoint-Spam-Details-Enc: AW1haW4tMjYwNzI0MDA5NCBTYWx0ZWRfX5YE9yfGYE837 LdFWPUiHUlYqGKS0b9d2RGMgz11BjIhJaeIOdirWrU0hGbYF8y8NaNF3lKsbi6r1FG8BWSGMpmZ FgHiapl9t4QzGMeomaU6IUCvviP/zKB33XaU6FQWsYPxcUj2vM1oMVSSvk6Fqe35m+jXBkbPNex x1818xnYHIVem0AGiBKFA38RmiXzy/9rZWfhLYferwTZpoUNCODOMdkTReX3RHbJOQxGbwtcJ08 StA4xE2Pw1gctyAtVJH28yBusSE1aEclact4zvxUD2LjeI6uJwjqP08s8jA8xZuRlLZmuB3Rj11 UIoxyhlknXrsujXnPOWhWWnoSe2aXArVXBuSs3nACj9lvL9ezCpmn6plyd88z0aK5IwN2BUdKzC Q2HACjLcmvpzxF+kkgAW6zkvmjf7eUwQaZbW77tYKUptV9wiTIhyaobdiza8cXZUNDw0xC4p/Ss EmCeI//EXHD1JrZNvQA== X-Proofpoint-Virus-Version: vendor=baseguard engine=ICAP:2.0.293,Aquarius:18.0.1143,Hydra:6.1.134,FMLib:17.12.100.49 definitions=2026-07-24_02,2026-07-22_02,2025-10-01_01 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 malwarescore=0 spamscore=0 priorityscore=1501 adultscore=0 clxscore=1015 suspectscore=0 phishscore=0 bulkscore=0 lowpriorityscore=0 impostorscore=0 classifier=typeunknown authscore=0 authtc= authcc= route=outbound adjust=0 reason=mlx scancount=1 engine=8.22.0-2606150000 definitions=main-2607240094 Relocate udma utility functions from k3-udma.c to k3-udma-common.c file. The implementation of these functions is largely shared between K3 UDMA and K3 UDMA v2. This refactor improves code reuse and maintainability across multiple variants. No functional changes intended. Signed-off-by: Sai Sree Kartheek Adivi --- drivers/dma/ti/k3-udma-common.c | 549 ++++++++++++++++++++++++++++++++ drivers/dma/ti/k3-udma.c | 531 ------------------------------ drivers/dma/ti/k3-udma.h | 28 ++ 3 files changed, 577 insertions(+), 531 deletions(-) diff --git a/drivers/dma/ti/k3-udma-common.c b/drivers/dma/ti/k3-udma-common.c index fc5e2c99ee28a..01aea77c25715 100644 --- a/drivers/dma/ti/k3-udma-common.c +++ b/drivers/dma/ti/k3-udma-common.c @@ -4,6 +4,7 @@ * Author: Peter Ujfalusi */ +#include #include #include #include @@ -46,6 +47,28 @@ struct udma_desc *udma_udma_desc_from_paddr(struct udma_chan *uc, } EXPORT_SYMBOL_GPL(udma_udma_desc_from_paddr); +void udma_start_desc(struct udma_chan *uc) +{ + struct udma_chan_config *ucc = &uc->config; + + if (uc->ud->match_data->type == DMA_TYPE_UDMA && ucc->pkt_mode && + (uc->cyclic || ucc->dir == DMA_DEV_TO_MEM)) { + int i; + + /* + * UDMA only: Push all descriptors to ring for packet mode + * cyclic or RX + * PKTDMA supports pre-linked descriptor and cyclic is not + * supported + */ + for (i = 0; i < uc->desc->sglen; i++) + udma_push_to_ring(uc, i); + } else { + udma_push_to_ring(uc, 0); + } +} +EXPORT_SYMBOL_GPL(udma_start_desc); + void udma_free_hwdesc(struct udma_chan *uc, struct udma_desc *d) { if (uc->use_dma_pool) { @@ -1342,5 +1365,531 @@ void udma_reset_rings(struct udma_chan *uc) } EXPORT_SYMBOL_GPL(udma_reset_rings); +u8 udma_get_chan_tpl_index(struct udma_tpl *tpl_map, int chan_id) +{ + int i; + + for (i = 0; i < tpl_map->levels; i++) { + if (chan_id >= tpl_map->start_idx[i]) + return i; + } + + return 0; +} +EXPORT_SYMBOL_GPL(udma_get_chan_tpl_index); + +void k3_configure_chan_coherency(struct dma_chan *chan, u32 asel) +{ + struct device *chan_dev = &chan->dev->device; + + if (asel == 0) { + /* No special handling for the channel */ + chan->dev->chan_dma_dev = false; + + dev_clear_dma_coherent(chan_dev); + chan_dev->dma_parms = NULL; + } else if (asel == 14 || asel == 15) { + chan->dev->chan_dma_dev = true; + + dev_set_dma_coherent(chan_dev); + dma_coerce_mask_and_coherent(chan_dev, DMA_BIT_MASK(48)); + chan_dev->dma_parms = chan_dev->parent->dma_parms; + } else { + dev_warn(chan->device->dev, "Invalid ASEL value: %u\n", asel); + + dev_clear_dma_coherent(chan_dev); + chan_dev->dma_parms = NULL; + } +} +EXPORT_SYMBOL_GPL(k3_configure_chan_coherency); + +void udma_reset_uchan(struct udma_chan *uc) +{ + memset(&uc->config, 0, sizeof(uc->config)); + uc->config.remote_thread_id = -1; + uc->config.mapped_channel_id = -1; + uc->config.default_flow_id = -1; + uc->state = UDMA_CHAN_IS_IDLE; +} +EXPORT_SYMBOL_GPL(udma_reset_uchan); + +void udma_dump_chan_stdata(struct udma_chan *uc) +{ + struct device *dev = uc->ud->dev; + u32 offset; + int i; + + if (uc->config.dir == DMA_MEM_TO_DEV || uc->config.dir == DMA_MEM_TO_MEM) { + dev_dbg(dev, "TCHAN State data:\n"); + for (i = 0; i < 32; i++) { + offset = UDMA_CHAN_RT_STDATA_REG + i * 4; + dev_dbg(dev, "TRT_STDATA[%02d]: 0x%08x\n", i, + udma_tchanrt_read(uc, offset)); + } + } + + if (uc->config.dir == DMA_DEV_TO_MEM || uc->config.dir == DMA_MEM_TO_MEM) { + dev_dbg(dev, "RCHAN State data:\n"); + for (i = 0; i < 32; i++) { + offset = UDMA_CHAN_RT_STDATA_REG + i * 4; + dev_dbg(dev, "RRT_STDATA[%02d]: 0x%08x\n", i, + udma_rchanrt_read(uc, offset)); + } + } +} + +bool udma_is_chan_running(struct udma_chan *uc) +{ + u32 trt_ctl = 0; + u32 rrt_ctl = 0; + + if (uc->tchan) + trt_ctl = udma_tchanrt_read(uc, UDMA_CHAN_RT_CTL_REG); + if (uc->rchan) + rrt_ctl = udma_rchanrt_read(uc, UDMA_CHAN_RT_CTL_REG); + + if (trt_ctl & UDMA_CHAN_RT_CTL_EN || rrt_ctl & UDMA_CHAN_RT_CTL_EN) + return true; + + return false; +} +EXPORT_SYMBOL_GPL(udma_is_chan_running); + +bool udma_chan_needs_reconfiguration(struct udma_chan *uc) +{ + /* Only PDMAs have staticTR */ + if (uc->config.ep_type == PSIL_EP_NATIVE) + return false; + + /* Check if the staticTR configuration has changed for TX */ + if (memcmp(&uc->static_tr, &uc->desc->static_tr, sizeof(uc->static_tr))) + return true; + + return false; +} +EXPORT_SYMBOL_GPL(udma_chan_needs_reconfiguration); + +void udma_cyclic_packet_elapsed(struct udma_chan *uc) +{ + struct udma_desc *d = uc->desc; + struct cppi5_host_desc_t *h_desc; + + h_desc = d->hwdesc[d->desc_idx].cppi5_desc_vaddr; + cppi5_hdesc_reset_to_original(h_desc); + udma_push_to_ring(uc, d->desc_idx); + d->desc_idx = (d->desc_idx + 1) % d->sglen; +} +EXPORT_SYMBOL_GPL(udma_cyclic_packet_elapsed); + +void udma_check_tx_completion(struct work_struct *work) +{ + struct udma_chan *uc = container_of(work, typeof(*uc), + tx_drain.work.work); + bool desc_done = true; + u32 residue_diff; + ktime_t time_diff; + unsigned long delay; + unsigned long flags; + + while (1) { + spin_lock_irqsave(&uc->vc.lock, flags); + + if (uc->desc) { + /* Get previous residue and time stamp */ + residue_diff = uc->tx_drain.residue; + time_diff = uc->tx_drain.tstamp; + /* + * Get current residue and time stamp or see if + * transfer is complete + */ + desc_done = udma_is_desc_really_done(uc, uc->desc); + } + + if (!desc_done) { + /* + * Find the time delta and residue delta w.r.t + * previous poll + */ + time_diff = ktime_sub(uc->tx_drain.tstamp, + time_diff) + 1; + residue_diff -= uc->tx_drain.residue; + if (residue_diff) { + /* + * Try to guess when we should check + * next time by calculating rate at + * which data is being drained at the + * peer device + */ + delay = (time_diff / residue_diff) * + uc->tx_drain.residue; + } else { + /* No progress, check again in 1 second */ + schedule_delayed_work(&uc->tx_drain.work, HZ); + break; + } + + spin_unlock_irqrestore(&uc->vc.lock, flags); + + usleep_range(ktime_to_us(delay), + ktime_to_us(delay) + 10); + continue; + } + + if (uc->desc) { + struct udma_desc *d = uc->desc; + + uc->ud->decrement_byte_counters(uc, d->residue); + uc->ud->start(uc); + vchan_cookie_complete(&d->vd); + break; + } + + break; + } + + spin_unlock_irqrestore(&uc->vc.lock, flags); +} +EXPORT_SYMBOL_GPL(udma_check_tx_completion); + +int udma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg) +{ + struct udma_chan *uc = to_udma_chan(chan); + + memcpy(&uc->cfg, cfg, sizeof(uc->cfg)); + + return 0; +} +EXPORT_SYMBOL_GPL(udma_slave_config); + +void udma_issue_pending(struct dma_chan *chan) +{ + struct udma_chan *uc = to_udma_chan(chan); + unsigned long flags; + + spin_lock_irqsave(&uc->vc.lock, flags); + + /* If we have something pending and no active descriptor, then */ + if (vchan_issue_pending(&uc->vc) && !uc->desc) { + /* + * start a descriptor if the channel is NOT [marked as + * terminating _and_ it is still running (teardown has not + * completed yet)]. + */ + if (!(uc->state == UDMA_CHAN_IS_TERMINATING && + udma_is_chan_running(uc))) + uc->ud->start(uc); + } + + spin_unlock_irqrestore(&uc->vc.lock, flags); +} +EXPORT_SYMBOL_GPL(udma_issue_pending); + +int udma_terminate_all(struct dma_chan *chan) +{ + struct udma_chan *uc = to_udma_chan(chan); + unsigned long flags; + LIST_HEAD(head); + + spin_lock_irqsave(&uc->vc.lock, flags); + + if (udma_is_chan_running(uc)) + uc->ud->stop(uc); + + if (uc->desc) { + uc->terminated_desc = uc->desc; + uc->desc = NULL; + uc->terminated_desc->terminated = true; + cancel_delayed_work(&uc->tx_drain.work); + } + + uc->paused = false; + + vchan_get_all_descriptors(&uc->vc, &head); + spin_unlock_irqrestore(&uc->vc.lock, flags); + vchan_dma_desc_free_list(&uc->vc, &head); + + return 0; +} +EXPORT_SYMBOL_GPL(udma_terminate_all); + +void udma_synchronize(struct dma_chan *chan) +{ + struct udma_chan *uc = to_udma_chan(chan); + unsigned long timeout = msecs_to_jiffies(1000); + + vchan_synchronize(&uc->vc); + + if (uc->state == UDMA_CHAN_IS_TERMINATING) { + timeout = wait_for_completion_timeout(&uc->teardown_completed, + timeout); + if (!timeout) { + dev_warn(uc->ud->dev, "chan%d teardown timeout!\n", + uc->id); + udma_dump_chan_stdata(uc); + uc->ud->reset_chan(uc, true); + } + } + + uc->ud->reset_chan(uc, false); + if (udma_is_chan_running(uc)) + dev_warn(uc->ud->dev, "chan%d refused to stop!\n", uc->id); + + cancel_delayed_work_sync(&uc->tx_drain.work); + udma_reset_rings(uc); +} +EXPORT_SYMBOL_GPL(udma_synchronize); + +/* + * This tasklet handles the completion of a DMA descriptor by + * calling its callback and freeing it. + */ +void udma_vchan_complete(struct tasklet_struct *t) +{ + struct virt_dma_chan *vc = from_tasklet(vc, t, task); + struct virt_dma_desc *vd, *_vd; + struct dmaengine_desc_callback cb; + LIST_HEAD(head); + + spin_lock_irq(&vc->lock); + list_splice_tail_init(&vc->desc_completed, &head); + vd = vc->cyclic; + if (vd) { + vc->cyclic = NULL; + dmaengine_desc_get_callback(&vd->tx, &cb); + } else { + memset(&cb, 0, sizeof(cb)); + } + spin_unlock_irq(&vc->lock); + + udma_desc_pre_callback(vc, vd, NULL); + dmaengine_desc_callback_invoke(&cb, NULL); + + list_for_each_entry_safe(vd, _vd, &head, node) { + struct dmaengine_result result; + + dmaengine_desc_get_callback(&vd->tx, &cb); + + list_del(&vd->node); + + udma_desc_pre_callback(vc, vd, &result); + dmaengine_desc_callback_invoke(&cb, &result); + + vchan_vdesc_fini(vd); + } +} +EXPORT_SYMBOL_GPL(udma_vchan_complete); + +void udma_mark_resource_ranges(struct udma_dev *ud, unsigned long *map, + struct ti_sci_resource_desc *rm_desc, + char *name) +{ + bitmap_clear(map, rm_desc->start, rm_desc->num); + bitmap_clear(map, rm_desc->start_sec, rm_desc->num_sec); + dev_dbg(ud->dev, "ti_sci resource range for %s: %d:%d | %d:%d\n", name, + rm_desc->start, rm_desc->num, rm_desc->start_sec, + rm_desc->num_sec); +} +EXPORT_SYMBOL_GPL(udma_mark_resource_ranges); + +int udma_setup_rx_flush(struct udma_dev *ud) +{ + struct udma_rx_flush *rx_flush = &ud->rx_flush; + struct cppi5_desc_hdr_t *tr_desc; + struct cppi5_tr_type1_t *tr_req; + struct cppi5_host_desc_t *desc; + struct device *dev = ud->dev; + struct udma_hwdesc *hwdesc; + size_t tr_size; + + /* Allocate 1K buffer for discarded data on RX channel teardown */ + rx_flush->buffer_size = SZ_1K; + rx_flush->buffer_vaddr = devm_kzalloc(dev, rx_flush->buffer_size, + GFP_KERNEL); + if (!rx_flush->buffer_vaddr) + return -ENOMEM; + + rx_flush->buffer_paddr = dma_map_single(dev, rx_flush->buffer_vaddr, + rx_flush->buffer_size, + DMA_TO_DEVICE); + if (dma_mapping_error(dev, rx_flush->buffer_paddr)) + return -ENOMEM; + + /* Set up descriptor to be used for TR mode */ + hwdesc = &rx_flush->hwdescs[0]; + tr_size = sizeof(struct cppi5_tr_type1_t); + hwdesc->cppi5_desc_size = cppi5_trdesc_calc_size(tr_size, 1); + hwdesc->cppi5_desc_size = ALIGN(hwdesc->cppi5_desc_size, + ud->desc_align); + + hwdesc->cppi5_desc_vaddr = devm_kzalloc(dev, hwdesc->cppi5_desc_size, + GFP_KERNEL); + if (!hwdesc->cppi5_desc_vaddr) + return -ENOMEM; + + hwdesc->cppi5_desc_paddr = dma_map_single(dev, hwdesc->cppi5_desc_vaddr, + hwdesc->cppi5_desc_size, + DMA_TO_DEVICE); + if (dma_mapping_error(dev, hwdesc->cppi5_desc_paddr)) + return -ENOMEM; + + /* Start of the TR req records */ + hwdesc->tr_req_base = hwdesc->cppi5_desc_vaddr + tr_size; + /* Start address of the TR response array */ + hwdesc->tr_resp_base = hwdesc->tr_req_base + tr_size; + + tr_desc = hwdesc->cppi5_desc_vaddr; + cppi5_trdesc_init(tr_desc, 1, tr_size, 0, 0); + cppi5_desc_set_pktids(tr_desc, 0, CPPI5_INFO1_DESC_FLOWID_DEFAULT); + cppi5_desc_set_retpolicy(tr_desc, 0, 0); + + tr_req = hwdesc->tr_req_base; + cppi5_tr_init(&tr_req->flags, CPPI5_TR_TYPE1, false, false, + CPPI5_TR_EVENT_SIZE_COMPLETION, 0); + cppi5_tr_csf_set(&tr_req->flags, CPPI5_TR_CSF_SUPR_EVT); + + tr_req->addr = rx_flush->buffer_paddr; + tr_req->icnt0 = rx_flush->buffer_size; + tr_req->icnt1 = 1; + + dma_sync_single_for_device(dev, hwdesc->cppi5_desc_paddr, + hwdesc->cppi5_desc_size, DMA_TO_DEVICE); + + /* Set up descriptor to be used for packet mode */ + hwdesc = &rx_flush->hwdescs[1]; + hwdesc->cppi5_desc_size = ALIGN(sizeof(struct cppi5_host_desc_t) + + CPPI5_INFO0_HDESC_EPIB_SIZE + + CPPI5_INFO0_HDESC_PSDATA_MAX_SIZE, + ud->desc_align); + + hwdesc->cppi5_desc_vaddr = devm_kzalloc(dev, hwdesc->cppi5_desc_size, + GFP_KERNEL); + if (!hwdesc->cppi5_desc_vaddr) + return -ENOMEM; + + hwdesc->cppi5_desc_paddr = dma_map_single(dev, hwdesc->cppi5_desc_vaddr, + hwdesc->cppi5_desc_size, + DMA_TO_DEVICE); + if (dma_mapping_error(dev, hwdesc->cppi5_desc_paddr)) + return -ENOMEM; + + desc = hwdesc->cppi5_desc_vaddr; + cppi5_hdesc_init(desc, 0, 0); + cppi5_desc_set_pktids(&desc->hdr, 0, CPPI5_INFO1_DESC_FLOWID_DEFAULT); + cppi5_desc_set_retpolicy(&desc->hdr, 0, 0); + + cppi5_hdesc_attach_buf(desc, + rx_flush->buffer_paddr, rx_flush->buffer_size, + rx_flush->buffer_paddr, rx_flush->buffer_size); + + dma_sync_single_for_device(dev, hwdesc->cppi5_desc_paddr, + hwdesc->cppi5_desc_size, DMA_TO_DEVICE); + return 0; +} +EXPORT_SYMBOL_GPL(udma_setup_rx_flush); + +#ifdef CONFIG_DEBUG_FS +void udma_dbg_summary_show_chan(struct seq_file *s, + struct dma_chan *chan) +{ + struct udma_chan *uc = to_udma_chan(chan); + struct udma_chan_config *ucc = &uc->config; + + seq_printf(s, " %-13s| %s", dma_chan_name(chan), + chan->dbg_client_name ?: "in-use"); + if (ucc->tr_trigger_type) + seq_puts(s, " (triggered, "); + else + seq_printf(s, " (%s, ", + dmaengine_get_direction_text(uc->config.dir)); + + switch (uc->config.dir) { + case DMA_MEM_TO_MEM: + if (uc->ud->match_data->type == DMA_TYPE_BCDMA) { + seq_printf(s, "bchan%d)\n", uc->bchan->id); + return; + } + + seq_printf(s, "chan%d pair [0x%04x -> 0x%04x], ", uc->tchan->id, + ucc->src_thread, ucc->dst_thread); + break; + case DMA_DEV_TO_MEM: + seq_printf(s, "rchan%d [0x%04x -> 0x%04x], ", uc->rchan->id, + ucc->src_thread, ucc->dst_thread); + if (uc->ud->match_data->type == DMA_TYPE_PKTDMA) + seq_printf(s, "rflow%d, ", uc->rflow->id); + break; + case DMA_MEM_TO_DEV: + seq_printf(s, "tchan%d [0x%04x -> 0x%04x], ", uc->tchan->id, + ucc->src_thread, ucc->dst_thread); + if (uc->ud->match_data->type == DMA_TYPE_PKTDMA) + seq_printf(s, "tflow%d, ", uc->tchan->tflow_id); + break; + default: + seq_puts(s, ")\n"); + return; + } + + if (ucc->ep_type == PSIL_EP_NATIVE) { + seq_puts(s, "PSI-L Native"); + if (ucc->metadata_size) { + seq_printf(s, "[%s", ucc->needs_epib ? " EPIB" : ""); + if (ucc->psd_size) + seq_printf(s, " PSDsize:%u", ucc->psd_size); + seq_puts(s, " ]"); + } + } else { + seq_puts(s, "PDMA"); + if (ucc->enable_acc32 || ucc->enable_burst) + seq_printf(s, "[%s%s ]", + ucc->enable_acc32 ? " ACC32" : "", + ucc->enable_burst ? " BURST" : ""); + } + + seq_printf(s, ", %s)\n", ucc->pkt_mode ? "Packet mode" : "TR mode"); +} + +void udma_dbg_summary_show(struct seq_file *s, + struct dma_device *dma_dev) +{ + struct dma_chan *chan; + + list_for_each_entry(chan, &dma_dev->channels, device_node) { + if (chan->client_count) + udma_dbg_summary_show_chan(s, chan); + } +} +EXPORT_SYMBOL_GPL(udma_dbg_summary_show); +#endif /* CONFIG_DEBUG_FS */ + +enum dmaengine_alignment udma_get_copy_align(struct udma_dev *ud) +{ + const struct udma_match_data *match_data = ud->match_data; + u8 tpl; + + if (!match_data->enable_memcpy_support) + return DMAENGINE_ALIGN_8_BYTES; + + /* Get the highest TPL level the device supports for memcpy */ + if (ud->bchan_cnt) + tpl = udma_get_chan_tpl_index(&ud->bchan_tpl, 0); + else if (ud->tchan_cnt) + tpl = udma_get_chan_tpl_index(&ud->tchan_tpl, 0); + else + return DMAENGINE_ALIGN_8_BYTES; + + switch (match_data->burst_size[tpl]) { + case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_256_BYTES: + return DMAENGINE_ALIGN_256_BYTES; + case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_128_BYTES: + return DMAENGINE_ALIGN_128_BYTES; + case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_64_BYTES: + fallthrough; + default: + return DMAENGINE_ALIGN_64_BYTES; + } +} +EXPORT_SYMBOL_GPL(udma_get_copy_align); + MODULE_DESCRIPTION("Texas Instruments K3 UDMA Common Library"); MODULE_LICENSE("GPL v2"); diff --git a/drivers/dma/ti/k3-udma.c b/drivers/dma/ti/k3-udma.c index 195ff948548c3..89c887bc86a34 100644 --- a/drivers/dma/ti/k3-udma.c +++ b/drivers/dma/ti/k3-udma.c @@ -61,92 +61,6 @@ int navss_psil_unpair(struct udma_dev *ud, u32 src_thread, src_thread, dst_thread); } -static void k3_configure_chan_coherency(struct dma_chan *chan, u32 asel) -{ - struct device *chan_dev = &chan->dev->device; - - if (asel == 0) { - /* No special handling for the channel */ - chan->dev->chan_dma_dev = false; - - dev_clear_dma_coherent(chan_dev); - chan_dev->dma_parms = NULL; - } else if (asel == 14 || asel == 15) { - chan->dev->chan_dma_dev = true; - - dev_set_dma_coherent(chan_dev); - dma_coerce_mask_and_coherent(chan_dev, DMA_BIT_MASK(48)); - chan_dev->dma_parms = chan_dev->parent->dma_parms; - } else { - dev_warn(chan->device->dev, "Invalid ASEL value: %u\n", asel); - - dev_clear_dma_coherent(chan_dev); - chan_dev->dma_parms = NULL; - } -} - -static u8 udma_get_chan_tpl_index(struct udma_tpl *tpl_map, int chan_id) -{ - int i; - - for (i = 0; i < tpl_map->levels; i++) { - if (chan_id >= tpl_map->start_idx[i]) - return i; - } - - return 0; -} - -static void udma_reset_uchan(struct udma_chan *uc) -{ - memset(&uc->config, 0, sizeof(uc->config)); - uc->config.remote_thread_id = -1; - uc->config.mapped_channel_id = -1; - uc->config.default_flow_id = -1; - uc->state = UDMA_CHAN_IS_IDLE; -} - -static void udma_dump_chan_stdata(struct udma_chan *uc) -{ - struct device *dev = uc->ud->dev; - u32 offset; - int i; - - if (uc->config.dir == DMA_MEM_TO_DEV || uc->config.dir == DMA_MEM_TO_MEM) { - dev_dbg(dev, "TCHAN State data:\n"); - for (i = 0; i < 32; i++) { - offset = UDMA_CHAN_RT_STDATA_REG + i * 4; - dev_dbg(dev, "TRT_STDATA[%02d]: 0x%08x\n", i, - udma_tchanrt_read(uc, offset)); - } - } - - if (uc->config.dir == DMA_DEV_TO_MEM || uc->config.dir == DMA_MEM_TO_MEM) { - dev_dbg(dev, "RCHAN State data:\n"); - for (i = 0; i < 32; i++) { - offset = UDMA_CHAN_RT_STDATA_REG + i * 4; - dev_dbg(dev, "RRT_STDATA[%02d]: 0x%08x\n", i, - udma_rchanrt_read(uc, offset)); - } - } -} - -static bool udma_is_chan_running(struct udma_chan *uc) -{ - u32 trt_ctl = 0; - u32 rrt_ctl = 0; - - if (uc->tchan) - trt_ctl = udma_tchanrt_read(uc, UDMA_CHAN_RT_CTL_REG); - if (uc->rchan) - rrt_ctl = udma_rchanrt_read(uc, UDMA_CHAN_RT_CTL_REG); - - if (trt_ctl & UDMA_CHAN_RT_CTL_EN || rrt_ctl & UDMA_CHAN_RT_CTL_EN) - return true; - - return false; -} - static bool udma_is_chan_paused(struct udma_chan *uc) { u32 val, pause_mask; @@ -275,40 +189,6 @@ static int udma_reset_chan(struct udma_chan *uc, bool hard) return 0; } -static void udma_start_desc(struct udma_chan *uc) -{ - struct udma_chan_config *ucc = &uc->config; - - if (uc->ud->match_data->type == DMA_TYPE_UDMA && ucc->pkt_mode && - (uc->cyclic || ucc->dir == DMA_DEV_TO_MEM)) { - int i; - - /* - * UDMA only: Push all descriptors to ring for packet mode - * cyclic or RX - * PKTDMA supports pre-linked descriptor and cyclic is not - * supported - */ - for (i = 0; i < uc->desc->sglen; i++) - udma_push_to_ring(uc, i); - } else { - udma_push_to_ring(uc, 0); - } -} - -static bool udma_chan_needs_reconfiguration(struct udma_chan *uc) -{ - /* Only PDMAs have staticTR */ - if (uc->config.ep_type == PSIL_EP_NATIVE) - return false; - - /* Check if the staticTR configuration has changed for TX */ - if (memcmp(&uc->static_tr, &uc->desc->static_tr, sizeof(uc->static_tr))) - return true; - - return false; -} - static int udma_start(struct udma_chan *uc) { struct virt_dma_desc *vd = vchan_next_desc(&uc->vc); @@ -453,86 +333,6 @@ static int udma_stop(struct udma_chan *uc) return 0; } -static void udma_cyclic_packet_elapsed(struct udma_chan *uc) -{ - struct udma_desc *d = uc->desc; - struct cppi5_host_desc_t *h_desc; - - h_desc = d->hwdesc[d->desc_idx].cppi5_desc_vaddr; - cppi5_hdesc_reset_to_original(h_desc); - udma_push_to_ring(uc, d->desc_idx); - d->desc_idx = (d->desc_idx + 1) % d->sglen; -} - -static void udma_check_tx_completion(struct work_struct *work) -{ - struct udma_chan *uc = container_of(work, typeof(*uc), - tx_drain.work.work); - bool desc_done = true; - u32 residue_diff; - ktime_t time_diff; - unsigned long delay; - unsigned long flags; - - while (1) { - spin_lock_irqsave(&uc->vc.lock, flags); - - if (uc->desc) { - /* Get previous residue and time stamp */ - residue_diff = uc->tx_drain.residue; - time_diff = uc->tx_drain.tstamp; - /* - * Get current residue and time stamp or see if - * transfer is complete - */ - desc_done = udma_is_desc_really_done(uc, uc->desc); - } - - if (!desc_done) { - /* - * Find the time delta and residue delta w.r.t - * previous poll - */ - time_diff = ktime_sub(uc->tx_drain.tstamp, - time_diff) + 1; - residue_diff -= uc->tx_drain.residue; - if (residue_diff) { - /* - * Try to guess when we should check - * next time by calculating rate at - * which data is being drained at the - * peer device - */ - delay = (time_diff / residue_diff) * - uc->tx_drain.residue; - } else { - /* No progress, check again in 1 second */ - schedule_delayed_work(&uc->tx_drain.work, HZ); - break; - } - - spin_unlock_irqrestore(&uc->vc.lock, flags); - - usleep_range(ktime_to_us(delay), - ktime_to_us(delay) + 10); - continue; - } - - if (uc->desc) { - struct udma_desc *d = uc->desc; - - uc->ud->decrement_byte_counters(uc, d->residue); - uc->ud->start(uc); - vchan_cookie_complete(&d->vd); - break; - } - - break; - } - - spin_unlock_irqrestore(&uc->vc.lock, flags); -} - static irqreturn_t udma_ring_irq_handler(int irq, void *data) { struct udma_chan *uc = data; @@ -2097,38 +1897,6 @@ static int pktdma_alloc_chan_resources(struct dma_chan *chan) return ret; } -static int udma_slave_config(struct dma_chan *chan, - struct dma_slave_config *cfg) -{ - struct udma_chan *uc = to_udma_chan(chan); - - memcpy(&uc->cfg, cfg, sizeof(uc->cfg)); - - return 0; -} - -static void udma_issue_pending(struct dma_chan *chan) -{ - struct udma_chan *uc = to_udma_chan(chan); - unsigned long flags; - - spin_lock_irqsave(&uc->vc.lock, flags); - - /* If we have something pending and no active descriptor, then */ - if (vchan_issue_pending(&uc->vc) && !uc->desc) { - /* - * start a descriptor if the channel is NOT [marked as - * terminating _and_ it is still running (teardown has not - * completed yet)]. - */ - if (!(uc->state == UDMA_CHAN_IS_TERMINATING && - udma_is_chan_running(uc))) - uc->ud->start(uc); - } - - spin_unlock_irqrestore(&uc->vc.lock, flags); -} - static enum dma_status udma_tx_status(struct dma_chan *chan, dma_cookie_t cookie, struct dma_tx_state *txstate) @@ -2256,98 +2024,6 @@ static int udma_resume(struct dma_chan *chan) return 0; } -static int udma_terminate_all(struct dma_chan *chan) -{ - struct udma_chan *uc = to_udma_chan(chan); - unsigned long flags; - LIST_HEAD(head); - - spin_lock_irqsave(&uc->vc.lock, flags); - - if (udma_is_chan_running(uc)) - uc->ud->stop(uc); - - if (uc->desc) { - uc->terminated_desc = uc->desc; - uc->desc = NULL; - uc->terminated_desc->terminated = true; - cancel_delayed_work(&uc->tx_drain.work); - } - - uc->paused = false; - - vchan_get_all_descriptors(&uc->vc, &head); - spin_unlock_irqrestore(&uc->vc.lock, flags); - vchan_dma_desc_free_list(&uc->vc, &head); - - return 0; -} - -static void udma_synchronize(struct dma_chan *chan) -{ - struct udma_chan *uc = to_udma_chan(chan); - unsigned long timeout = msecs_to_jiffies(1000); - - vchan_synchronize(&uc->vc); - - if (uc->state == UDMA_CHAN_IS_TERMINATING) { - timeout = wait_for_completion_timeout(&uc->teardown_completed, - timeout); - if (!timeout) { - dev_warn(uc->ud->dev, "chan%d teardown timeout!\n", - uc->id); - udma_dump_chan_stdata(uc); - uc->ud->reset_chan(uc, true); - } - } - - uc->ud->reset_chan(uc, false); - if (udma_is_chan_running(uc)) - dev_warn(uc->ud->dev, "chan%d refused to stop!\n", uc->id); - - cancel_delayed_work_sync(&uc->tx_drain.work); - udma_reset_rings(uc); -} - -/* - * This tasklet handles the completion of a DMA descriptor by - * calling its callback and freeing it. - */ -static void udma_vchan_complete(struct tasklet_struct *t) -{ - struct virt_dma_chan *vc = from_tasklet(vc, t, task); - struct virt_dma_desc *vd, *_vd; - struct dmaengine_desc_callback cb; - LIST_HEAD(head); - - spin_lock_irq(&vc->lock); - list_splice_tail_init(&vc->desc_completed, &head); - vd = vc->cyclic; - if (vd) { - vc->cyclic = NULL; - dmaengine_desc_get_callback(&vd->tx, &cb); - } else { - memset(&cb, 0, sizeof(cb)); - } - spin_unlock_irq(&vc->lock); - - udma_desc_pre_callback(vc, vd, NULL); - dmaengine_desc_callback_invoke(&cb, NULL); - - list_for_each_entry_safe(vd, _vd, &head, node) { - struct dmaengine_result result; - - dmaengine_desc_get_callback(&vd->tx, &cb); - - list_del(&vd->node); - - udma_desc_pre_callback(vc, vd, &result); - dmaengine_desc_callback_invoke(&cb, &result); - - vchan_vdesc_fini(vd); - } -} - static void udma_free_chan_resources(struct dma_chan *chan) { struct udma_chan *uc = to_udma_chan(chan); @@ -2822,17 +2498,6 @@ static int udma_get_mmrs(struct platform_device *pdev, struct udma_dev *ud) return 0; } -static void udma_mark_resource_ranges(struct udma_dev *ud, unsigned long *map, - struct ti_sci_resource_desc *rm_desc, - char *name) -{ - bitmap_clear(map, rm_desc->start, rm_desc->num); - bitmap_clear(map, rm_desc->start_sec, rm_desc->num_sec); - dev_dbg(ud->dev, "ti_sci resource range for %s: %d:%d | %d:%d\n", name, - rm_desc->start, rm_desc->num, rm_desc->start_sec, - rm_desc->num_sec); -} - static const char * const range_names[] = { [RM_RANGE_BCHAN] = "ti,sci-rm-range-bchan", [RM_RANGE_TCHAN] = "ti,sci-rm-range-tchan", @@ -3463,202 +3128,6 @@ static int setup_resources(struct udma_dev *ud) return ch_count; } -static int udma_setup_rx_flush(struct udma_dev *ud) -{ - struct udma_rx_flush *rx_flush = &ud->rx_flush; - struct cppi5_desc_hdr_t *tr_desc; - struct cppi5_tr_type1_t *tr_req; - struct cppi5_host_desc_t *desc; - struct device *dev = ud->dev; - struct udma_hwdesc *hwdesc; - size_t tr_size; - - /* Allocate 1K buffer for discarded data on RX channel teardown */ - rx_flush->buffer_size = SZ_1K; - rx_flush->buffer_vaddr = devm_kzalloc(dev, rx_flush->buffer_size, - GFP_KERNEL); - if (!rx_flush->buffer_vaddr) - return -ENOMEM; - - rx_flush->buffer_paddr = dma_map_single(dev, rx_flush->buffer_vaddr, - rx_flush->buffer_size, - DMA_TO_DEVICE); - if (dma_mapping_error(dev, rx_flush->buffer_paddr)) - return -ENOMEM; - - /* Set up descriptor to be used for TR mode */ - hwdesc = &rx_flush->hwdescs[0]; - tr_size = sizeof(struct cppi5_tr_type1_t); - hwdesc->cppi5_desc_size = cppi5_trdesc_calc_size(tr_size, 1); - hwdesc->cppi5_desc_size = ALIGN(hwdesc->cppi5_desc_size, - ud->desc_align); - - hwdesc->cppi5_desc_vaddr = devm_kzalloc(dev, hwdesc->cppi5_desc_size, - GFP_KERNEL); - if (!hwdesc->cppi5_desc_vaddr) - return -ENOMEM; - - hwdesc->cppi5_desc_paddr = dma_map_single(dev, hwdesc->cppi5_desc_vaddr, - hwdesc->cppi5_desc_size, - DMA_TO_DEVICE); - if (dma_mapping_error(dev, hwdesc->cppi5_desc_paddr)) - return -ENOMEM; - - /* Start of the TR req records */ - hwdesc->tr_req_base = hwdesc->cppi5_desc_vaddr + tr_size; - /* Start address of the TR response array */ - hwdesc->tr_resp_base = hwdesc->tr_req_base + tr_size; - - tr_desc = hwdesc->cppi5_desc_vaddr; - cppi5_trdesc_init(tr_desc, 1, tr_size, 0, 0); - cppi5_desc_set_pktids(tr_desc, 0, CPPI5_INFO1_DESC_FLOWID_DEFAULT); - cppi5_desc_set_retpolicy(tr_desc, 0, 0); - - tr_req = hwdesc->tr_req_base; - cppi5_tr_init(&tr_req->flags, CPPI5_TR_TYPE1, false, false, - CPPI5_TR_EVENT_SIZE_COMPLETION, 0); - cppi5_tr_csf_set(&tr_req->flags, CPPI5_TR_CSF_SUPR_EVT); - - tr_req->addr = rx_flush->buffer_paddr; - tr_req->icnt0 = rx_flush->buffer_size; - tr_req->icnt1 = 1; - - dma_sync_single_for_device(dev, hwdesc->cppi5_desc_paddr, - hwdesc->cppi5_desc_size, DMA_TO_DEVICE); - - /* Set up descriptor to be used for packet mode */ - hwdesc = &rx_flush->hwdescs[1]; - hwdesc->cppi5_desc_size = ALIGN(sizeof(struct cppi5_host_desc_t) + - CPPI5_INFO0_HDESC_EPIB_SIZE + - CPPI5_INFO0_HDESC_PSDATA_MAX_SIZE, - ud->desc_align); - - hwdesc->cppi5_desc_vaddr = devm_kzalloc(dev, hwdesc->cppi5_desc_size, - GFP_KERNEL); - if (!hwdesc->cppi5_desc_vaddr) - return -ENOMEM; - - hwdesc->cppi5_desc_paddr = dma_map_single(dev, hwdesc->cppi5_desc_vaddr, - hwdesc->cppi5_desc_size, - DMA_TO_DEVICE); - if (dma_mapping_error(dev, hwdesc->cppi5_desc_paddr)) - return -ENOMEM; - - desc = hwdesc->cppi5_desc_vaddr; - cppi5_hdesc_init(desc, 0, 0); - cppi5_desc_set_pktids(&desc->hdr, 0, CPPI5_INFO1_DESC_FLOWID_DEFAULT); - cppi5_desc_set_retpolicy(&desc->hdr, 0, 0); - - cppi5_hdesc_attach_buf(desc, - rx_flush->buffer_paddr, rx_flush->buffer_size, - rx_flush->buffer_paddr, rx_flush->buffer_size); - - dma_sync_single_for_device(dev, hwdesc->cppi5_desc_paddr, - hwdesc->cppi5_desc_size, DMA_TO_DEVICE); - return 0; -} - -#ifdef CONFIG_DEBUG_FS -static void udma_dbg_summary_show_chan(struct seq_file *s, - struct dma_chan *chan) -{ - struct udma_chan *uc = to_udma_chan(chan); - struct udma_chan_config *ucc = &uc->config; - - seq_printf(s, " %-13s| %s", dma_chan_name(chan), - chan->dbg_client_name ?: "in-use"); - if (ucc->tr_trigger_type) - seq_puts(s, " (triggered, "); - else - seq_printf(s, " (%s, ", - dmaengine_get_direction_text(uc->config.dir)); - - switch (uc->config.dir) { - case DMA_MEM_TO_MEM: - if (uc->ud->match_data->type == DMA_TYPE_BCDMA) { - seq_printf(s, "bchan%d)\n", uc->bchan->id); - return; - } - - seq_printf(s, "chan%d pair [0x%04x -> 0x%04x], ", uc->tchan->id, - ucc->src_thread, ucc->dst_thread); - break; - case DMA_DEV_TO_MEM: - seq_printf(s, "rchan%d [0x%04x -> 0x%04x], ", uc->rchan->id, - ucc->src_thread, ucc->dst_thread); - if (uc->ud->match_data->type == DMA_TYPE_PKTDMA) - seq_printf(s, "rflow%d, ", uc->rflow->id); - break; - case DMA_MEM_TO_DEV: - seq_printf(s, "tchan%d [0x%04x -> 0x%04x], ", uc->tchan->id, - ucc->src_thread, ucc->dst_thread); - if (uc->ud->match_data->type == DMA_TYPE_PKTDMA) - seq_printf(s, "tflow%d, ", uc->tchan->tflow_id); - break; - default: - seq_printf(s, ")\n"); - return; - } - - if (ucc->ep_type == PSIL_EP_NATIVE) { - seq_printf(s, "PSI-L Native"); - if (ucc->metadata_size) { - seq_printf(s, "[%s", ucc->needs_epib ? " EPIB" : ""); - if (ucc->psd_size) - seq_printf(s, " PSDsize:%u", ucc->psd_size); - seq_printf(s, " ]"); - } - } else { - seq_printf(s, "PDMA"); - if (ucc->enable_acc32 || ucc->enable_burst) - seq_printf(s, "[%s%s ]", - ucc->enable_acc32 ? " ACC32" : "", - ucc->enable_burst ? " BURST" : ""); - } - - seq_printf(s, ", %s)\n", ucc->pkt_mode ? "Packet mode" : "TR mode"); -} - -static void udma_dbg_summary_show(struct seq_file *s, - struct dma_device *dma_dev) -{ - struct dma_chan *chan; - - list_for_each_entry(chan, &dma_dev->channels, device_node) { - if (chan->client_count) - udma_dbg_summary_show_chan(s, chan); - } -} -#endif /* CONFIG_DEBUG_FS */ - -static enum dmaengine_alignment udma_get_copy_align(struct udma_dev *ud) -{ - const struct udma_match_data *match_data = ud->match_data; - u8 tpl; - - if (!match_data->enable_memcpy_support) - return DMAENGINE_ALIGN_8_BYTES; - - /* Get the highest TPL level the device supports for memcpy */ - if (ud->bchan_cnt) - tpl = udma_get_chan_tpl_index(&ud->bchan_tpl, 0); - else if (ud->tchan_cnt) - tpl = udma_get_chan_tpl_index(&ud->tchan_tpl, 0); - else - return DMAENGINE_ALIGN_8_BYTES; - - switch (match_data->burst_size[tpl]) { - case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_256_BYTES: - return DMAENGINE_ALIGN_256_BYTES; - case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_128_BYTES: - return DMAENGINE_ALIGN_128_BYTES; - case TI_SCI_RM_UDMAP_CHAN_BURST_SIZE_64_BYTES: - fallthrough; - default: - return DMAENGINE_ALIGN_64_BYTES; - } -} - static int udma_probe(struct platform_device *pdev) { struct device_node *navss_node = pdev->dev.parent->of_node; diff --git a/drivers/dma/ti/k3-udma.h b/drivers/dma/ti/k3-udma.h index 16de0b218d8ad..4878d13d84ed0 100644 --- a/drivers/dma/ti/k3-udma.h +++ b/drivers/dma/ti/k3-udma.h @@ -629,6 +629,34 @@ void udma_reset_rings(struct udma_chan *uc); int navss_psil_pair(struct udma_dev *ud, u32 src_thread, u32 dst_thread); int navss_psil_unpair(struct udma_dev *ud, u32 src_thread, u32 dst_thread); +void udma_start_desc(struct udma_chan *uc); +u8 udma_get_chan_tpl_index(struct udma_tpl *tpl_map, int chan_id); +void k3_configure_chan_coherency(struct dma_chan *chan, u32 asel); +void udma_reset_uchan(struct udma_chan *uc); +void udma_dump_chan_stdata(struct udma_chan *uc); +bool udma_is_chan_running(struct udma_chan *uc); + +bool udma_chan_needs_reconfiguration(struct udma_chan *uc); +void udma_cyclic_packet_elapsed(struct udma_chan *uc); +void udma_check_tx_completion(struct work_struct *work); +int udma_slave_config(struct dma_chan *chan, + struct dma_slave_config *cfg); +void udma_issue_pending(struct dma_chan *chan); +int udma_terminate_all(struct dma_chan *chan); +void udma_synchronize(struct dma_chan *chan); +void udma_vchan_complete(struct tasklet_struct *t); +void udma_mark_resource_ranges(struct udma_dev *ud, unsigned long *map, + struct ti_sci_resource_desc *rm_desc, + char *name); +int udma_setup_rx_flush(struct udma_dev *ud); +enum dmaengine_alignment udma_get_copy_align(struct udma_dev *ud); + +#ifdef CONFIG_DEBUG_FS +void udma_dbg_summary_show_chan(struct seq_file *s, + struct dma_chan *chan); +void udma_dbg_summary_show(struct seq_file *s, + struct dma_device *dma_dev); +#endif /* CONFIG_DEBUG_FS */ /* Direct access to UDMA low lever resources for the glue layer */ int xudma_navss_psil_pair(struct udma_dev *ud, u32 src_thread, u32 dst_thread); -- 2.54.0