From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-lf1-f44.google.com (mail-lf1-f44.google.com [209.85.167.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id A06FD19F495 for ; Wed, 20 Nov 2024 11:42:06 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.167.44 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732102928; cv=none; b=pmazBzqffDW0IFYneKDDtU/4GAghmol6SX7eDCtUH6d7ySlxsG4JTaYWXtIc/VkOXMF9BVG7VqNL85wR0U+Ib8BbYtKwtpxOhCVHkR7o9HZb3O6xwKjvQ+afOuwZExzEL58Ldcx1aTCXC5+czqS9R149ctyHCDSwgqWo6z2LpBY= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1732102928; c=relaxed/simple; bh=uAZHXeWUvxu5L1xWBaFF/xqeVuYVrREZHcSXPJSLYJM=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=onLcu5tqAWc+cCIfh4p4HdY2Q2vaI13Afafss2ioJP850dhpq2a6jv4w3e8sKFGMXwWo4cnpz8jr5FWH31Wt4tDbMJG8J6DGLHzQMd13ng3qneVc7V85xuUhEeCtDxBA7iQopPR52ywoKFiEonnWmYNAElkzZJ6tyiPsFMCJh8E= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org; spf=pass smtp.mailfrom=linaro.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b=NZ4g6eDh; arc=none smtp.client-ip=209.85.167.44 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linaro.org Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linaro.org Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=linaro.org header.i=@linaro.org header.b="NZ4g6eDh" Received: by mail-lf1-f44.google.com with SMTP id 2adb3069b0e04-539f84907caso2662912e87.3 for ; Wed, 20 Nov 2024 03:42:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; t=1732102925; x=1732707725; darn=vger.kernel.org; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:from:to:cc:subject:date:message-id:reply-to; bh=J85HbDSvNnxr9Fvax8uh24ZXRAm6smu4V5FrFzPRi0A=; b=NZ4g6eDhJxNZw9RLK/+lhLkIbBilH/cqtLoq9feRBLruJugxhQYY2KzmlCu7jmHmUm A4/Zs6w4Eq/Twkr5eiZqhn9k3Ld0CeQIJBLXhXsLFkqJEReAMyCj4eqt0e17JjhRZxDW kcINWz3pBUzPXu9LsYxkda3/0JgSsevqeRr6lKGs57HUXFOmCh9jftdp/hqZhoT/trO8 C+cGO2/JFEy2wt+Wry8elFHegYs2r/2xxrEcI6Fm4eBlAItETXw34dQoVE+Z4nP9Se6s Jl/unSvcDaFMxKZae9FDnr/eb3Q5BmeuOlWU4aGVaXznqTzjqq05NOA7Htpw/YDEh/Tf j/8w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1732102925; x=1732707725; h=in-reply-to:content-disposition:mime-version:references:message-id :subject:cc:to:from:date:x-gm-message-state:from:to:cc:subject:date :message-id:reply-to; bh=J85HbDSvNnxr9Fvax8uh24ZXRAm6smu4V5FrFzPRi0A=; b=aGRPU+BuH0U2bq8F2OgoxbHrWVsVn9/cICSh/tmUiFrrEFj0j7t4G1t8a8YgyXWNk4 B28PKw3GXuCIkZ7RTiD8lYPkPZ/cTJYlLBMg+Mmadx7e34Sj/FeBHqx5SyrhLzSSQcoi OkKaoPFAqrm17htMO79jBrVjzmPJpbyIDMx9uB5NkHT6m4fJu2hQoqYbRak4QiiMmPBR a4LPl6mcN7+tkejAyuZL3hdk4yGX0Gfm1fQrrRL3ODnCxQDOBQCzMVM67PFoUjR9YOUD wJrh9vHEe/z83ENUwVMmlwyIo5l1yB9uurZfYZaVWe5cgmNKZ42S8yDsCkU90wFZW/oV k6yQ== X-Forwarded-Encrypted: i=1; AJvYcCWZ67vEIcNcmjSVY88na9fF9kwuk5khd1d8tKogpZJpDKzM+A3wg6KdcLhCqNOIpUBrOmLPtJa8cQr/N18=@vger.kernel.org X-Gm-Message-State: AOJu0YzRHHb4XFy0P40gkcLVs4h5xfM45257LA1+pgYfCoUtmtCKRJat RjND6mB35p1gqcqJqCVZaD7VZWAkiorNwlGUcRxKJqXe5ymlsD1KyiFDv0+SLB0= X-Gm-Gg: ASbGncvnz/5UVx3VNce9dKGOIwUuv++6OTRjWvvOKtgbSmEacAUAWbxXcqIMjMSYWmT FZOueU6IhqRji2NtlYdCawu+rfp9yF7ifli5S5OLP6UhrBsRe5C5JlerUoHi9ZIrlxpqCYIxEsQ BmVa5+j2qvMi7cd12vfsy6T9PxN1XoFqW0Df2l99SZ36WHCeuD56jPlDxbRCpTpYyBqKul67R3n Y0qhbfEZRR7pAAEFtcIh4q0+hX4z0yH4B0OnMVfFd9joOkei7GKobnl7FKCEzpTjNIoV4rS8Vxv 0vLjMNSyTjN3d4MyoJZO3m3SirOb6g== X-Google-Smtp-Source: AGHT+IFkFODBIkQm7rhS+vBVQwoV7NHSlKbbb/SOrg2jNIR8W7E3SUhD1vgmyHOq/ELKF2MNxNfxpw== X-Received: by 2002:a05:6512:3d22:b0:53d:a68f:d901 with SMTP id 2adb3069b0e04-53dc1333415mr1130478e87.15.1732102924653; Wed, 20 Nov 2024 03:42:04 -0800 (PST) Received: from eriador.lumag.spb.ru (2001-14ba-a0c3-3a00--b8c.rev.dnainternet.fi. [2001:14ba:a0c3:3a00::b8c]) by smtp.gmail.com with ESMTPSA id 2adb3069b0e04-53dbd477f8dsm606183e87.241.2024.11.20.03.42.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 20 Nov 2024 03:42:04 -0800 (PST) Date: Wed, 20 Nov 2024 13:42:01 +0200 From: Dmitry Baryshkov To: Neil Armstrong Cc: Akhil P Oommen , Viresh Kumar , Nishanth Menon , Stephen Boyd , "Rafael J. Wysocki" , Rob Clark , Sean Paul , Konrad Dybcio , Abhinav Kumar , Marijn Suijten , David Airlie , Simona Vetter , Bjorn Andersson , Rob Herring , Krzysztof Kozlowski , Conor Dooley , Connor Abbott , linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-msm@vger.kernel.org, dri-devel@lists.freedesktop.org, freedreno@lists.freedesktop.org, devicetree@vger.kernel.org Subject: Re: [PATCH v2 05/11] drm/msm: adreno: add plumbing to generate bandwidth vote table for GMU Message-ID: References: <20241119-topic-sm8x50-gpu-bw-vote-v2-0-4deb87be2498@linaro.org> <20241119-topic-sm8x50-gpu-bw-vote-v2-5-4deb87be2498@linaro.org> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20241119-topic-sm8x50-gpu-bw-vote-v2-5-4deb87be2498@linaro.org> On Tue, Nov 19, 2024 at 06:56:40PM +0100, Neil Armstrong wrote: > The Adreno GMU Management Unit (GMU) can also scale DDR Bandwidth along > the Frequency and Power Domain level, but by default we leave the > OPP core scale the interconnect ddr path. > > In order to calculate vote values used by the GPU Management > Unit (GMU), we need to parse all the possible OPP Bandwidths and > create a vote value to be sent to the appropriate Bus Control > Modules (BCMs) declared in the GPU info struct. > > The vote array will then be used to dynamically generate the GMU > bw_table sent during the GMU power-up. > > Signed-off-by: Neil Armstrong LGTM, two minor nits below. > --- > drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 153 ++++++++++++++++++++++++++++++++++ > drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 14 ++++ > drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 1 + > 3 files changed, 168 insertions(+) > > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > index 14db7376c712d19446b38152e480bd5a1e0a5198..f6814d92a4edb29ba8a34a34aabb8b2324e9c6a4 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c > @@ -9,6 +9,7 @@ > #include > #include > #include > +#include > #include > > #include "a6xx_gpu.h" > @@ -1287,6 +1288,109 @@ static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu) > return 0; > } > > +/** > + * struct bcm_db - Auxiliary data pertaining to each Bus Clock Manager (BCM) > + * @unit: divisor used to convert bytes/sec bw value to an RPMh msg > + * @width: multiplier used to convert bytes/sec bw value to an RPMh msg > + * @vcd: virtual clock domain that this bcm belongs to > + * @reserved: reserved field > + */ > +struct bcm_db { > + __le32 unit; > + __le16 width; > + u8 vcd; > + u8 reserved; > +}; > + > +static u64 bcm_div(u64 num, u32 base) > +{ > + /* Ensure that small votes aren't lost. */ > + if (num && num < base) > + return 1; > + > + do_div(num, base); > + > + return num; > +} > + > +static int a6xx_gmu_rpmh_bw_votes_init(const struct a6xx_info *info, > + struct a6xx_gmu *gmu) > +{ > + const struct bcm_db *bcm_data[GMU_MAX_BCMS] = { 0 }; > + unsigned int bcm_index, bw_index; > + > + /* Retrieve BCM data from cmd-db */ > + for (bcm_index = 0; bcm_index < GMU_MAX_BCMS; bcm_index++) { > + size_t count; > + > + /* Skip unconfigured BCM */ > + if (!info->bcm[bcm_index].name) > + continue; > + > + bcm_data[bcm_index] = cmd_db_read_aux_data( > + info->bcm[bcm_index].name, > + &count); > + if (IS_ERR(bcm_data[bcm_index])) > + return PTR_ERR(bcm_data[bcm_index]); > + > + if (!count) > + return -EINVAL; > + } > + > + /* Generate BCM votes values for each bandwidth & BCM */ > + for (bw_index = 0; bw_index < gmu->nr_gpu_bws; bw_index++) { > + u32 *data = gmu->gpu_bw_votes[bw_index]; > + u32 bw = gmu->gpu_bw_table[bw_index]; > + > + /* Calculations loosely copied from bcm_aggregate() & tcs_cmd_gen() */ > + for (bcm_index = 0; bcm_index < GMU_MAX_BCMS; bcm_index++) { > + bool commit = false; > + u64 peak, vote; > + u16 width; > + u32 unit; > + > + /* Skip unconfigured BCM */ > + if (!info->bcm[bcm_index].name || !bcm_data[bcm_index]) > + continue; Nit: you don't care about the .name anymore, the first check can be dropped. > + > + if (bcm_index == GMU_MAX_BCMS - 1 || > + (bcm_data[bcm_index + 1] && > + bcm_data[bcm_index]->vcd != bcm_data[bcm_index + 1]->vcd)) > + commit = true; > + > + if (!bw) { > + data[bcm_index] = BCM_TCS_CMD(commit, false, 0, 0); > + continue; > + } > + > + if (info->bcm[bcm_index].fixed) { > + u32 perfmode = 0; > + > + if (bw >= info->bcm[bcm_index].perfmode_bw) > + perfmode = info->bcm[bcm_index].perfmode; > + > + data[bcm_index] = BCM_TCS_CMD(commit, true, 0, perfmode); > + continue; > + } > + > + /* Multiply the bandwidth by the width of the connection */ > + width = le16_to_cpu(bcm_data[bcm_index]->width); > + peak = bcm_div((u64)bw * width, info->bcm[bcm_index].buswidth); > + > + /* Input bandwidth value is in KBps, scale the value to BCM unit */ > + unit = le32_to_cpu(bcm_data[bcm_index]->unit); > + vote = bcm_div(peak * 1000ULL, unit); > + > + if (vote > BCM_TCS_CMD_VOTE_MASK) > + vote = BCM_TCS_CMD_VOTE_MASK; > + > + data[bcm_index] = BCM_TCS_CMD(commit, true, vote, vote); > + } > + } > + > + return 0; > +} > + > /* Return the 'arc-level' for the given frequency */ > static unsigned int a6xx_gmu_get_arc_level(struct device *dev, > unsigned long freq) > @@ -1390,12 +1494,15 @@ static int a6xx_gmu_rpmh_arc_votes_init(struct device *dev, u32 *votes, > * The GMU votes with the RPMh for itself and on behalf of the GPU but we need > * to construct the list of votes on the CPU and send it over. Query the RPMh > * voltage levels and build the votes > + * The GMU can also vote for DDR interconnects, use the OPP bandwidth entries > + * and BCM parameters to build the votes. > */ > > static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) > { > struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); > struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; > + const struct a6xx_info *info = adreno_gpu->info->a6xx; > struct msm_gpu *gpu = &adreno_gpu->base; > int ret; > > @@ -1407,6 +1514,10 @@ static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) > ret |= a6xx_gmu_rpmh_arc_votes_init(gmu->dev, gmu->cx_arc_votes, > gmu->gmu_freqs, gmu->nr_gmu_freqs, "cx.lvl"); > > + /* Build the interconnect votes */ > + if (adreno_gpu->info->features & ADRENO_FEAT_GMU_BW_VOTE) > + ret |= a6xx_gmu_rpmh_bw_votes_init(info, gmu); > + > return ret; > } > > @@ -1442,6 +1553,38 @@ static int a6xx_gmu_build_freq_table(struct device *dev, unsigned long *freqs, > return index; > } > > +static int a6xx_gmu_build_bw_table(struct device *dev, unsigned long *bandwidths, > + u32 size) > +{ > + int count = dev_pm_opp_get_opp_count(dev); > + struct dev_pm_opp *opp; > + int i, index = 0; > + unsigned int bandwidth = 1; > + > + /* > + * The OPP table doesn't contain the "off" bandwidth level so we need to > + * add 1 to the table size to account for it > + */ > + > + if (WARN(count + 1 > size, > + "The GMU bandwidth table is being truncated\n")) > + count = size - 1; > + > + /* Set the "off" bandwidth */ > + bandwidths[index++] = 0; > + > + for (i = 0; i < count; i++) { > + opp = dev_pm_opp_find_bw_ceil(dev, &bandwidth, 0); > + if (IS_ERR(opp)) > + break; > + > + dev_pm_opp_put(opp); > + bandwidths[index++] = bandwidth++; > + } > + > + return index; > +} > + > static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) > { > struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); > @@ -1472,6 +1615,16 @@ static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) > > gmu->current_perf_index = gmu->nr_gpu_freqs - 1; > > + /* > + * The GMU also handles GPU Interconnect Votes so build a list > + * of DDR bandwidths from the GPU OPP table > + */ > + if (adreno_gpu->info->features & ADRENO_FEAT_GMU_BW_VOTE) > + gmu->nr_gpu_bws = a6xx_gmu_build_bw_table(&gpu->pdev->dev, > + gmu->gpu_bw_table, ARRAY_SIZE(gmu->gpu_bw_table)); > + > + gmu->current_perf_index = gmu->nr_gpu_freqs - 1; > + > /* Build the list of RPMh votes that we'll send to the GMU */ > return a6xx_gmu_rpmh_votes_init(gmu); > } > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > index b4a79f88ccf45cfe651c86d2a9da39541c5772b3..03603eadc0f9ed866899c95e99f333a511ebc3c1 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h > @@ -19,6 +19,16 @@ struct a6xx_gmu_bo { > u64 iova; > }; > > +#define GMU_MAX_BCMS 3 > + > +struct a6xx_bcm { > + char *name; > + unsigned int buswidth; > + bool fixed; > + unsigned int perfmode; > + unsigned int perfmode_bw; > +}; > + > /* > * These define the different GMU wake up options - these define how both the > * CPU and the GMU bring up the hardware > @@ -82,6 +92,10 @@ struct a6xx_gmu { > unsigned long gpu_freqs[16]; > u32 gx_arc_votes[16]; > > + int nr_gpu_bws; > + unsigned long gpu_bw_table[16]; > + u32 gpu_bw_votes[16][GMU_MAX_BCMS]; We still have magic 16 here. GPU_MAX_FREQUENCIES? GPU_FREQ_TABLE_SIZE? > + > int nr_gmu_freqs; > unsigned long gmu_freqs[4]; > u32 cx_arc_votes[4]; > diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > index 4aceffb6aae89c781facc2a6e4a82b20b341b6cb..5b80919e595fa1ba0a3afcca55feb89e60870cb1 100644 > --- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h > @@ -44,6 +44,7 @@ struct a6xx_info { > u32 gmu_chipid; > u32 gmu_cgc_mode; > u32 prim_fifo_threshold; > + const struct a6xx_bcm bcm[GMU_MAX_BCMS]; > }; > > struct a6xx_gpu { > > -- > 2.34.1 > -- With best wishes Dmitry