* [PATCH 0/5] drm/msm: Add support for Adreno a6xx
@ 2018-06-11 18:25 Jordan Crouse
[not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Jordan Crouse @ 2018-06-11 18:25 UTC (permalink / raw)
To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW,
freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA
This is an initial version of support for the Adreno a6xx GPU family starting
with the a630 from the sdm845 SoC. This code is ahead of much of the sdm845
code that would be needed to actually bring up a device and it is also in
advance of any user side support for the a6xx GPU so this is mainly just a
chance to look over the code structure and get a feel for the
direction that the hardware is going in.
The a6xx GPU is an iteration of a5xx so most of the GPU side code looks pretty
identical except for the usual register differences. The big different is in
power control. On the a5xx there was a rudimentary device called the GPMU that
did some basic power stuff but left most of the complexity to the kernel.
On the a6xx the power complexity is being moved to a component
called the GMU (graphics management unit) which handles the power control of the
GPU and shuts it down quickly after it goes idle.
This stack has a number of dependencies that are in various stages of being
merged:
https://patchwork.kernel.org/patch/10286375/ - dev_pm_opp_get_of_node()
https://patchwork.kernel.org/patch/10333575/ - command DB
https://patchwork.kernel.org/patch/10301163/ - pm_runtime ops for arm-smmu
[v5 - Move firmware loading to adreno_load_gpu() so it is easier to fail if the
firmware isn't found ]
[v4 - Add clock parsing helper function and use it for GPU and GMU. Update
recommended clock gating settings. Fix a bug in the CMD DB parser. Update
register values from updated database ]
[v3 - fix inverted register definition for GMU_SPTPRAC_CLK_STATUS; fix incorrect
register check in a5xx_gmu_gx_is_on(), use dev_pm_opp_get_of_node() from
Rajendra and Viresh to read the qcom,level from the device tree; read qcom,level
from the DT to get the voltage level to pass to the GMU, fix issues
identified by smatch]
[v2 - addressed comments from Lucas Stach; added pm_runtime_get_supplier calls
for accesses to the GMU IOMMU; moved to SPDX headers for new files]
Jordan Crouse (4):
drm/msm: Remove pm_runtime operations from msm_iommu
drm/msm: Add a helper function to parse clock names
drm/msm/adreno: Load the firmware before bringing up the hardware
drm/msm: Add A6XX device support
Sharat Masetty (1):
drm/msm: Add generated headers for A6XX
drivers/gpu/drm/msm/Makefile | 3 +
drivers/gpu/drm/msm/adreno/a6xx.xml.h | 1784 ++++++++++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1172 +++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 ++
drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h | 382 +++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 807 +++++++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 +
drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 +++++
drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 ++
drivers/gpu/drm/msm/adreno/adreno_device.c | 35 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.h | 4 +
drivers/gpu/drm/msm/msm_drv.c | 57 +
drivers/gpu/drm/msm/msm_drv.h | 4 +
drivers/gpu/drm/msm/msm_gpu.c | 62 +-
drivers/gpu/drm/msm/msm_gpu.h | 2 +-
drivers/gpu/drm/msm/msm_iommu.c | 8 -
17 files changed, 5044 insertions(+), 62 deletions(-)
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h
--
2.17.1
_______________________________________________
Freedreno mailing list
Freedreno@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/freedreno
^ permalink raw reply [flat|nested] 12+ messages in thread[parent not found: <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>]
* [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu [not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> @ 2018-06-11 18:26 ` Jordan Crouse [not found] ` <20180611182604.30467-2-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2018-06-11 18:26 ` [PATCH 2/5] drm/msm: Add a helper function to parse clock names Jordan Crouse ` (3 subsequent siblings) 4 siblings, 1 reply; 12+ messages in thread From: Jordan Crouse @ 2018-06-11 18:26 UTC (permalink / raw) To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA Now that the IOMMU is the master of it's own power we don't need to bring up the GPU to do IOMMU operations. This is good because bringing up a6xx requires the GMU so calling pm_runtime_get_sync() too early in the process gets us into some nasty circular dependency situations. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/msm_iommu.c | 8 -------- 1 file changed, 8 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c index b23d33622f37..ccd93ac6a4d8 100644 --- a/drivers/gpu/drm/msm/msm_iommu.c +++ b/drivers/gpu/drm/msm/msm_iommu.c @@ -40,9 +40,7 @@ static int msm_iommu_attach(struct msm_mmu *mmu, const char * const *names, struct msm_iommu *iommu = to_msm_iommu(mmu); int ret; - pm_runtime_get_sync(mmu->dev); ret = iommu_attach_device(iommu->domain, mmu->dev); - pm_runtime_put_sync(mmu->dev); return ret; } @@ -52,9 +50,7 @@ static void msm_iommu_detach(struct msm_mmu *mmu, const char * const *names, { struct msm_iommu *iommu = to_msm_iommu(mmu); - pm_runtime_get_sync(mmu->dev); iommu_detach_device(iommu->domain, mmu->dev); - pm_runtime_put_sync(mmu->dev); } static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, @@ -63,9 +59,7 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, struct msm_iommu *iommu = to_msm_iommu(mmu); size_t ret; -// pm_runtime_get_sync(mmu->dev); ret = iommu_map_sg(iommu->domain, iova, sgt->sgl, sgt->nents, prot); -// pm_runtime_put_sync(mmu->dev); WARN_ON(ret < 0); return (ret == len) ? 0 : -EINVAL; @@ -76,9 +70,7 @@ static int msm_iommu_unmap(struct msm_mmu *mmu, uint64_t iova, { struct msm_iommu *iommu = to_msm_iommu(mmu); - pm_runtime_get_sync(mmu->dev); iommu_unmap(iommu->domain, iova, len); - pm_runtime_put_sync(mmu->dev); return 0; } -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
[parent not found: <20180611182604.30467-2-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>]
* Re: [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu [not found] ` <20180611182604.30467-2-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> @ 2018-06-12 16:55 ` Kristian Høgsberg 2018-06-14 7:00 ` Vivek Gautam 1 sibling, 0 replies; 12+ messages in thread From: Kristian Høgsberg @ 2018-06-12 16:55 UTC (permalink / raw) To: jcrouse-sgV2jX0FEOL9JmXXK+q4OQ Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, dri-devel On Mon, Jun 11, 2018 at 11:26 AM Jordan Crouse <jcrouse@codeaurora.org> wrote: > > Now that the IOMMU is the master of it's own power we don't need to bring > up the GPU to do IOMMU operations. This is good because bringing up a6xx > requires the GMU so calling pm_runtime_get_sync() too early in the process > gets us into some nasty circular dependency situations. > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> Reviewed-by: Kristian H. Kristensen <hoegsberg@google.com> > --- > drivers/gpu/drm/msm/msm_iommu.c | 8 -------- > 1 file changed, 8 deletions(-) > > diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c > index b23d33622f37..ccd93ac6a4d8 100644 > --- a/drivers/gpu/drm/msm/msm_iommu.c > +++ b/drivers/gpu/drm/msm/msm_iommu.c > @@ -40,9 +40,7 @@ static int msm_iommu_attach(struct msm_mmu *mmu, const char * const *names, > struct msm_iommu *iommu = to_msm_iommu(mmu); > int ret; > > - pm_runtime_get_sync(mmu->dev); > ret = iommu_attach_device(iommu->domain, mmu->dev); > - pm_runtime_put_sync(mmu->dev); > > return ret; > } > @@ -52,9 +50,7 @@ static void msm_iommu_detach(struct msm_mmu *mmu, const char * const *names, > { > struct msm_iommu *iommu = to_msm_iommu(mmu); > > - pm_runtime_get_sync(mmu->dev); > iommu_detach_device(iommu->domain, mmu->dev); > - pm_runtime_put_sync(mmu->dev); > } > > static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, > @@ -63,9 +59,7 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, > struct msm_iommu *iommu = to_msm_iommu(mmu); > size_t ret; > > -// pm_runtime_get_sync(mmu->dev); > ret = iommu_map_sg(iommu->domain, iova, sgt->sgl, sgt->nents, prot); > -// pm_runtime_put_sync(mmu->dev); > WARN_ON(ret < 0); > > return (ret == len) ? 0 : -EINVAL; > @@ -76,9 +70,7 @@ static int msm_iommu_unmap(struct msm_mmu *mmu, uint64_t iova, > { > struct msm_iommu *iommu = to_msm_iommu(mmu); > > - pm_runtime_get_sync(mmu->dev); > iommu_unmap(iommu->domain, iova, len); > - pm_runtime_put_sync(mmu->dev); > > return 0; > } > -- > 2.17.1 > > _______________________________________________ > dri-devel mailing list > dri-devel@lists.freedesktop.org > https://lists.freedesktop.org/mailman/listinfo/dri-devel _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu [not found] ` <20180611182604.30467-2-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2018-06-12 16:55 ` Kristian Høgsberg @ 2018-06-14 7:00 ` Vivek Gautam 2018-06-14 16:09 ` Jordan Crouse 1 sibling, 1 reply; 12+ messages in thread From: Vivek Gautam @ 2018-06-14 7:00 UTC (permalink / raw) To: Jordan Crouse; +Cc: linux-arm-msm, freedreno, dri-devel Hi Jordan, On Mon, Jun 11, 2018 at 11:56 PM, Jordan Crouse <jcrouse@codeaurora.org> wrote: > Now that the IOMMU is the master of it's own power we don't need to bring > up the GPU to do IOMMU operations. This is good because bringing up a6xx > requires the GMU so calling pm_runtime_get_sync() too early in the process > gets us into some nasty circular dependency situations. Thanks for the patch. While you are removing these calls, we should add pm_runtime calls to qcom_iommu_map(). That should make qcom_iommu too master of its power control. Then we should be good to go with this patch. > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> > --- > drivers/gpu/drm/msm/msm_iommu.c | 8 -------- > 1 file changed, 8 deletions(-) > > diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c > index b23d33622f37..ccd93ac6a4d8 100644 > --- a/drivers/gpu/drm/msm/msm_iommu.c > +++ b/drivers/gpu/drm/msm/msm_iommu.c > @@ -40,9 +40,7 @@ static int msm_iommu_attach(struct msm_mmu *mmu, const char * const *names, > struct msm_iommu *iommu = to_msm_iommu(mmu); > int ret; > > - pm_runtime_get_sync(mmu->dev); > ret = iommu_attach_device(iommu->domain, mmu->dev); > - pm_runtime_put_sync(mmu->dev); may be just do the following here. return iommu_attach_device(iommu->domain, mmu->dev); Rest looks good. So after the change. Reviewed-by: Vivek Gautam <vivek.gautam@codeaurora.org> Best regards Vivek > > return ret; > } > @@ -52,9 +50,7 @@ static void msm_iommu_detach(struct msm_mmu *mmu, const char * const *names, > { > struct msm_iommu *iommu = to_msm_iommu(mmu); > > - pm_runtime_get_sync(mmu->dev); > iommu_detach_device(iommu->domain, mmu->dev); > - pm_runtime_put_sync(mmu->dev); > } > > static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, > @@ -63,9 +59,7 @@ static int msm_iommu_map(struct msm_mmu *mmu, uint64_t iova, > struct msm_iommu *iommu = to_msm_iommu(mmu); > size_t ret; > > -// pm_runtime_get_sync(mmu->dev); > ret = iommu_map_sg(iommu->domain, iova, sgt->sgl, sgt->nents, prot); > -// pm_runtime_put_sync(mmu->dev); > WARN_ON(ret < 0); > > return (ret == len) ? 0 : -EINVAL; > @@ -76,9 +70,7 @@ static int msm_iommu_unmap(struct msm_mmu *mmu, uint64_t iova, > { > struct msm_iommu *iommu = to_msm_iommu(mmu); > > - pm_runtime_get_sync(mmu->dev); > iommu_unmap(iommu->domain, iova, len); > - pm_runtime_put_sync(mmu->dev); > > return 0; > } > -- > 2.17.1 > > -- > To unsubscribe from this list: send the line "unsubscribe linux-arm-msm" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- QUALCOMM INDIA, on behalf of Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, hosted by The Linux Foundation _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu 2018-06-14 7:00 ` Vivek Gautam @ 2018-06-14 16:09 ` Jordan Crouse 0 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-06-14 16:09 UTC (permalink / raw) To: Vivek Gautam; +Cc: linux-arm-msm, freedreno, dri-devel On Thu, Jun 14, 2018 at 12:30:35PM +0530, Vivek Gautam wrote: > Hi Jordan, > > On Mon, Jun 11, 2018 at 11:56 PM, Jordan Crouse <jcrouse@codeaurora.org> wrote: > > Now that the IOMMU is the master of it's own power we don't need to bring > > up the GPU to do IOMMU operations. This is good because bringing up a6xx > > requires the GMU so calling pm_runtime_get_sync() too early in the process > > gets us into some nasty circular dependency situations. > > Thanks for the patch. > While you are removing these calls, we should add pm_runtime calls > to qcom_iommu_map(). That should make qcom_iommu too master of > its power control. > Then we should be good to go with this patch. Right - I told Rob about that in IRC but I should have mentioned it here as well. In order to do this we need to be sure that all of of the possible MMU combinations are covered. > > > > Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> > > --- > > drivers/gpu/drm/msm/msm_iommu.c | 8 -------- > > 1 file changed, 8 deletions(-) > > > > diff --git a/drivers/gpu/drm/msm/msm_iommu.c b/drivers/gpu/drm/msm/msm_iommu.c > > index b23d33622f37..ccd93ac6a4d8 100644 > > --- a/drivers/gpu/drm/msm/msm_iommu.c > > +++ b/drivers/gpu/drm/msm/msm_iommu.c > > @@ -40,9 +40,7 @@ static int msm_iommu_attach(struct msm_mmu *mmu, const char * const *names, > > struct msm_iommu *iommu = to_msm_iommu(mmu); > > int ret; > > > > - pm_runtime_get_sync(mmu->dev); > > ret = iommu_attach_device(iommu->domain, mmu->dev); > > - pm_runtime_put_sync(mmu->dev); > > may be just do the following here. > return iommu_attach_device(iommu->domain, mmu->dev); I'll do that. Thanks. Jordan -- The Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project _______________________________________________ dri-devel mailing list dri-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/dri-devel ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 2/5] drm/msm: Add a helper function to parse clock names [not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2018-06-11 18:26 ` [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu Jordan Crouse @ 2018-06-11 18:26 ` Jordan Crouse 2018-06-11 18:26 ` [PATCH 3/5] drm/msm/adreno: Load the firmware before bringing up the hardware Jordan Crouse ` (2 subsequent siblings) 4 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-06-11 18:26 UTC (permalink / raw) To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA Add a helper function to parse the clock names and set up the bulk data so we can take advantage of the bulk clock functions instead of rolling our own. This is added as a helper function so the upcoming a6xx GMU code can also take advantage of it. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/msm_drv.c | 57 +++++++++++++++++++++++++++++++++ drivers/gpu/drm/msm/msm_drv.h | 4 +++ drivers/gpu/drm/msm/msm_gpu.c | 60 ++++++----------------------------- drivers/gpu/drm/msm/msm_gpu.h | 2 +- 4 files changed, 72 insertions(+), 51 deletions(-) diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c index 7f7321eb5312..3e86fc9be39f 100644 --- a/drivers/gpu/drm/msm/msm_drv.c +++ b/drivers/gpu/drm/msm/msm_drv.c @@ -78,6 +78,63 @@ module_param(modeset, bool, 0600); * Util/helpers: */ +int msm_clk_bulk_get(struct device *dev, struct clk_bulk_data **bulk) +{ + struct property *prop; + const char *name; + struct clk_bulk_data *local; + int i = 0, ret, count; + + count = of_property_count_strings(dev->of_node, "clock-names"); + if (count < 1) + return 0; + + local = devm_kcalloc(dev, sizeof(struct clk_bulk_data *), + count, GFP_KERNEL); + if (!local) + return -ENOMEM; + + of_property_for_each_string(dev->of_node, "clock-names", prop, name) { + local[i].id = devm_kstrdup(dev, name, GFP_KERNEL); + if (!local[i].id) { + devm_kfree(dev, local); + return -ENOMEM; + } + + i++; + } + + ret = devm_clk_bulk_get(dev, count, local); + + if (ret) { + for (i = 0; i < count; i++) + devm_kfree(dev, (void *) local[i].id); + devm_kfree(dev, local); + + return ret; + } + + *bulk = local; + return count; +} + +struct clk *msm_clk_bulk_get_clock(struct clk_bulk_data *bulk, int count, + const char *name) +{ + int i; + char n[32]; + + snprintf(n, sizeof(n), "%s_clk", name); + + for (i = 0; bulk && i < count; i++) { + if (!strcmp(bulk[i].id, name) || !strcmp(bulk[i].id, n)) + return bulk[i].clk; + } + + + return NULL; +} + struct clk *msm_clk_get(struct platform_device *pdev, const char *name) { struct clk *clk; diff --git a/drivers/gpu/drm/msm/msm_drv.h b/drivers/gpu/drm/msm/msm_drv.h index 17cefca1d566..011c34ad1f93 100644 --- a/drivers/gpu/drm/msm/msm_drv.h +++ b/drivers/gpu/drm/msm/msm_drv.h @@ -307,6 +307,10 @@ static inline void msm_perf_debugfs_cleanup(struct msm_drm_private *priv) {} #endif struct clk *msm_clk_get(struct platform_device *pdev, const char *name); +int msm_clk_bulk_get(struct device *dev, struct clk_bulk_data **bulk); + +struct clk *msm_clk_bulk_get_clock(struct clk_bulk_data *bulk, int count, + const char *name); void __iomem *msm_ioremap(struct platform_device *pdev, const char *name, const char *dbgname); void msm_writel(u32 data, void __iomem *addr); diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 1c09acfb4028..d4d5a68e468d 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -141,8 +141,6 @@ static int disable_pwrrail(struct msm_gpu *gpu) static int enable_clk(struct msm_gpu *gpu) { - int i; - if (gpu->core_clk && gpu->fast_rate) clk_set_rate(gpu->core_clk, gpu->fast_rate); @@ -150,28 +148,14 @@ static int enable_clk(struct msm_gpu *gpu) if (gpu->rbbmtimer_clk) clk_set_rate(gpu->rbbmtimer_clk, 19200000); - for (i = gpu->nr_clocks - 1; i >= 0; i--) - if (gpu->grp_clks[i]) - clk_prepare(gpu->grp_clks[i]); - - for (i = gpu->nr_clocks - 1; i >= 0; i--) - if (gpu->grp_clks[i]) - clk_enable(gpu->grp_clks[i]); + clk_bulk_prepare_enable(gpu->nr_clocks, gpu->grp_clks); return 0; } static int disable_clk(struct msm_gpu *gpu) { - int i; - - for (i = gpu->nr_clocks - 1; i >= 0; i--) - if (gpu->grp_clks[i]) - clk_disable(gpu->grp_clks[i]); - - for (i = gpu->nr_clocks - 1; i >= 0; i--) - if (gpu->grp_clks[i]) - clk_unprepare(gpu->grp_clks[i]); + clk_bulk_disable_unprepare(gpu->nr_clocks, gpu->grp_clks); /* * Set the clock to a deliberately low rate. On older targets the clock @@ -660,44 +644,20 @@ static irqreturn_t irq_handler(int irq, void *data) return gpu->funcs->irq(gpu); } -static struct clk *get_clock(struct device *dev, const char *name) -{ - struct clk *clk = devm_clk_get(dev, name); - - return IS_ERR(clk) ? NULL : clk; -} - static int get_clocks(struct platform_device *pdev, struct msm_gpu *gpu) { - struct device *dev = &pdev->dev; - struct property *prop; - const char *name; - int i = 0; - - gpu->nr_clocks = of_property_count_strings(dev->of_node, "clock-names"); - if (gpu->nr_clocks < 1) { - gpu->nr_clocks = 0; - return 0; - } + int ret = msm_clk_bulk_get(&pdev->dev, &gpu->grp_clks); - gpu->grp_clks = devm_kcalloc(dev, sizeof(struct clk *), gpu->nr_clocks, - GFP_KERNEL); - if (!gpu->grp_clks) { - gpu->nr_clocks = 0; - return -ENOMEM; - } + if (ret < 1) + return ret; - of_property_for_each_string(dev->of_node, "clock-names", prop, name) { - gpu->grp_clks[i] = get_clock(dev, name); + gpu->nr_clocks = ret; - /* Remember the key clocks that we need to control later */ - if (!strcmp(name, "core") || !strcmp(name, "core_clk")) - gpu->core_clk = gpu->grp_clks[i]; - else if (!strcmp(name, "rbbmtimer") || !strcmp(name, "rbbmtimer_clk")) - gpu->rbbmtimer_clk = gpu->grp_clks[i]; + gpu->core_clk = msm_clk_bulk_get_clock(gpu->grp_clks, + gpu->nr_clocks, "core"); - ++i; - } + gpu->rbbmtimer_clk = msm_clk_bulk_get_clock(gpu->grp_clks, + gpu->nr_clocks, "rbbmtimer"); return 0; } diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h index b8241179175a..c9bc277e2349 100644 --- a/drivers/gpu/drm/msm/msm_gpu.h +++ b/drivers/gpu/drm/msm/msm_gpu.h @@ -108,7 +108,7 @@ struct msm_gpu { /* Power Control: */ struct regulator *gpu_reg, *gpu_cx; - struct clk **grp_clks; + struct clk_bulk_data *grp_clks; int nr_clocks; struct clk *ebi1_clk, *core_clk, *rbbmtimer_clk; uint32_t fast_rate; -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 3/5] drm/msm/adreno: Load the firmware before bringing up the hardware [not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2018-06-11 18:26 ` [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu Jordan Crouse 2018-06-11 18:26 ` [PATCH 2/5] drm/msm: Add a helper function to parse clock names Jordan Crouse @ 2018-06-11 18:26 ` Jordan Crouse [not found] ` <20180611182604.30467-4-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> 2018-06-11 18:26 ` [PATCH 4/5] drm/msm: Add generated headers for A6XX Jordan Crouse 2018-06-11 18:26 ` [PATCH 5/5] drm/msm: Add A6XX device support Jordan Crouse 4 siblings, 1 reply; 12+ messages in thread From: Jordan Crouse @ 2018-06-11 18:26 UTC (permalink / raw) To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA Failure to load firwmare is the primary reason to fail adreno_load_gpu(). Try to load it first before going into the hardware initialization code and unwinding it. This is important for a6xx because the GMU gets loaded from the runtime power code and it is more costly to fail in that path because of missing firmware. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/adreno/adreno_device.c | 23 +++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index 0c0eaad68065..d70e7d145dae 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -155,6 +155,7 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev) struct msm_drm_private *priv = dev->dev_private; struct platform_device *pdev = priv->gpu_pdev; struct msm_gpu *gpu = NULL; + struct adreno_gpu *adreno_gpu; int ret; if (pdev) @@ -165,7 +166,27 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev) return NULL; } - pm_runtime_get_sync(&pdev->dev); + adreno_gpu = to_adreno_gpu(gpu); + + /* + * The number one reason for HW init to fail is if the firmware isn't + * loaded yet. Try that first and don't bother continuing on + * otherwise + */ + + ret = adreno_load_fw(adreno_gpu); + if (ret) + return NULL; + + /* Make sure pm runtime is active and reset any previous errors */ + pm_runtime_set_active(&pdev->dev); + + ret = pm_runtime_get_sync(&pdev->dev); + if (ret < 0) { + dev_err(dev->dev, "Couldn't power up the GPU: %d\n", ret); + return NULL; + } + mutex_lock(&dev->struct_mutex); ret = msm_gpu_hw_init(gpu); mutex_unlock(&dev->struct_mutex); -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
[parent not found: <20180611182604.30467-4-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>]
* Re: [PATCH 3/5] drm/msm/adreno: Load the firmware before bringing up the hardware [not found] ` <20180611182604.30467-4-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> @ 2018-06-11 21:50 ` kbuild test robot 0 siblings, 0 replies; 12+ messages in thread From: kbuild test robot @ 2018-06-11 21:50 UTC (permalink / raw) To: Jordan Crouse Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, kbuild-all-JC7UmRfGjtg, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW [-- Attachment #1: Type: text/plain, Size: 3039 bytes --] Hi Jordan, Thank you for the patch! Yet something to improve: [auto build test ERROR on robclark/msm-next] [also build test ERROR on v4.17 next-20180608] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits/Jordan-Crouse/drm-msm-Add-support-for-Adreno-a6xx/20180612-022828 base: git://people.freedesktop.org/~robclark/linux msm-next config: arm64-defconfig (attached as .config) compiler: aarch64-linux-gnu-gcc (Debian 7.2.0-11) 7.2.0 reproduce: wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross chmod +x ~/bin/make.cross # save the attached .config to linux build tree GCC_VERSION=7.2.0 make.cross ARCH=arm64 Note: the linux-review/Jordan-Crouse/drm-msm-Add-support-for-Adreno-a6xx/20180612-022828 HEAD 716a0bd2934f1b01e622ac7a6714789861824f8f builds fine. It only hurts bisectibility. All errors (new ones prefixed by >>): drivers/gpu/drm/msm/adreno/adreno_device.c: In function 'adreno_load_gpu': >> drivers/gpu/drm/msm/adreno/adreno_device.c:177:8: error: implicit declaration of function 'adreno_load_fw'; did you mean 'adreno_load_gpu'? [-Werror=implicit-function-declaration] ret = adreno_load_fw(adreno_gpu); ^~~~~~~~~~~~~~ adreno_load_gpu cc1: some warnings being treated as errors vim +177 drivers/gpu/drm/msm/adreno/adreno_device.c 152 153 struct msm_gpu *adreno_load_gpu(struct drm_device *dev) 154 { 155 struct msm_drm_private *priv = dev->dev_private; 156 struct platform_device *pdev = priv->gpu_pdev; 157 struct msm_gpu *gpu = NULL; 158 struct adreno_gpu *adreno_gpu; 159 int ret; 160 161 if (pdev) 162 gpu = platform_get_drvdata(pdev); 163 164 if (!gpu) { 165 dev_err_once(dev->dev, "no GPU device was found\n"); 166 return NULL; 167 } 168 169 adreno_gpu = to_adreno_gpu(gpu); 170 171 /* 172 * The number one reason for HW init to fail is if the firmware isn't 173 * loaded yet. Try that first and don't bother continuing on 174 * otherwise 175 */ 176 > 177 ret = adreno_load_fw(adreno_gpu); 178 if (ret) 179 return NULL; 180 181 /* Make sure pm runtime is active and reset any previous errors */ 182 pm_runtime_set_active(&pdev->dev); 183 184 ret = pm_runtime_get_sync(&pdev->dev); 185 if (ret < 0) { 186 dev_err(dev->dev, "Couldn't power up the GPU: %d\n", ret); 187 return NULL; 188 } 189 190 mutex_lock(&dev->struct_mutex); 191 ret = msm_gpu_hw_init(gpu); 192 mutex_unlock(&dev->struct_mutex); 193 pm_runtime_put_autosuspend(&pdev->dev); 194 if (ret) { 195 dev_err(dev->dev, "gpu hw init failed: %d\n", ret); 196 return NULL; 197 } 198 --- 0-DAY kernel test infrastructure Open Source Technology Center https://lists.01.org/pipermail/kbuild-all Intel Corporation [-- Attachment #2: .config.gz --] [-- Type: application/gzip, Size: 37441 bytes --] [-- Attachment #3: Type: text/plain, Size: 160 bytes --] _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply [flat|nested] 12+ messages in thread
* [PATCH 4/5] drm/msm: Add generated headers for A6XX [not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> ` (2 preceding siblings ...) 2018-06-11 18:26 ` [PATCH 3/5] drm/msm/adreno: Load the firmware before bringing up the hardware Jordan Crouse @ 2018-06-11 18:26 ` Jordan Crouse 2018-06-11 18:26 ` [PATCH 5/5] drm/msm: Add A6XX device support Jordan Crouse 4 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-06-11 18:26 UTC (permalink / raw) To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, Sharat Masetty From: Sharat Masetty <smasetty@codeaurora.org> Add initial register headers for A6XX targets. Signed-off-by: Sharat Masetty <smasetty@codeaurora.org> Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/adreno/a6xx.xml.h | 1784 +++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h | 382 +++++ 2 files changed, 2166 insertions(+) create mode 100644 drivers/gpu/drm/msm/adreno/a6xx.xml.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h diff --git a/drivers/gpu/drm/msm/adreno/a6xx.xml.h b/drivers/gpu/drm/msm/adreno/a6xx.xml.h new file mode 100644 index 000000000000..5af12fe3f95c --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx.xml.h @@ -0,0 +1,1784 @@ +#ifndef A6XX_XML +#define A6XX_XML + +/* Autogenerated file, DO NOT EDIT manually! + +This file was generated by the rules-ng-ng headergen tool in this git repository: +http://github.com/freedreno/envytools/ +git clone https://github.com/freedreno/envytools.git + +The rules-ng-ng source files this header was generated from are: +- ./adreno.xml ( 501 bytes, from 2018-05-23 16:51:57) +- ./freedreno_copyright.xml ( 1572 bytes, from 2016-10-24 21:12:27) +- ./adreno/a2xx.xml ( 36805 bytes, from 2018-05-23 16:51:57) +- ./adreno/adreno_common.xml ( 13634 bytes, from 2018-05-23 16:51:57) +- ./adreno/adreno_pm4.xml ( 38703 bytes, from 2018-05-23 16:51:57) +- ./adreno/a3xx.xml ( 83840 bytes, from 2017-12-05 18:20:27) +- ./adreno/a4xx.xml ( 112086 bytes, from 2018-05-23 16:51:57) +- ./adreno/a5xx.xml ( 146494 bytes, from 2018-05-23 16:51:57) +- ./adreno/a6xx.xml ( 69957 bytes, from 2018-05-23 17:09:08) +- ./adreno/a6xx_gmu.xml ( 10431 bytes, from 2018-05-23 16:52:19) +- ./adreno/ocmem.xml ( 1773 bytes, from 2016-10-24 21:12:27) + +Copyright (C) 2013-2018 by the following authors: +- Rob Clark <robdclark@gmail.com> (robclark) +- Ilia Mirkin <imirkin@alum.mit.edu> (imirkin) + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice (including the +next paragraph) shall be included in all copies or substantial +portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +*/ + + +enum a6xx_cp_perfcounter_select { + PERF_CP_ALWAYS_COUNT = 0, +}; + +enum a6xx_event_write { + PC_CCU_INVALIDATE_DEPTH = 24, + PC_CCU_INVALIDATE_COLOR = 25, +}; + +#define A6XX_RBBM_INT_0_MASK_RBBM_GPU_IDLE 0x00000001 +#define A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR 0x00000002 +#define A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW 0x00000040 +#define A6XX_RBBM_INT_0_MASK_RBBM_GPC_ERROR 0x00000080 +#define A6XX_RBBM_INT_0_MASK_CP_SW 0x00000100 +#define A6XX_RBBM_INT_0_MASK_CP_HW_ERROR 0x00000200 +#define A6XX_RBBM_INT_0_MASK_CP_CCU_FLUSH_DEPTH_TS 0x00000400 +#define A6XX_RBBM_INT_0_MASK_CP_CCU_FLUSH_COLOR_TS 0x00000800 +#define A6XX_RBBM_INT_0_MASK_CP_CCU_RESOLVE_TS 0x00001000 +#define A6XX_RBBM_INT_0_MASK_CP_IB2 0x00002000 +#define A6XX_RBBM_INT_0_MASK_CP_IB1 0x00004000 +#define A6XX_RBBM_INT_0_MASK_CP_RB 0x00008000 +#define A6XX_RBBM_INT_0_MASK_CP_RB_DONE_TS 0x00020000 +#define A6XX_RBBM_INT_0_MASK_CP_WT_DONE_TS 0x00040000 +#define A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS 0x00100000 +#define A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW 0x00400000 +#define A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT 0x00800000 +#define A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS 0x01000000 +#define A6XX_RBBM_INT_0_MASK_UCHE_TRAP_INTR 0x02000000 +#define A6XX_RBBM_INT_0_MASK_DEBBUS_INTR_0 0x04000000 +#define A6XX_RBBM_INT_0_MASK_DEBBUS_INTR_1 0x08000000 +#define A6XX_RBBM_INT_0_MASK_ISDB_CPU_IRQ 0x40000000 +#define A6XX_RBBM_INT_0_MASK_ISDB_UNDER_DEBUG 0x80000000 +#define A6XX_CP_INT_CP_OPCODE_ERROR 0x00000001 +#define A6XX_CP_INT_CP_UCODE_ERROR 0x00000002 +#define A6XX_CP_INT_CP_HW_FAULT_ERROR 0x00000004 +#define A6XX_CP_INT_CP_REGISTER_PROTECTION_ERROR 0x00000010 +#define A6XX_CP_INT_CP_AHB_ERROR 0x00000020 +#define A6XX_CP_INT_CP_VSD_PARITY_ERROR 0x00000040 +#define A6XX_CP_INT_CP_ILLEGAL_INSTR_ERROR 0x00000080 +#define REG_A6XX_CP_RB_BASE 0x00000800 + +#define REG_A6XX_CP_RB_BASE_HI 0x00000801 + +#define REG_A6XX_CP_RB_CNTL 0x00000802 + +#define REG_A6XX_CP_RB_RPTR_ADDR_LO 0x00000804 + +#define REG_A6XX_CP_RB_RPTR_ADDR_HI 0x00000805 + +#define REG_A6XX_CP_RB_RPTR 0x00000806 + +#define REG_A6XX_CP_RB_WPTR 0x00000807 + +#define REG_A6XX_CP_SQE_CNTL 0x00000808 + +#define REG_A6XX_CP_HW_FAULT 0x00000821 + +#define REG_A6XX_CP_INTERRUPT_STATUS 0x00000823 + +#define REG_A6XX_CP_PROTECT_STATUS 0x00000824 + +#define REG_A6XX_CP_SQE_INSTR_BASE_LO 0x00000830 + +#define REG_A6XX_CP_SQE_INSTR_BASE_HI 0x00000831 + +#define REG_A6XX_CP_MISC_CNTL 0x00000840 + +#define REG_A6XX_CP_ROQ_THRESHOLDS_1 0x000008c1 + +#define REG_A6XX_CP_ROQ_THRESHOLDS_2 0x000008c2 + +#define REG_A6XX_CP_MEM_POOL_SIZE 0x000008c3 + +#define REG_A6XX_CP_CHICKEN_DBG 0x00000841 + +#define REG_A6XX_CP_ADDR_MODE_CNTL 0x00000842 + +#define REG_A6XX_CP_DBG_ECO_CNTL 0x00000843 + +#define REG_A6XX_CP_PROTECT_CNTL 0x0000084f + +static inline uint32_t REG_A6XX_CP_SCRATCH(uint32_t i0) { return 0x00000883 + 0x1*i0; } + +static inline uint32_t REG_A6XX_CP_SCRATCH_REG(uint32_t i0) { return 0x00000883 + 0x1*i0; } + +static inline uint32_t REG_A6XX_CP_PROTECT(uint32_t i0) { return 0x00000850 + 0x1*i0; } + +static inline uint32_t REG_A6XX_CP_PROTECT_REG(uint32_t i0) { return 0x00000850 + 0x1*i0; } +#define A6XX_CP_PROTECT_REG_BASE_ADDR__MASK 0x0003ffff +#define A6XX_CP_PROTECT_REG_BASE_ADDR__SHIFT 0 +static inline uint32_t A6XX_CP_PROTECT_REG_BASE_ADDR(uint32_t val) +{ + return ((val) << A6XX_CP_PROTECT_REG_BASE_ADDR__SHIFT) & A6XX_CP_PROTECT_REG_BASE_ADDR__MASK; +} +#define A6XX_CP_PROTECT_REG_MASK_LEN__MASK 0x7ffc0000 +#define A6XX_CP_PROTECT_REG_MASK_LEN__SHIFT 18 +static inline uint32_t A6XX_CP_PROTECT_REG_MASK_LEN(uint32_t val) +{ + return ((val) << A6XX_CP_PROTECT_REG_MASK_LEN__SHIFT) & A6XX_CP_PROTECT_REG_MASK_LEN__MASK; +} +#define A6XX_CP_PROTECT_REG_READ 0x80000000 + +#define REG_A6XX_CP_CONTEXT_SWITCH_CNTL 0x000008a0 + +#define REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO_LO 0x000008a1 + +#define REG_A6XX_CP_CONTEXT_SWITCH_SMMU_INFO_HI 0x000008a2 + +#define REG_A6XX_CP_CONTEXT_SWITCH_PRIV_NON_SECURE_RESTORE_ADDR_LO 0x000008a3 + +#define REG_A6XX_CP_CONTEXT_SWITCH_PRIV_NON_SECURE_RESTORE_ADDR_HI 0x000008a4 + +#define REG_A6XX_CP_CONTEXT_SWITCH_PRIV_SECURE_RESTORE_ADDR_LO 0x000008a5 + +#define REG_A6XX_CP_CONTEXT_SWITCH_PRIV_SECURE_RESTORE_ADDR_HI 0x000008a6 + +#define REG_A6XX_CP_CONTEXT_SWITCH_NON_PRIV_RESTORE_ADDR_LO 0x000008a7 + +#define REG_A6XX_CP_CONTEXT_SWITCH_NON_PRIV_RESTORE_ADDR_HI 0x000008a8 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_0 0x000008d0 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_1 0x000008d1 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_2 0x000008d2 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_3 0x000008d3 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_4 0x000008d4 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_5 0x000008d5 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_6 0x000008d6 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_7 0x000008d7 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_8 0x000008d8 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_9 0x000008d9 + +#define REG_A6XX_CP_PERFCTR_CP_SEL_10 0x000008da + +#define REG_A6XX_CP_PERFCTR_CP_SEL_11 0x000008db + +#define REG_A6XX_CP_PERFCTR_CP_SEL_12 0x000008dc + +#define REG_A6XX_CP_PERFCTR_CP_SEL_13 0x000008dd + +#define REG_A6XX_CP_CRASH_SCRIPT_BASE_LO 0x00000900 + +#define REG_A6XX_CP_CRASH_SCRIPT_BASE_HI 0x00000901 + +#define REG_A6XX_CP_CRASH_DUMP_CNTL 0x00000902 + +#define REG_A6XX_CP_CRASH_DUMP_STATUS 0x00000903 + +#define REG_A6XX_CP_SQE_STAT_ADDR 0x00000908 + +#define REG_A6XX_CP_SQE_STAT_DATA 0x00000909 + +#define REG_A6XX_CP_DRAW_STATE_ADDR 0x0000090a + +#define REG_A6XX_CP_DRAW_STATE_DATA 0x0000090b + +#define REG_A6XX_CP_ROQ_DBG_ADDR 0x0000090c + +#define REG_A6XX_CP_ROQ_DBG_DATA 0x0000090d + +#define REG_A6XX_CP_MEM_POOL_DBG_ADDR 0x0000090e + +#define REG_A6XX_CP_MEM_POOL_DBG_DATA 0x0000090f + +#define REG_A6XX_CP_SQE_UCODE_DBG_ADDR 0x00000910 + +#define REG_A6XX_CP_SQE_UCODE_DBG_DATA 0x00000911 + +#define REG_A6XX_CP_IB1_BASE 0x00000928 + +#define REG_A6XX_CP_IB1_BASE_HI 0x00000929 + +#define REG_A6XX_CP_IB1_REM_SIZE 0x0000092a + +#define REG_A6XX_CP_IB2_BASE 0x0000092b + +#define REG_A6XX_CP_IB2_BASE_HI 0x0000092c + +#define REG_A6XX_CP_IB2_REM_SIZE 0x0000092d + +#define REG_A6XX_CP_ALWAYS_ON_COUNTER_LO 0x00000980 + +#define REG_A6XX_CP_ALWAYS_ON_COUNTER_HI 0x00000981 + +#define REG_A6XX_CP_AHB_CNTL 0x0000098d + +#define REG_A6XX_CP_APERTURE_CNTL_HOST 0x00000a00 + +#define REG_A6XX_CP_APERTURE_CNTL_CD 0x00000a03 + +#define REG_A6XX_VSC_ADDR_MODE_CNTL 0x00000c01 + +#define REG_A6XX_RBBM_INT_0_STATUS 0x00000201 + +#define REG_A6XX_RBBM_STATUS 0x00000210 +#define A6XX_RBBM_STATUS_GPU_BUSY_IGN_AHB 0x00800000 +#define A6XX_RBBM_STATUS_GPU_BUSY_IGN_AHB_CP 0x00400000 +#define A6XX_RBBM_STATUS_HLSQ_BUSY 0x00200000 +#define A6XX_RBBM_STATUS_VSC_BUSY 0x00100000 +#define A6XX_RBBM_STATUS_TPL1_BUSY 0x00080000 +#define A6XX_RBBM_STATUS_SP_BUSY 0x00040000 +#define A6XX_RBBM_STATUS_UCHE_BUSY 0x00020000 +#define A6XX_RBBM_STATUS_VPC_BUSY 0x00010000 +#define A6XX_RBBM_STATUS_VFD_BUSY 0x00008000 +#define A6XX_RBBM_STATUS_TESS_BUSY 0x00004000 +#define A6XX_RBBM_STATUS_PC_VSD_BUSY 0x00002000 +#define A6XX_RBBM_STATUS_PC_DCALL_BUSY 0x00001000 +#define A6XX_RBBM_STATUS_COM_DCOM_BUSY 0x00000800 +#define A6XX_RBBM_STATUS_LRZ_BUSY 0x00000400 +#define A6XX_RBBM_STATUS_A2D_BUSY 0x00000200 +#define A6XX_RBBM_STATUS_CCU_BUSY 0x00000100 +#define A6XX_RBBM_STATUS_RB_BUSY 0x00000080 +#define A6XX_RBBM_STATUS_RAS_BUSY 0x00000040 +#define A6XX_RBBM_STATUS_TSE_BUSY 0x00000020 +#define A6XX_RBBM_STATUS_VBIF_BUSY 0x00000010 +#define A6XX_RBBM_STATUS_GFX_DBGC_BUSY 0x00000008 +#define A6XX_RBBM_STATUS_CP_BUSY 0x00000004 +#define A6XX_RBBM_STATUS_CP_AHB_BUSY_CP_MASTER 0x00000002 +#define A6XX_RBBM_STATUS_CP_AHB_BUSY_CX_MASTER 0x00000001 + +#define REG_A6XX_RBBM_STATUS3 0x00000213 + +#define REG_A6XX_RBBM_VBIF_GX_RESET_STATUS 0x00000215 + +#define REG_A6XX_RBBM_PERFCTR_CP_0_LO 0x00000400 + +#define REG_A6XX_RBBM_PERFCTR_CP_0_HI 0x00000401 + +#define REG_A6XX_RBBM_PERFCTR_CP_1_LO 0x00000402 + +#define REG_A6XX_RBBM_PERFCTR_CP_1_HI 0x00000403 + +#define REG_A6XX_RBBM_PERFCTR_CP_2_LO 0x00000404 + +#define REG_A6XX_RBBM_PERFCTR_CP_2_HI 0x00000405 + +#define REG_A6XX_RBBM_PERFCTR_CP_3_LO 0x00000406 + +#define REG_A6XX_RBBM_PERFCTR_CP_3_HI 0x00000407 + +#define REG_A6XX_RBBM_PERFCTR_CP_4_LO 0x00000408 + +#define REG_A6XX_RBBM_PERFCTR_CP_4_HI 0x00000409 + +#define REG_A6XX_RBBM_PERFCTR_CP_5_LO 0x0000040a + +#define REG_A6XX_RBBM_PERFCTR_CP_5_HI 0x0000040b + +#define REG_A6XX_RBBM_PERFCTR_CP_6_LO 0x0000040c + +#define REG_A6XX_RBBM_PERFCTR_CP_6_HI 0x0000040d + +#define REG_A6XX_RBBM_PERFCTR_CP_7_LO 0x0000040e + +#define REG_A6XX_RBBM_PERFCTR_CP_7_HI 0x0000040f + +#define REG_A6XX_RBBM_PERFCTR_CP_8_LO 0x00000410 + +#define REG_A6XX_RBBM_PERFCTR_CP_8_HI 0x00000411 + +#define REG_A6XX_RBBM_PERFCTR_CP_9_LO 0x00000412 + +#define REG_A6XX_RBBM_PERFCTR_CP_9_HI 0x00000413 + +#define REG_A6XX_RBBM_PERFCTR_CP_10_LO 0x00000414 + +#define REG_A6XX_RBBM_PERFCTR_CP_10_HI 0x00000415 + +#define REG_A6XX_RBBM_PERFCTR_CP_11_LO 0x00000416 + +#define REG_A6XX_RBBM_PERFCTR_CP_11_HI 0x00000417 + +#define REG_A6XX_RBBM_PERFCTR_CP_12_LO 0x00000418 + +#define REG_A6XX_RBBM_PERFCTR_CP_12_HI 0x00000419 + +#define REG_A6XX_RBBM_PERFCTR_CP_13_LO 0x0000041a + +#define REG_A6XX_RBBM_PERFCTR_CP_13_HI 0x0000041b + +#define REG_A6XX_RBBM_PERFCTR_RBBM_0_LO 0x0000041c + +#define REG_A6XX_RBBM_PERFCTR_RBBM_0_HI 0x0000041d + +#define REG_A6XX_RBBM_PERFCTR_RBBM_1_LO 0x0000041e + +#define REG_A6XX_RBBM_PERFCTR_RBBM_1_HI 0x0000041f + +#define REG_A6XX_RBBM_PERFCTR_RBBM_2_LO 0x00000420 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_2_HI 0x00000421 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_3_LO 0x00000422 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_3_HI 0x00000423 + +#define REG_A6XX_RBBM_PERFCTR_PC_0_LO 0x00000424 + +#define REG_A6XX_RBBM_PERFCTR_PC_0_HI 0x00000425 + +#define REG_A6XX_RBBM_PERFCTR_PC_1_LO 0x00000426 + +#define REG_A6XX_RBBM_PERFCTR_PC_1_HI 0x00000427 + +#define REG_A6XX_RBBM_PERFCTR_PC_2_LO 0x00000428 + +#define REG_A6XX_RBBM_PERFCTR_PC_2_HI 0x00000429 + +#define REG_A6XX_RBBM_PERFCTR_PC_3_LO 0x0000042a + +#define REG_A6XX_RBBM_PERFCTR_PC_3_HI 0x0000042b + +#define REG_A6XX_RBBM_PERFCTR_PC_4_LO 0x0000042c + +#define REG_A6XX_RBBM_PERFCTR_PC_4_HI 0x0000042d + +#define REG_A6XX_RBBM_PERFCTR_PC_5_LO 0x0000042e + +#define REG_A6XX_RBBM_PERFCTR_PC_5_HI 0x0000042f + +#define REG_A6XX_RBBM_PERFCTR_PC_6_LO 0x00000430 + +#define REG_A6XX_RBBM_PERFCTR_PC_6_HI 0x00000431 + +#define REG_A6XX_RBBM_PERFCTR_PC_7_LO 0x00000432 + +#define REG_A6XX_RBBM_PERFCTR_PC_7_HI 0x00000433 + +#define REG_A6XX_RBBM_PERFCTR_VFD_0_LO 0x00000434 + +#define REG_A6XX_RBBM_PERFCTR_VFD_0_HI 0x00000435 + +#define REG_A6XX_RBBM_PERFCTR_VFD_1_LO 0x00000436 + +#define REG_A6XX_RBBM_PERFCTR_VFD_1_HI 0x00000437 + +#define REG_A6XX_RBBM_PERFCTR_VFD_2_LO 0x00000438 + +#define REG_A6XX_RBBM_PERFCTR_VFD_2_HI 0x00000439 + +#define REG_A6XX_RBBM_PERFCTR_VFD_3_LO 0x0000043a + +#define REG_A6XX_RBBM_PERFCTR_VFD_3_HI 0x0000043b + +#define REG_A6XX_RBBM_PERFCTR_VFD_4_LO 0x0000043c + +#define REG_A6XX_RBBM_PERFCTR_VFD_4_HI 0x0000043d + +#define REG_A6XX_RBBM_PERFCTR_VFD_5_LO 0x0000043e + +#define REG_A6XX_RBBM_PERFCTR_VFD_5_HI 0x0000043f + +#define REG_A6XX_RBBM_PERFCTR_VFD_6_LO 0x00000440 + +#define REG_A6XX_RBBM_PERFCTR_VFD_6_HI 0x00000441 + +#define REG_A6XX_RBBM_PERFCTR_VFD_7_LO 0x00000442 + +#define REG_A6XX_RBBM_PERFCTR_VFD_7_HI 0x00000443 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_0_LO 0x00000444 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_0_HI 0x00000445 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_1_LO 0x00000446 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_1_HI 0x00000447 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_2_LO 0x00000448 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_2_HI 0x00000449 + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_3_LO 0x0000044a + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_3_HI 0x0000044b + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_4_LO 0x0000044c + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_4_HI 0x0000044d + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_5_LO 0x0000044e + +#define REG_A6XX_RBBM_PERFCTR_HLSQ_5_HI 0x0000044f + +#define REG_A6XX_RBBM_PERFCTR_VPC_0_LO 0x00000450 + +#define REG_A6XX_RBBM_PERFCTR_VPC_0_HI 0x00000451 + +#define REG_A6XX_RBBM_PERFCTR_VPC_1_LO 0x00000452 + +#define REG_A6XX_RBBM_PERFCTR_VPC_1_HI 0x00000453 + +#define REG_A6XX_RBBM_PERFCTR_VPC_2_LO 0x00000454 + +#define REG_A6XX_RBBM_PERFCTR_VPC_2_HI 0x00000455 + +#define REG_A6XX_RBBM_PERFCTR_VPC_3_LO 0x00000456 + +#define REG_A6XX_RBBM_PERFCTR_VPC_3_HI 0x00000457 + +#define REG_A6XX_RBBM_PERFCTR_VPC_4_LO 0x00000458 + +#define REG_A6XX_RBBM_PERFCTR_VPC_4_HI 0x00000459 + +#define REG_A6XX_RBBM_PERFCTR_VPC_5_LO 0x0000045a + +#define REG_A6XX_RBBM_PERFCTR_VPC_5_HI 0x0000045b + +#define REG_A6XX_RBBM_PERFCTR_CCU_0_LO 0x0000045c + +#define REG_A6XX_RBBM_PERFCTR_CCU_0_HI 0x0000045d + +#define REG_A6XX_RBBM_PERFCTR_CCU_1_LO 0x0000045e + +#define REG_A6XX_RBBM_PERFCTR_CCU_1_HI 0x0000045f + +#define REG_A6XX_RBBM_PERFCTR_CCU_2_LO 0x00000460 + +#define REG_A6XX_RBBM_PERFCTR_CCU_2_HI 0x00000461 + +#define REG_A6XX_RBBM_PERFCTR_CCU_3_LO 0x00000462 + +#define REG_A6XX_RBBM_PERFCTR_CCU_3_HI 0x00000463 + +#define REG_A6XX_RBBM_PERFCTR_CCU_4_LO 0x00000464 + +#define REG_A6XX_RBBM_PERFCTR_CCU_4_HI 0x00000465 + +#define REG_A6XX_RBBM_PERFCTR_TSE_0_LO 0x00000466 + +#define REG_A6XX_RBBM_PERFCTR_TSE_0_HI 0x00000467 + +#define REG_A6XX_RBBM_PERFCTR_TSE_1_LO 0x00000468 + +#define REG_A6XX_RBBM_PERFCTR_TSE_1_HI 0x00000469 + +#define REG_A6XX_RBBM_PERFCTR_TSE_2_LO 0x0000046a + +#define REG_A6XX_RBBM_PERFCTR_CCU_4_HI 0x00000465 + +#define REG_A6XX_RBBM_PERFCTR_TSE_0_LO 0x00000466 + +#define REG_A6XX_RBBM_PERFCTR_TSE_0_HI 0x00000467 + +#define REG_A6XX_RBBM_PERFCTR_TSE_1_LO 0x00000468 + +#define REG_A6XX_RBBM_PERFCTR_TSE_1_HI 0x00000469 + +#define REG_A6XX_RBBM_PERFCTR_TSE_2_LO 0x0000046a + +#define REG_A6XX_RBBM_PERFCTR_TSE_2_HI 0x0000046b + +#define REG_A6XX_RBBM_PERFCTR_TSE_3_LO 0x0000046c + +#define REG_A6XX_RBBM_PERFCTR_TSE_3_HI 0x0000046d + +#define REG_A6XX_RBBM_PERFCTR_RAS_0_LO 0x0000046e + +#define REG_A6XX_RBBM_PERFCTR_RAS_0_HI 0x0000046f + +#define REG_A6XX_RBBM_PERFCTR_RAS_1_LO 0x00000470 + +#define REG_A6XX_RBBM_PERFCTR_RAS_1_HI 0x00000471 + +#define REG_A6XX_RBBM_PERFCTR_RAS_2_LO 0x00000472 + +#define REG_A6XX_RBBM_PERFCTR_RAS_2_HI 0x00000473 + +#define REG_A6XX_RBBM_PERFCTR_RAS_3_LO 0x00000474 + +#define REG_A6XX_RBBM_PERFCTR_RAS_3_HI 0x00000475 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_0_LO 0x00000476 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_0_HI 0x00000477 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_1_LO 0x00000478 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_1_HI 0x00000479 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_2_LO 0x0000047a + +#define REG_A6XX_RBBM_PERFCTR_UCHE_2_HI 0x0000047b + +#define REG_A6XX_RBBM_PERFCTR_UCHE_3_LO 0x0000047c + +#define REG_A6XX_RBBM_PERFCTR_UCHE_3_HI 0x0000047d + +#define REG_A6XX_RBBM_PERFCTR_UCHE_4_LO 0x0000047e + +#define REG_A6XX_RBBM_PERFCTR_UCHE_4_HI 0x0000047f + +#define REG_A6XX_RBBM_PERFCTR_UCHE_5_LO 0x00000480 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_5_HI 0x00000481 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_6_LO 0x00000482 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_6_HI 0x00000483 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_7_LO 0x00000484 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_7_HI 0x00000485 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_8_LO 0x00000486 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_8_HI 0x00000487 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_9_LO 0x00000488 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_9_HI 0x00000489 + +#define REG_A6XX_RBBM_PERFCTR_UCHE_10_LO 0x0000048a + +#define REG_A6XX_RBBM_PERFCTR_UCHE_10_HI 0x0000048b + +#define REG_A6XX_RBBM_PERFCTR_UCHE_11_LO 0x0000048c + +#define REG_A6XX_RBBM_PERFCTR_UCHE_11_HI 0x0000048d + +#define REG_A6XX_RBBM_PERFCTR_TP_0_LO 0x0000048e + +#define REG_A6XX_RBBM_PERFCTR_TP_0_HI 0x0000048f + +#define REG_A6XX_RBBM_PERFCTR_TP_1_LO 0x00000490 + +#define REG_A6XX_RBBM_PERFCTR_TP_1_HI 0x00000491 + +#define REG_A6XX_RBBM_PERFCTR_TP_2_LO 0x00000492 + +#define REG_A6XX_RBBM_PERFCTR_TP_2_HI 0x00000493 + +#define REG_A6XX_RBBM_PERFCTR_TP_3_LO 0x00000494 + +#define REG_A6XX_RBBM_PERFCTR_TP_3_HI 0x00000495 + +#define REG_A6XX_RBBM_PERFCTR_TP_4_LO 0x00000496 + +#define REG_A6XX_RBBM_PERFCTR_TP_4_HI 0x00000497 + +#define REG_A6XX_RBBM_PERFCTR_TP_5_LO 0x00000498 + +#define REG_A6XX_RBBM_PERFCTR_TP_5_HI 0x00000499 + +#define REG_A6XX_RBBM_PERFCTR_TP_6_LO 0x0000049a + +#define REG_A6XX_RBBM_PERFCTR_TP_6_HI 0x0000049b + +#define REG_A6XX_RBBM_PERFCTR_TP_7_LO 0x0000049c + +#define REG_A6XX_RBBM_PERFCTR_TP_7_HI 0x0000049d + +#define REG_A6XX_RBBM_PERFCTR_TP_8_LO 0x0000049e + +#define REG_A6XX_RBBM_PERFCTR_TP_8_HI 0x0000049f + +#define REG_A6XX_RBBM_PERFCTR_TP_9_LO 0x000004a0 + +#define REG_A6XX_RBBM_PERFCTR_TP_9_HI 0x000004a1 + +#define REG_A6XX_RBBM_PERFCTR_TP_10_LO 0x000004a2 + +#define REG_A6XX_RBBM_PERFCTR_TP_10_HI 0x000004a3 + +#define REG_A6XX_RBBM_PERFCTR_TP_11_LO 0x000004a4 + +#define REG_A6XX_RBBM_PERFCTR_TP_11_HI 0x000004a5 + +#define REG_A6XX_RBBM_PERFCTR_SP_0_LO 0x000004a6 + +#define REG_A6XX_RBBM_PERFCTR_SP_0_HI 0x000004a7 + +#define REG_A6XX_RBBM_PERFCTR_SP_1_LO 0x000004a8 + +#define REG_A6XX_RBBM_PERFCTR_SP_1_HI 0x000004a9 + +#define REG_A6XX_RBBM_PERFCTR_SP_2_LO 0x000004aa + +#define REG_A6XX_RBBM_PERFCTR_SP_2_HI 0x000004ab + +#define REG_A6XX_RBBM_PERFCTR_SP_3_LO 0x000004ac + +#define REG_A6XX_RBBM_PERFCTR_SP_3_HI 0x000004ad + +#define REG_A6XX_RBBM_PERFCTR_SP_4_LO 0x000004ae + +#define REG_A6XX_RBBM_PERFCTR_SP_4_HI 0x000004af + +#define REG_A6XX_RBBM_PERFCTR_SP_5_LO 0x000004b0 + +#define REG_A6XX_RBBM_PERFCTR_SP_5_HI 0x000004b1 + +#define REG_A6XX_RBBM_PERFCTR_SP_6_LO 0x000004b2 + +#define REG_A6XX_RBBM_PERFCTR_SP_6_HI 0x000004b3 + +#define REG_A6XX_RBBM_PERFCTR_SP_7_LO 0x000004b4 + +#define REG_A6XX_RBBM_PERFCTR_SP_7_HI 0x000004b5 + +#define REG_A6XX_RBBM_PERFCTR_SP_8_LO 0x000004b6 + +#define REG_A6XX_RBBM_PERFCTR_SP_8_HI 0x000004b7 + +#define REG_A6XX_RBBM_PERFCTR_SP_9_LO 0x000004b8 + +#define REG_A6XX_RBBM_PERFCTR_SP_9_HI 0x000004b9 + +#define REG_A6XX_RBBM_PERFCTR_SP_10_LO 0x000004ba + +#define REG_A6XX_RBBM_PERFCTR_SP_10_HI 0x000004bb + +#define REG_A6XX_RBBM_PERFCTR_SP_11_LO 0x000004bc + +#define REG_A6XX_RBBM_PERFCTR_SP_11_HI 0x000004bd + +#define REG_A6XX_RBBM_PERFCTR_SP_12_LO 0x000004be + +#define REG_A6XX_RBBM_PERFCTR_SP_12_HI 0x000004bf + +#define REG_A6XX_RBBM_PERFCTR_SP_13_LO 0x000004c0 + +#define REG_A6XX_RBBM_PERFCTR_SP_13_HI 0x000004c1 + +#define REG_A6XX_RBBM_PERFCTR_SP_14_LO 0x000004c2 + +#define REG_A6XX_RBBM_PERFCTR_SP_14_HI 0x000004c3 + +#define REG_A6XX_RBBM_PERFCTR_SP_15_LO 0x000004c4 + +#define REG_A6XX_RBBM_PERFCTR_SP_15_HI 0x000004c5 + +#define REG_A6XX_RBBM_PERFCTR_SP_16_LO 0x000004c6 + +#define REG_A6XX_RBBM_PERFCTR_SP_16_HI 0x000004c7 + +#define REG_A6XX_RBBM_PERFCTR_SP_17_LO 0x000004c8 + +#define REG_A6XX_RBBM_PERFCTR_SP_17_HI 0x000004c9 + +#define REG_A6XX_RBBM_PERFCTR_SP_18_LO 0x000004ca + +#define REG_A6XX_RBBM_PERFCTR_SP_18_HI 0x000004cb + +#define REG_A6XX_RBBM_PERFCTR_SP_19_LO 0x000004cc + +#define REG_A6XX_RBBM_PERFCTR_SP_19_HI 0x000004cd + +#define REG_A6XX_RBBM_PERFCTR_SP_20_LO 0x000004ce + +#define REG_A6XX_RBBM_PERFCTR_SP_20_HI 0x000004cf + +#define REG_A6XX_RBBM_PERFCTR_SP_21_LO 0x000004d0 + +#define REG_A6XX_RBBM_PERFCTR_SP_21_HI 0x000004d1 + +#define REG_A6XX_RBBM_PERFCTR_SP_22_LO 0x000004d2 + +#define REG_A6XX_RBBM_PERFCTR_SP_22_HI 0x000004d3 + +#define REG_A6XX_RBBM_PERFCTR_SP_23_LO 0x000004d4 + +#define REG_A6XX_RBBM_PERFCTR_SP_23_HI 0x000004d5 + +#define REG_A6XX_RBBM_PERFCTR_RB_0_LO 0x000004d6 + +#define REG_A6XX_RBBM_PERFCTR_RB_0_HI 0x000004d7 + +#define REG_A6XX_RBBM_PERFCTR_RB_1_LO 0x000004d8 + +#define REG_A6XX_RBBM_PERFCTR_RB_1_HI 0x000004d9 + +#define REG_A6XX_RBBM_PERFCTR_RB_2_LO 0x000004da + +#define REG_A6XX_RBBM_PERFCTR_RB_2_HI 0x000004db + +#define REG_A6XX_RBBM_PERFCTR_RB_3_LO 0x000004dc + +#define REG_A6XX_RBBM_PERFCTR_RB_3_HI 0x000004dd + +#define REG_A6XX_RBBM_PERFCTR_RB_4_LO 0x000004de + +#define REG_A6XX_RBBM_PERFCTR_RB_4_HI 0x000004df + +#define REG_A6XX_RBBM_PERFCTR_RB_5_LO 0x000004e0 + +#define REG_A6XX_RBBM_PERFCTR_RB_5_HI 0x000004e1 + +#define REG_A6XX_RBBM_PERFCTR_RB_6_LO 0x000004e2 + +#define REG_A6XX_RBBM_PERFCTR_RB_6_HI 0x000004e3 + +#define REG_A6XX_RBBM_PERFCTR_RB_7_LO 0x000004e4 + +#define REG_A6XX_RBBM_PERFCTR_RB_7_HI 0x000004e5 + +#define REG_A6XX_RBBM_PERFCTR_VSC_0_LO 0x000004e6 + +#define REG_A6XX_RBBM_PERFCTR_VSC_0_HI 0x000004e7 + +#define REG_A6XX_RBBM_PERFCTR_VSC_1_LO 0x000004e8 + +#define REG_A6XX_RBBM_PERFCTR_VSC_1_HI 0x000004e9 + +#define REG_A6XX_RBBM_PERFCTR_LRZ_0_LO 0x000004ea + +#define REG_A6XX_RBBM_PERFCTR_LRZ_0_HI 0x000004eb + +#define REG_A6XX_RBBM_PERFCTR_LRZ_1_LO 0x000004ec + +#define REG_A6XX_RBBM_PERFCTR_LRZ_1_HI 0x000004ed + +#define REG_A6XX_RBBM_PERFCTR_LRZ_2_LO 0x000004ee + +#define REG_A6XX_RBBM_PERFCTR_LRZ_2_HI 0x000004ef + +#define REG_A6XX_RBBM_PERFCTR_LRZ_3_LO 0x000004f0 + +#define REG_A6XX_RBBM_PERFCTR_LRZ_3_HI 0x000004f1 + +#define REG_A6XX_RBBM_PERFCTR_CMP_0_LO 0x000004f2 + +#define REG_A6XX_RBBM_PERFCTR_CMP_0_HI 0x000004f3 + +#define REG_A6XX_RBBM_PERFCTR_CMP_1_LO 0x000004f4 + +#define REG_A6XX_RBBM_PERFCTR_CMP_1_HI 0x000004f5 + +#define REG_A6XX_RBBM_PERFCTR_CMP_2_LO 0x000004f6 + +#define REG_A6XX_RBBM_PERFCTR_CMP_2_HI 0x000004f7 + +#define REG_A6XX_RBBM_PERFCTR_CMP_3_LO 0x000004f8 + +#define REG_A6XX_RBBM_PERFCTR_CMP_3_HI 0x000004f9 + +#define REG_A6XX_RBBM_PERFCTR_CNTL 0x00000500 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_CMD0 0x00000501 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_CMD1 0x00000502 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_CMD2 0x00000503 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_CMD3 0x00000504 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_VALUE_LO 0x00000505 + +#define REG_A6XX_RBBM_PERFCTR_LOAD_VALUE_HI 0x00000506 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_SEL_0 0x00000507 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_SEL_1 0x00000508 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_SEL_2 0x00000509 + +#define REG_A6XX_RBBM_PERFCTR_RBBM_SEL_3 0x0000050a + +#define REG_A6XX_RBBM_PERFCTR_GPU_BUSY_MASKED 0x0000050b + +#define REG_A6XX_RBBM_ISDB_CNT 0x00000533 + +#define REG_A6XX_RBBM_SECVID_TRUST_CNTL 0x0000f400 + +#define REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO 0x0000f800 + +#define REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI 0x0000f801 + +#define REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE 0x0000f802 + +#define REG_A6XX_RBBM_SECVID_TSB_CNTL 0x0000f803 + +#define REG_A6XX_RBBM_SECVID_TSB_ADDR_MODE_CNTL 0x0000f810 + +#define REG_A6XX_RBBM_VBIF_CLIENT_QOS_CNTL 0x00000010 + +#define REG_A6XX_RBBM_INTERFACE_HANG_INT_CNTL 0x0000001f + +#define REG_A6XX_RBBM_INT_CLEAR_CMD 0x00000037 + +#define REG_A6XX_RBBM_INT_0_MASK 0x00000038 + +#define REG_A6XX_RBBM_SP_HYST_CNT 0x00000042 + +#define REG_A6XX_RBBM_SW_RESET_CMD 0x00000043 + +#define REG_A6XX_RBBM_RAC_THRESHOLD_CNT 0x00000044 + +#define REG_A6XX_RBBM_BLOCK_SW_RESET_CMD 0x00000045 + +#define REG_A6XX_RBBM_BLOCK_SW_RESET_CMD2 0x00000046 + +#define REG_A6XX_RBBM_CLOCK_CNTL 0x000000ae + +#define REG_A6XX_RBBM_CLOCK_CNTL_SP0 0x000000b0 + +#define REG_A6XX_RBBM_CLOCK_CNTL_SP1 0x000000b1 + +#define REG_A6XX_RBBM_CLOCK_CNTL_SP2 0x000000b2 + +#define REG_A6XX_RBBM_CLOCK_CNTL_SP3 0x000000b3 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_SP0 0x000000b4 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_SP1 0x000000b5 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_SP2 0x000000b6 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_SP3 0x000000b7 + +#define REG_A6XX_RBBM_CLOCK_DELAY_SP0 0x000000b8 + +#define REG_A6XX_RBBM_CLOCK_DELAY_SP1 0x000000b9 + +#define REG_A6XX_RBBM_CLOCK_DELAY_SP2 0x000000ba + +#define REG_A6XX_RBBM_CLOCK_DELAY_SP3 0x000000bb + +#define REG_A6XX_RBBM_CLOCK_HYST_SP0 0x000000bc + +#define REG_A6XX_RBBM_CLOCK_HYST_SP1 0x000000bd + +#define REG_A6XX_RBBM_CLOCK_HYST_SP2 0x000000be + +#define REG_A6XX_RBBM_CLOCK_HYST_SP3 0x000000bf + +#define REG_A6XX_RBBM_CLOCK_CNTL_TP0 0x000000c0 + +#define REG_A6XX_RBBM_CLOCK_CNTL_TP1 0x000000c1 + +#define REG_A6XX_RBBM_CLOCK_CNTL_TP2 0x000000c2 + +#define REG_A6XX_RBBM_CLOCK_CNTL_TP3 0x000000c3 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_TP0 0x000000c4 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_TP1 0x000000c5 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_TP2 0x000000c6 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_TP3 0x000000c7 + +#define REG_A6XX_RBBM_CLOCK_CNTL3_TP0 0x000000c8 + +#define REG_A6XX_RBBM_CLOCK_CNTL3_TP1 0x000000c9 + +#define REG_A6XX_RBBM_CLOCK_CNTL3_TP2 0x000000ca + +#define REG_A6XX_RBBM_CLOCK_CNTL3_TP3 0x000000cb + +#define REG_A6XX_RBBM_CLOCK_CNTL4_TP0 0x000000cc + +#define REG_A6XX_RBBM_CLOCK_CNTL4_TP1 0x000000cd + +#define REG_A6XX_RBBM_CLOCK_CNTL4_TP2 0x000000ce + +#define REG_A6XX_RBBM_CLOCK_CNTL4_TP3 0x000000cf + +#define REG_A6XX_RBBM_CLOCK_DELAY_TP0 0x000000d0 + +#define REG_A6XX_RBBM_CLOCK_DELAY_TP1 0x000000d1 + +#define REG_A6XX_RBBM_CLOCK_DELAY_TP2 0x000000d2 + +#define REG_A6XX_RBBM_CLOCK_DELAY_TP3 0x000000d3 + +#define REG_A6XX_RBBM_CLOCK_DELAY2_TP0 0x000000d4 + +#define REG_A6XX_RBBM_CLOCK_DELAY2_TP1 0x000000d5 + +#define REG_A6XX_RBBM_CLOCK_DELAY2_TP2 0x000000d6 + +#define REG_A6XX_RBBM_CLOCK_DELAY2_TP3 0x000000d7 + +#define REG_A6XX_RBBM_CLOCK_DELAY3_TP0 0x000000d8 + +#define REG_A6XX_RBBM_CLOCK_DELAY3_TP1 0x000000d9 + +#define REG_A6XX_RBBM_CLOCK_DELAY3_TP2 0x000000da + +#define REG_A6XX_RBBM_CLOCK_DELAY3_TP3 0x000000db + +#define REG_A6XX_RBBM_CLOCK_DELAY4_TP0 0x000000dc + +#define REG_A6XX_RBBM_CLOCK_DELAY4_TP1 0x000000dd + +#define REG_A6XX_RBBM_CLOCK_DELAY4_TP2 0x000000de + +#define REG_A6XX_RBBM_CLOCK_DELAY4_TP3 0x000000df + +#define REG_A6XX_RBBM_CLOCK_HYST_TP0 0x000000e0 + +#define REG_A6XX_RBBM_CLOCK_HYST_TP1 0x000000e1 + +#define REG_A6XX_RBBM_CLOCK_HYST_TP2 0x000000e2 + +#define REG_A6XX_RBBM_CLOCK_HYST_TP3 0x000000e3 + +#define REG_A6XX_RBBM_CLOCK_HYST2_TP0 0x000000e4 + +#define REG_A6XX_RBBM_CLOCK_HYST2_TP1 0x000000e5 + +#define REG_A6XX_RBBM_CLOCK_HYST2_TP2 0x000000e6 + +#define REG_A6XX_RBBM_CLOCK_HYST2_TP3 0x000000e7 + +#define REG_A6XX_RBBM_CLOCK_HYST3_TP0 0x000000e8 + +#define REG_A6XX_RBBM_CLOCK_HYST3_TP1 0x000000e9 + +#define REG_A6XX_RBBM_CLOCK_HYST3_TP2 0x000000ea + +#define REG_A6XX_RBBM_CLOCK_HYST3_TP3 0x000000eb + +#define REG_A6XX_RBBM_CLOCK_HYST4_TP0 0x000000ec + +#define REG_A6XX_RBBM_CLOCK_HYST4_TP1 0x000000ed + +#define REG_A6XX_RBBM_CLOCK_HYST4_TP2 0x000000ee + +#define REG_A6XX_RBBM_CLOCK_HYST4_TP3 0x000000ef + +#define REG_A6XX_RBBM_CLOCK_CNTL_RB0 0x000000f0 + +#define REG_A6XX_RBBM_CLOCK_CNTL_RB1 0x000000f1 + +#define REG_A6XX_RBBM_CLOCK_CNTL_RB2 0x000000f2 + +#define REG_A6XX_RBBM_CLOCK_CNTL_RB3 0x000000f3 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_RB0 0x000000f4 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_RB1 0x000000f5 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_RB2 0x000000f6 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_RB3 0x000000f7 + +#define REG_A6XX_RBBM_CLOCK_CNTL_CCU0 0x000000f8 + +#define REG_A6XX_RBBM_CLOCK_CNTL_CCU1 0x000000f9 + +#define REG_A6XX_RBBM_CLOCK_CNTL_CCU2 0x000000fa + +#define REG_A6XX_RBBM_CLOCK_CNTL_CCU3 0x000000fb + +#define REG_A6XX_RBBM_CLOCK_HYST_RB_CCU0 0x00000100 + +#define REG_A6XX_RBBM_CLOCK_HYST_RB_CCU1 0x00000101 + +#define REG_A6XX_RBBM_CLOCK_HYST_RB_CCU2 0x00000102 + +#define REG_A6XX_RBBM_CLOCK_HYST_RB_CCU3 0x00000103 + +#define REG_A6XX_RBBM_CLOCK_CNTL_RAC 0x00000104 + +#define REG_A6XX_RBBM_CLOCK_CNTL2_RAC 0x00000105 + +#define REG_A6XX_RBBM_CLOCK_DELAY_RAC 0x00000106 + +#define REG_A6XX_RBBM_CLOCK_HYST_RAC 0x00000107 + +#define REG_A6XX_RBBM_CLOCK_CNTL_TSE_RAS_RBBM 0x00000108 + +#define REG_A6XX_RBBM_CLOCK_DELAY_TSE_RAS_RBBM 0x00000109 + +#define REG_A6XX_RBBM_CLOCK_HYST_TSE_RAS_RBBM 0x0000010a + +#define REG_A6XX_RBBM_CLOCK_CNTL_UCHE 0x0000010b + +#define REG_A6XX_RBBM_CLOCK_CNTL2_UCHE 0x0000010c + +#define REG_A6XX_RBBM_CLOCK_CNTL3_UCHE 0x0000010d + +#define REG_A6XX_RBBM_CLOCK_CNTL4_UCHE 0x0000010e + +#define REG_A6XX_RBBM_CLOCK_DELAY_UCHE 0x0000010f + +#define REG_A6XX_RBBM_CLOCK_HYST_UCHE 0x00000110 + +#define REG_A6XX_RBBM_CLOCK_MODE_VFD 0x00000111 + +#define REG_A6XX_RBBM_CLOCK_DELAY_VFD 0x00000112 + +#define REG_A6XX_RBBM_CLOCK_HYST_VFD 0x00000113 + +#define REG_A6XX_RBBM_CLOCK_MODE_GPC 0x00000114 + +#define REG_A6XX_RBBM_CLOCK_DELAY_GPC 0x00000115 + +#define REG_A6XX_RBBM_CLOCK_HYST_GPC 0x00000116 + +#define REG_A6XX_RBBM_CLOCK_DELAY_HLSQ_2 0x00000117 + +#define REG_A6XX_RBBM_CLOCK_CNTL_GMU_GX 0x00000118 + +#define REG_A6XX_RBBM_CLOCK_DELAY_GMU_GX 0x00000119 + +#define REG_A6XX_RBBM_CLOCK_HYST_GMU_GX 0x0000011a + +#define REG_A6XX_RBBM_CLOCK_MODE_HLSQ 0x0000011b + +#define REG_A6XX_RBBM_CLOCK_DELAY_HLSQ 0x0000011c + +#define REG_A6XX_DBGC_CFG_DBGBUS_SEL_A 0x00000600 + +#define REG_A6XX_DBGC_CFG_DBGBUS_SEL_B 0x00000601 + +#define REG_A6XX_DBGC_CFG_DBGBUS_SEL_C 0x00000602 + +#define REG_A6XX_DBGC_CFG_DBGBUS_SEL_D 0x00000603 +#define A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK 0x000000ff +#define A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT 0 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT) & A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK 0x0000ff00 +#define A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT 8 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT) & A6XX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK; +} + +#define REG_A6XX_DBGC_CFG_DBGBUS_CNTLT 0x00000604 +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK 0x0000003f +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT 0 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT) & A6XX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK 0x00007000 +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT 12 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_CNTLT_GRANU(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT) & A6XX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK 0xf0000000 +#define A6XX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT 28 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_CNTLT_SEGT(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT) & A6XX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK; +} + +#define REG_A6XX_DBGC_CFG_DBGBUS_CNTLM 0x00000605 +#define A6XX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK 0x0f000000 +#define A6XX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT 24 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_CNTLM_ENABLE(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT) & A6XX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK; +} + +#define REG_A6XX_DBGC_CFG_DBGBUS_IVTL_0 0x00000608 + +#define REG_A6XX_DBGC_CFG_DBGBUS_IVTL_1 0x00000609 + +#define REG_A6XX_DBGC_CFG_DBGBUS_IVTL_2 0x0000060a + +#define REG_A6XX_DBGC_CFG_DBGBUS_IVTL_3 0x0000060b + +#define REG_A6XX_DBGC_CFG_DBGBUS_MASKL_0 0x0000060c + +#define REG_A6XX_DBGC_CFG_DBGBUS_MASKL_1 0x0000060d + +#define REG_A6XX_DBGC_CFG_DBGBUS_MASKL_2 0x0000060e + +#define REG_A6XX_DBGC_CFG_DBGBUS_MASKL_3 0x0000060f + +#define REG_A6XX_DBGC_CFG_DBGBUS_BYTEL_0 0x00000610 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK 0x0000000f +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT 0 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK 0x000000f0 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT 4 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK 0x00000f00 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT 8 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK 0x0000f000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT 12 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK 0x000f0000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT 16 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK 0x00f00000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT 20 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK 0x0f000000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT 24 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK 0xf0000000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT 28 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK; +} + +#define REG_A6XX_DBGC_CFG_DBGBUS_BYTEL_1 0x00000611 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK 0x0000000f +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT 0 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK 0x000000f0 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT 4 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK 0x00000f00 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT 8 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK 0x0000f000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT 12 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK 0x000f0000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT 16 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK 0x00f00000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT 20 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK 0x0f000000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT 24 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK; +} +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK 0xf0000000 +#define A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT 28 +static inline uint32_t A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15(uint32_t val) +{ + return ((val) << A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT) & A6XX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK; +} + +#define REG_A6XX_DBGC_CFG_DBGBUS_TRACE_BUF1 0x0000062f + +#define REG_A6XX_DBGC_CFG_DBGBUS_TRACE_BUF2 0x00000630 + +#define REG_A6XX_VSC_PERFCTR_VSC_SEL_0 0x00000cd8 + +#define REG_A6XX_VSC_PERFCTR_VSC_SEL_1 0x00000cd9 + +#define REG_A6XX_GRAS_ADDR_MODE_CNTL 0x00008601 + +#define REG_A6XX_GRAS_PERFCTR_TSE_SEL_0 0x00008610 + +#define REG_A6XX_GRAS_PERFCTR_TSE_SEL_1 0x00008611 + +#define REG_A6XX_GRAS_PERFCTR_TSE_SEL_2 0x00008612 + +#define REG_A6XX_GRAS_PERFCTR_TSE_SEL_3 0x00008613 + +#define REG_A6XX_GRAS_PERFCTR_RAS_SEL_0 0x00008614 + +#define REG_A6XX_GRAS_PERFCTR_RAS_SEL_1 0x00008615 + +#define REG_A6XX_GRAS_PERFCTR_RAS_SEL_2 0x00008616 + +#define REG_A6XX_GRAS_PERFCTR_RAS_SEL_3 0x00008617 + +#define REG_A6XX_GRAS_PERFCTR_LRZ_SEL_0 0x00008618 + +#define REG_A6XX_GRAS_PERFCTR_LRZ_SEL_1 0x00008619 + +#define REG_A6XX_GRAS_PERFCTR_LRZ_SEL_2 0x0000861a + +#define REG_A6XX_GRAS_PERFCTR_LRZ_SEL_3 0x0000861b + +#define REG_A6XX_RB_ADDR_MODE_CNTL 0x00008e05 + +#define REG_A6XX_RB_NC_MODE_CNTL 0x00008e08 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_0 0x00008e10 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_1 0x00008e11 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_2 0x00008e12 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_3 0x00008e13 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_4 0x00008e14 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_5 0x00008e15 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_6 0x00008e16 + +#define REG_A6XX_RB_PERFCTR_RB_SEL_7 0x00008e17 + +#define REG_A6XX_RB_PERFCTR_CCU_SEL_0 0x00008e18 + +#define REG_A6XX_RB_PERFCTR_CCU_SEL_1 0x00008e19 + +#define REG_A6XX_RB_PERFCTR_CCU_SEL_2 0x00008e1a + +#define REG_A6XX_RB_PERFCTR_CCU_SEL_3 0x00008e1b + +#define REG_A6XX_RB_PERFCTR_CCU_SEL_4 0x00008e1c + +#define REG_A6XX_RB_PERFCTR_CMP_SEL_0 0x00008e2c + +#define REG_A6XX_RB_PERFCTR_CMP_SEL_1 0x00008e2d + +#define REG_A6XX_RB_PERFCTR_CMP_SEL_2 0x00008e2e + +#define REG_A6XX_RB_PERFCTR_CMP_SEL_3 0x00008e2f + +#define REG_A6XX_RB_RB_SUB_BLOCK_SEL_CNTL_CD 0x00008e3d + +#define REG_A6XX_RB_CONTEXT_SWITCH_GMEM_SAVE_RESTORE 0x00008e50 + +#define REG_A6XX_PC_DBG_ECO_CNTL 0x00009e00 + +#define REG_A6XX_PC_ADDR_MODE_CNTL 0x00009e01 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_0 0x00009e34 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_1 0x00009e35 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_2 0x00009e36 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_3 0x00009e37 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_4 0x00009e38 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_5 0x00009e39 + +#define REG_A6XX_PC_PERFCTR_PC_SEL_6 0x00009e3a + +#define REG_A6XX_PC_PERFCTR_PC_SEL_7 0x00009e3b + +#define REG_A6XX_HLSQ_ADDR_MODE_CNTL 0x0000be05 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_0 0x0000be10 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_1 0x0000be11 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_2 0x0000be12 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_3 0x0000be13 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_4 0x0000be14 + +#define REG_A6XX_HLSQ_PERFCTR_HLSQ_SEL_5 0x0000be15 + +#define REG_A6XX_HLSQ_DBG_AHB_READ_APERTURE 0x0000c800 + +#define REG_A6XX_HLSQ_DBG_READ_SEL 0x0000d000 + +#define REG_A6XX_VFD_ADDR_MODE_CNTL 0x0000a601 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_0 0x0000a610 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_1 0x0000a611 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_2 0x0000a612 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_3 0x0000a613 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_4 0x0000a614 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_5 0x0000a615 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_6 0x0000a616 + +#define REG_A6XX_VFD_PERFCTR_VFD_SEL_7 0x0000a617 + +#define REG_A6XX_VPC_ADDR_MODE_CNTL 0x00009601 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_0 0x00009604 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_1 0x00009605 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_2 0x00009606 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_3 0x00009607 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_4 0x00009608 + +#define REG_A6XX_VPC_PERFCTR_VPC_SEL_5 0x00009609 + +#define REG_A6XX_UCHE_ADDR_MODE_CNTL 0x00000e00 + +#define REG_A6XX_UCHE_MODE_CNTL 0x00000e01 + +#define REG_A6XX_UCHE_WRITE_RANGE_MAX_LO 0x00000e05 + +#define REG_A6XX_UCHE_WRITE_RANGE_MAX_HI 0x00000e06 + +#define REG_A6XX_UCHE_WRITE_THRU_BASE_LO 0x00000e07 + +#define REG_A6XX_UCHE_WRITE_THRU_BASE_HI 0x00000e08 + +#define REG_A6XX_UCHE_TRAP_BASE_LO 0x00000e09 + +#define REG_A6XX_UCHE_TRAP_BASE_HI 0x00000e0a + +#define REG_A6XX_UCHE_GMEM_RANGE_MIN_LO 0x00000e0b + +#define REG_A6XX_UCHE_GMEM_RANGE_MIN_HI 0x00000e0c + +#define REG_A6XX_UCHE_GMEM_RANGE_MAX_LO 0x00000e0d + +#define REG_A6XX_UCHE_GMEM_RANGE_MAX_HI 0x00000e0e + +#define REG_A6XX_UCHE_CACHE_WAYS 0x00000e17 + +#define REG_A6XX_UCHE_FILTER_CNTL 0x00000e18 + +#define REG_A6XX_UCHE_CLIENT_PF 0x00000e19 +#define A6XX_UCHE_CLIENT_PF_PERFSEL__MASK 0x000000ff +#define A6XX_UCHE_CLIENT_PF_PERFSEL__SHIFT 0 +static inline uint32_t A6XX_UCHE_CLIENT_PF_PERFSEL(uint32_t val) +{ + return ((val) << A6XX_UCHE_CLIENT_PF_PERFSEL__SHIFT) & A6XX_UCHE_CLIENT_PF_PERFSEL__MASK; +} + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_0 0x00000e1c + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_1 0x00000e1d + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_2 0x00000e1e + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_3 0x00000e1f + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_4 0x00000e20 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_5 0x00000e21 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_6 0x00000e22 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_7 0x00000e23 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_8 0x00000e24 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_9 0x00000e25 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_10 0x00000e26 + +#define REG_A6XX_UCHE_PERFCTR_UCHE_SEL_11 0x00000e27 + +#define REG_A6XX_SP_ADDR_MODE_CNTL 0x0000ae01 + +#define REG_A6XX_SP_NC_MODE_CNTL 0x0000ae02 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_0 0x0000ae10 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_1 0x0000ae11 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_2 0x0000ae12 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_3 0x0000ae13 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_4 0x0000ae14 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_5 0x0000ae15 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_6 0x0000ae16 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_7 0x0000ae17 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_8 0x0000ae18 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_9 0x0000ae19 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_10 0x0000ae1a + +#define REG_A6XX_SP_PERFCTR_SP_SEL_11 0x0000ae1b + +#define REG_A6XX_SP_PERFCTR_SP_SEL_12 0x0000ae1c + +#define REG_A6XX_SP_PERFCTR_SP_SEL_13 0x0000ae1d + +#define REG_A6XX_SP_PERFCTR_SP_SEL_14 0x0000ae1e + +#define REG_A6XX_SP_PERFCTR_SP_SEL_15 0x0000ae1f + +#define REG_A6XX_SP_PERFCTR_SP_SEL_16 0x0000ae20 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_17 0x0000ae21 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_18 0x0000ae22 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_19 0x0000ae23 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_20 0x0000ae24 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_21 0x0000ae25 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_22 0x0000ae26 + +#define REG_A6XX_SP_PERFCTR_SP_SEL_23 0x0000ae27 + +#define REG_A6XX_TPL1_ADDR_MODE_CNTL 0x0000b601 + +#define REG_A6XX_TPL1_NC_MODE_CNTL 0x0000b604 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_0 0x0000b610 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_1 0x0000b611 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_2 0x0000b612 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_3 0x0000b613 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_4 0x0000b614 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_5 0x0000b615 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_6 0x0000b616 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_7 0x0000b617 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_8 0x0000b618 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_9 0x0000b619 + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_10 0x0000b61a + +#define REG_A6XX_TPL1_PERFCTR_TP_SEL_11 0x0000b61b + +#define REG_A6XX_VBIF_VERSION 0x00003000 + +#define REG_A6XX_VBIF_GATE_OFF_WRREQ_EN 0x0000302a + +#define REG_A6XX_VBIF_XIN_HALT_CTRL0 0x00003080 + +#define REG_A6XX_VBIF_XIN_HALT_CTRL1 0x00003081 + +#define REG_A6XX_VBIF_PERF_CNT_SEL0 0x000030d0 + +#define REG_A6XX_VBIF_PERF_CNT_SEL1 0x000030d1 + +#define REG_A6XX_VBIF_PERF_CNT_SEL2 0x000030d2 + +#define REG_A6XX_VBIF_PERF_CNT_SEL3 0x000030d3 + +#define REG_A6XX_VBIF_PERF_CNT_LOW0 0x000030d8 + +#define REG_A6XX_VBIF_PERF_CNT_LOW1 0x000030d9 + +#define REG_A6XX_VBIF_PERF_CNT_LOW2 0x000030da + +#define REG_A6XX_VBIF_PERF_CNT_LOW3 0x000030db + +#define REG_A6XX_VBIF_PERF_CNT_HIGH0 0x000030e0 + +#define REG_A6XX_VBIF_PERF_CNT_HIGH1 0x000030e1 + +#define REG_A6XX_VBIF_PERF_CNT_HIGH2 0x000030e2 + +#define REG_A6XX_VBIF_PERF_CNT_HIGH3 0x000030e3 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_EN0 0x00003100 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_EN1 0x00003101 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_EN2 0x00003102 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_LOW0 0x00003110 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_LOW1 0x00003111 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_LOW2 0x00003112 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_HIGH0 0x00003118 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_HIGH1 0x00003119 + +#define REG_A6XX_VBIF_PERF_PWR_CNT_HIGH2 0x0000311a + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_A 0x00018400 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_B 0x00018401 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_C 0x00018402 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_SEL_D 0x00018403 +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK 0x000000ff +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_INDEX__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK 0x0000ff00 +#define A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_SEL_D_PING_BLK_SEL__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLT 0x00018404 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK 0x0000003f +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_TRACEEN__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK 0x00007000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_GRANU__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLT_SEGT__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_CNTLM 0x00018405 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_CNTLM_ENABLE__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_0 0x00018408 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_1 0x00018409 + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_2 0x0001840a + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_IVTL_3 0x0001840b + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_0 0x0001840c + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_1 0x0001840d + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_2 0x0001840e + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_MASKL_3 0x0001840f + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0 0x00018410 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK 0x0000000f +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL0__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK 0x000000f0 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT 4 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL1__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK 0x00000f00 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL2__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK 0x0000f000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL3__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK 0x000f0000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT 16 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL4__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK 0x00f00000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT 20 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL5__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL6__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_0_BYTEL7__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1 0x00018411 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK 0x0000000f +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT 0 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL8__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK 0x000000f0 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT 4 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL9__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK 0x00000f00 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT 8 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL10__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK 0x0000f000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT 12 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL11__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK 0x000f0000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT 16 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL12__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK 0x00f00000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT 20 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL13__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK 0x0f000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT 24 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL14__MASK; +} +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK 0xf0000000 +#define A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT 28 +static inline uint32_t A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15(uint32_t val) +{ + return ((val) << A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__SHIFT) & A6XX_CX_DBGC_CFG_DBGBUS_BYTEL_1_BYTEL15__MASK; +} + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF1 0x0001842f + +#define REG_A6XX_CX_DBGC_CFG_DBGBUS_TRACE_BUF2 0x00018430 + +#define REG_A6XX_PDC_GPU_ENABLE_PDC 0x00021140 + +#define REG_A6XX_PDC_GPU_SEQ_START_ADDR 0x00021148 + +#define REG_A6XX_PDC_GPU_TCS0_CONTROL 0x00021540 + +#define REG_A6XX_PDC_GPU_TCS0_CMD_ENABLE_BANK 0x00021541 + +#define REG_A6XX_PDC_GPU_TCS0_CMD_WAIT_FOR_CMPL_BANK 0x00021542 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_MSGID 0x00021543 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_ADDR 0x00021544 + +#define REG_A6XX_PDC_GPU_TCS0_CMD0_DATA 0x00021545 + +#define REG_A6XX_PDC_GPU_TCS1_CONTROL 0x00021572 + +#define REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK 0x00021573 + +#define REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK 0x00021574 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID 0x00021575 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR 0x00021576 + +#define REG_A6XX_PDC_GPU_TCS1_CMD0_DATA 0x00021577 + +#define REG_A6XX_PDC_GPU_TCS2_CONTROL 0x000215a4 + +#define REG_A6XX_PDC_GPU_TCS2_CMD_ENABLE_BANK 0x000215a5 + +#define REG_A6XX_PDC_GPU_TCS2_CMD_WAIT_FOR_CMPL_BANK 0x000215a6 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_MSGID 0x000215a7 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_ADDR 0x000215a8 + +#define REG_A6XX_PDC_GPU_TCS2_CMD0_DATA 0x000215a9 + +#define REG_A6XX_PDC_GPU_TCS3_CONTROL 0x000215d6 + +#define REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK 0x000215d7 + +#define REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK 0x000215d8 + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID 0x000215d9 + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR 0x000215da + +#define REG_A6XX_PDC_GPU_TCS3_CMD0_DATA 0x000215db + +#define REG_A6XX_PDC_GPU_SEQ_MEM_0 0x000a0000 + + +#endif /* A6XX_XML */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h new file mode 100644 index 000000000000..ff0b1064f332 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h @@ -0,0 +1,382 @@ +#ifndef A6XX_GMU_XML +#define A6XX_GMU_XML + +/* Autogenerated file, DO NOT EDIT manually! + +This file was generated by the rules-ng-ng headergen tool in this git repository: +http://github.com/freedreno/envytools/ +git clone https://github.com/freedreno/envytools.git + +The rules-ng-ng source files this header was generated from are: +- /local3/projects/drm/envytools/rnndb//adreno.xml ( 501 bytes, from 2017-11-20 17:36:01) +- /local3/projects/drm/envytools/rnndb//freedreno_copyright.xml ( 1572 bytes, from 2016-10-24 21:12:27) +- /local3/projects/drm/envytools/rnndb//adreno/a2xx.xml ( 32901 bytes, from 2016-10-24 21:12:27) +- /local3/projects/drm/envytools/rnndb//adreno/adreno_common.xml ( 11792 bytes, from 2017-11-17 23:22:22) +- /local3/projects/drm/envytools/rnndb//adreno/adreno_pm4.xml ( 17205 bytes, from 2017-11-17 23:22:22) +- /local3/projects/drm/envytools/rnndb//adreno/a3xx.xml ( 83693 bytes, from 2017-11-17 23:22:22) +- /local3/projects/drm/envytools/rnndb//adreno/a4xx.xml ( 110372 bytes, from 2017-11-17 23:22:22) +- /local3/projects/drm/envytools/rnndb//adreno/a5xx.xml ( 66793 bytes, from 2017-11-17 23:22:22) +- /local3/projects/drm/envytools/rnndb//adreno/a6xx.xml ( 44551 bytes, from 2017-11-20 19:30:19) +- /local3/projects/drm/envytools/rnndb//adreno/a6xx_gmu.xml ( 10431 bytes, from 2017-11-20 17:59:44) +- /local3/projects/drm/envytools/rnndb//adreno/ocmem.xml ( 1773 bytes, from 2016-10-24 21:12:27) + +Copyright (C) 2013-2017 by the following authors: +- Rob Clark <robdclark@gmail.com> (robclark) +- Ilia Mirkin <imirkin@alum.mit.edu> (imirkin) + +Permission is hereby granted, free of charge, to any person obtaining +a copy of this software and associated documentation files (the +"Software"), to deal in the Software without restriction, including +without limitation the rights to use, copy, modify, merge, publish, +distribute, sublicense, and/or sell copies of the Software, and to +permit persons to whom the Software is furnished to do so, subject to +the following conditions: + +The above copyright notice and this permission notice (including the +next paragraph) shall be included in all copies or substantial +portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, +EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF +MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. +IN NO EVENT SHALL THE COPYRIGHT OWNER(S) AND/OR ITS SUPPLIERS BE +LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION +OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION +WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. +*/ + + +#define A6XX_GMU_GPU_IDLE_STATUS_BUSY_IGN_AHB 0x00800000 +#define A6XX_GMU_GPU_IDLE_STATUS_CX_GX_CPU_BUSY_IGN_AHB 0x40000000 +#define A6XX_GMU_OOB_BOOT_SLUMBER_SET_MASK 0x00400000 +#define A6XX_GMU_OOB_BOOT_SLUMBER_CHECK_MASK 0x40000000 +#define A6XX_GMU_OOB_BOOT_SLUMBER_CLEAR_MASK 0x40000000 +#define A6XX_GMU_OOB_DCVS_SET_MASK 0x00800000 +#define A6XX_GMU_OOB_DCVS_CHECK_MASK 0x80000000 +#define A6XX_GMU_OOB_DCVS_CLEAR_MASK 0x80000000 +#define A6XX_GMU_OOB_GPU_SET_MASK 0x00040000 +#define A6XX_GMU_OOB_GPU_CHECK_MASK 0x04000000 +#define A6XX_GMU_OOB_GPU_CLEAR_MASK 0x04000000 +#define A6XX_GMU_OOB_PERFCNTR_SET_MASK 0x00020000 +#define A6XX_GMU_OOB_PERFCNTR_CHECK_MASK 0x02000000 +#define A6XX_GMU_OOB_PERFCNTR_CLEAR_MASK 0x02000000 +#define A6XX_HFI_IRQ_MSGQ_MASK 0x00000001 +#define A6XX_HFI_IRQ_DSGQ_MASK 0x00000002 +#define A6XX_HFI_IRQ_BLOCKED_MSG_MASK 0x00000004 +#define A6XX_HFI_IRQ_CM3_FAULT_MASK 0x00800000 +#define A6XX_HFI_IRQ_GMU_ERR_MASK__MASK 0x007f0000 +#define A6XX_HFI_IRQ_GMU_ERR_MASK__SHIFT 16 +static inline uint32_t A6XX_HFI_IRQ_GMU_ERR_MASK(uint32_t val) +{ + return ((val) << A6XX_HFI_IRQ_GMU_ERR_MASK__SHIFT) & A6XX_HFI_IRQ_GMU_ERR_MASK__MASK; +} +#define A6XX_HFI_IRQ_OOB_MASK__MASK 0xff000000 +#define A6XX_HFI_IRQ_OOB_MASK__SHIFT 24 +static inline uint32_t A6XX_HFI_IRQ_OOB_MASK(uint32_t val) +{ + return ((val) << A6XX_HFI_IRQ_OOB_MASK__SHIFT) & A6XX_HFI_IRQ_OOB_MASK__MASK; +} +#define A6XX_HFI_H2F_IRQ_MASK_BIT 0x00000001 +#define REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL 0x00000080 + +#define REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL 0x00000081 + +#define REG_A6XX_GMU_CM3_ITCM_START 0x00000c00 + +#define REG_A6XX_GMU_CM3_DTCM_START 0x00001c00 + +#define REG_A6XX_GMU_NMI_CONTROL_STATUS 0x000023f0 + +#define REG_A6XX_GMU_BOOT_SLUMBER_OPTION 0x000023f8 + +#define REG_A6XX_GMU_GX_VOTE_IDX 0x000023f9 + +#define REG_A6XX_GMU_MX_VOTE_IDX 0x000023fa + +#define REG_A6XX_GMU_DCVS_ACK_OPTION 0x000023fc + +#define REG_A6XX_GMU_DCVS_PERF_SETTING 0x000023fd + +#define REG_A6XX_GMU_DCVS_BW_SETTING 0x000023fe + +#define REG_A6XX_GMU_DCVS_RETURN 0x000023ff + +#define REG_A6XX_GMU_SYS_BUS_CONFIG 0x00004c0f + +#define REG_A6XX_GMU_CM3_SYSRESET 0x00005000 + +#define REG_A6XX_GMU_CM3_BOOT_CONFIG 0x00005001 + +#define REG_A6XX_GMU_CM3_FW_BUSY 0x0000501a + +#define REG_A6XX_GMU_CM3_FW_INIT_RESULT 0x0000501c + +#define REG_A6XX_GMU_CM3_CFG 0x0000502d + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE 0x00005040 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_SELECT_0 0x00005041 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_SELECT_1 0x00005042 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_0_L 0x00005044 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_0_H 0x00005045 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_1_L 0x00005046 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_1_H 0x00005047 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_2_L 0x00005048 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_2_H 0x00005049 + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_3_L 0x0000504a + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_3_H 0x0000504b + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_4_L 0x0000504c + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_4_H 0x0000504d + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_5_L 0x0000504e + +#define REG_A6XX_GMU_CX_GMU_POWER_COUNTER_XOCLK_5_H 0x0000504f + +#define REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL 0x000050c0 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE 0x00000001 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_HM_POWER_COLLAPSE_ENABLE 0x00000002 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_SPTPRAC_POWER_CONTROL_ENABLE 0x00000004 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_NUM_PASS_SKIPS__MASK 0x00003c00 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_NUM_PASS_SKIPS__SHIFT 10 +static inline uint32_t A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_NUM_PASS_SKIPS(uint32_t val) +{ + return ((val) << A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_NUM_PASS_SKIPS__SHIFT) & A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_NUM_PASS_SKIPS__MASK; +} +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH__MASK 0xffffc000 +#define A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH__SHIFT 14 +static inline uint32_t A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH(uint32_t val) +{ + return ((val) << A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH__SHIFT) & A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_MIN_PASS_LENGTH__MASK; +} + +#define REG_A6XX_GMU_PWR_COL_INTER_FRAME_HYST 0x000050c1 + +#define REG_A6XX_GMU_PWR_COL_SPTPRAC_HYST 0x000050c2 + +#define REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS 0x000050d0 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWERING_OFF 0x00000001 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWERING_ON 0x00000002 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_ON 0x00000004 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SPTPRAC_GDSC_POWER_OFF 0x00000008 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_SP_CLOCK_OFF 0x00000010 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GMU_UP_POWER_STATE 0x00000020 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_GDSC_POWER_OFF 0x00000040 +#define A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_CLK_OFF 0x00000080 + +#define REG_A6XX_GMU_GPU_NAP_CTRL 0x000050e4 +#define A6XX_GMU_GPU_NAP_CTRL_HW_NAP_ENABLE 0x00000001 +#define A6XX_GMU_GPU_NAP_CTRL_SID__MASK 0x000001f0 +#define A6XX_GMU_GPU_NAP_CTRL_SID__SHIFT 4 +static inline uint32_t A6XX_GMU_GPU_NAP_CTRL_SID(uint32_t val) +{ + return ((val) << A6XX_GMU_GPU_NAP_CTRL_SID__SHIFT) & A6XX_GMU_GPU_NAP_CTRL_SID__MASK; +} + +#define REG_A6XX_GMU_RPMH_CTRL 0x000050e8 +#define A6XX_GMU_RPMH_CTRL_RPMH_INTERFACE_ENABLE 0x00000001 +#define A6XX_GMU_RPMH_CTRL_LLC_VOTE_ENABLE 0x00000010 +#define A6XX_GMU_RPMH_CTRL_DDR_VOTE_ENABLE 0x00000100 +#define A6XX_GMU_RPMH_CTRL_MX_VOTE_ENABLE 0x00000200 +#define A6XX_GMU_RPMH_CTRL_CX_VOTE_ENABLE 0x00000400 +#define A6XX_GMU_RPMH_CTRL_GFX_VOTE_ENABLE 0x00000800 +#define A6XX_GMU_RPMH_CTRL_DDR_MIN_VOTE_ENABLE 0x00001000 +#define A6XX_GMU_RPMH_CTRL_MX_MIN_VOTE_ENABLE 0x00002000 +#define A6XX_GMU_RPMH_CTRL_CX_MIN_VOTE_ENABLE 0x00004000 +#define A6XX_GMU_RPMH_CTRL_GFX_MIN_VOTE_ENABLE 0x00008000 + +#define REG_A6XX_GMU_RPMH_HYST_CTRL 0x000050e9 + +#define REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE 0x000050ec + +#define REG_A6XX_GMU_BOOT_KMD_LM_HANDSHAKE 0x000051f0 + +#define REG_A6XX_GMU_LLM_GLM_SLEEP_CTRL 0x00005157 + +#define REG_A6XX_GMU_LLM_GLM_SLEEP_STATUS 0x00005158 + +#define REG_A6XX_GMU_ALWAYS_ON_COUNTER_L 0x00005088 + +#define REG_A6XX_GMU_ALWAYS_ON_COUNTER_H 0x00005089 + +#define REG_A6XX_GMU_GMU_PWR_COL_KEEPALIVE 0x000050c3 + +#define REG_A6XX_GMU_HFI_CTRL_STATUS 0x00005180 + +#define REG_A6XX_GMU_HFI_VERSION_INFO 0x00005181 + +#define REG_A6XX_GMU_HFI_SFR_ADDR 0x00005182 + +#define REG_A6XX_GMU_HFI_MMAP_ADDR 0x00005183 + +#define REG_A6XX_GMU_HFI_QTBL_INFO 0x00005184 + +#define REG_A6XX_GMU_HFI_QTBL_ADDR 0x00005185 + +#define REG_A6XX_GMU_HFI_CTRL_INIT 0x00005186 + +#define REG_A6XX_GMU_GMU2HOST_INTR_SET 0x00005190 + +#define REG_A6XX_GMU_GMU2HOST_INTR_CLR 0x00005191 + +#define REG_A6XX_GMU_GMU2HOST_INTR_INFO 0x00005192 +#define A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ 0x00000001 +#define A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT 0x00800000 + +#define REG_A6XX_GMU_GMU2HOST_INTR_MASK 0x00005193 + +#define REG_A6XX_GMU_HOST2GMU_INTR_SET 0x00005194 + +#define REG_A6XX_GMU_HOST2GMU_INTR_CLR 0x00005195 + +#define REG_A6XX_GMU_HOST2GMU_INTR_RAW_INFO 0x00005196 + +#define REG_A6XX_GMU_HOST2GMU_INTR_EN_0 0x00005197 + +#define REG_A6XX_GMU_HOST2GMU_INTR_EN_1 0x00005198 + +#define REG_A6XX_GMU_HOST2GMU_INTR_EN_2 0x00005199 + +#define REG_A6XX_GMU_HOST2GMU_INTR_EN_3 0x0000519a + +#define REG_A6XX_GMU_HOST2GMU_INTR_INFO_0 0x0000519b + +#define REG_A6XX_GMU_HOST2GMU_INTR_INFO_1 0x0000519c + +#define REG_A6XX_GMU_HOST2GMU_INTR_INFO_2 0x0000519d + +#define REG_A6XX_GMU_HOST2GMU_INTR_INFO_3 0x0000519e + +#define REG_A6XX_GMU_GENERAL_1 0x000051c6 + +#define REG_A6XX_GMU_GENERAL_7 0x000051cc + +#define REG_A6XX_GMU_ISENSE_CTRL 0x0000515d + +#define REG_A6XX_GPU_CS_ENABLE_REG 0x00008920 + +#define REG_A6XX_GPU_GMU_CX_GMU_ISENSE_CTRL 0x0000515d + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_CONTROL3 0x00008578 + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_CONTROL2 0x00008558 + +#define REG_A6XX_GPU_CS_A_SENSOR_CTRL_0 0x00008580 + +#define REG_A6XX_GPU_CS_A_SENSOR_CTRL_2 0x00027ada + +#define REG_A6XX_GPU_CS_SENSOR_GENERAL_STATUS 0x0000881a + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_CONTROL1 0x00008957 + +#define REG_A6XX_GPU_CS_SENSOR_GENERAL_STATUS 0x0000881a + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_STATUS1_0 0x0000881d + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_STATUS1_2 0x0000881f + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_STATUS1_4 0x00008821 + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_DONE 0x00008965 + +#define REG_A6XX_GPU_CS_AMP_PERIOD_CTRL 0x0000896d + +#define REG_A6XX_GPU_CS_AMP_CALIBRATION_DONE 0x00008965 + +#define REG_A6XX_GPU_GMU_CX_GMU_PWR_THRESHOLD 0x0000514d + +#define REG_A6XX_GMU_AO_INTERRUPT_EN 0x00009303 + +#define REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR 0x00009304 + +#define REG_A6XX_GMU_AO_HOST_INTERRUPT_STATUS 0x00009305 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE 0x00000001 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_RSCC_COMP 0x00000002 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_VDROOP 0x00000004 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR 0x00000008 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_DBD_WAKEUP 0x00000010 +#define A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR 0x00000020 + +#define REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK 0x00009306 + +#define REG_A6XX_GPU_GMU_AO_GMU_CGC_MODE_CNTL 0x00009309 + +#define REG_A6XX_GPU_GMU_AO_GMU_CGC_DELAY_CNTL 0x0000930a + +#define REG_A6XX_GPU_GMU_AO_GMU_CGC_HYST_CNTL 0x0000930b + +#define REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS 0x0000930c +#define A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB 0x00800000 + +#define REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS2 0x0000930d + +#define REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_MASK 0x0000930e + +#define REG_A6XX_GMU_AO_AHB_FENCE_CTRL 0x00009310 + +#define REG_A6XX_GMU_AHB_FENCE_STATUS 0x00009313 + +#define REG_A6XX_GMU_RBBM_INT_UNMASKED_STATUS 0x00009315 + +#define REG_A6XX_GMU_AO_SPARE_CNTL 0x00009316 + +#define REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0 0x00008c04 + +#define REG_A6XX_GMU_RSCC_CONTROL_REQ 0x00009307 + +#define REG_A6XX_GMU_RSCC_CONTROL_ACK 0x00009308 + +#define REG_A6XX_GMU_AHB_FENCE_RANGE_0 0x00009311 + +#define REG_A6XX_GMU_AHB_FENCE_RANGE_1 0x00009312 + +#define REG_A6XX_GPU_CC_GX_GDSCR 0x00009c03 + +#define REG_A6XX_GPU_CC_GX_DOMAIN_MISC 0x00009d42 + +#define REG_A6XX_RSCC_PDC_SEQ_START_ADDR 0x00008c08 + +#define REG_A6XX_RSCC_PDC_MATCH_VALUE_LO 0x00008c09 + +#define REG_A6XX_RSCC_PDC_MATCH_VALUE_HI 0x00008c0a + +#define REG_A6XX_RSCC_PDC_SLAVE_ID_DRV0 0x00008c0b + +#define REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR 0x00008c0d + +#define REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA 0x00008c0e + +#define REG_A6XX_RSCC_TIMESTAMP_UNIT0_TIMESTAMP_L_DRV0 0x00008c82 + +#define REG_A6XX_RSCC_TIMESTAMP_UNIT0_TIMESTAMP_H_DRV0 0x00008c83 + +#define REG_A6XX_RSCC_TIMESTAMP_UNIT1_EN_DRV0 0x00008c89 + +#define REG_A6XX_RSCC_TIMESTAMP_UNIT1_OUTPUT_DRV0 0x00008c8c + +#define REG_A6XX_RSCC_OVERRIDE_START_ADDR 0x00008d00 + +#define REG_A6XX_RSCC_SEQ_BUSY_DRV0 0x00008d01 + +#define REG_A6XX_RSCC_SEQ_MEM_0_DRV0 0x00008d80 + +#define REG_A6XX_RSCC_TCS0_DRV0_STATUS 0x00008f46 + +#define REG_A6XX_RSCC_TCS1_DRV0_STATUS 0x000090ae + +#define REG_A6XX_RSCC_TCS2_DRV0_STATUS 0x00009216 + +#define REG_A6XX_RSCC_TCS3_DRV0_STATUS 0x0000937e + + +#endif /* A6XX_GMU_XML */ -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [PATCH 5/5] drm/msm: Add A6XX device support [not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> ` (3 preceding siblings ...) 2018-06-11 18:26 ` [PATCH 4/5] drm/msm: Add generated headers for A6XX Jordan Crouse @ 2018-06-11 18:26 ` Jordan Crouse 4 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-06-11 18:26 UTC (permalink / raw) To: dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW, freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA Add support for the A6XX family of Adreno GPUs. The biggest addition is the GMU (Graphics Management Unit) which takes over most of the power management of the GPU itself but in a ironic twist of fate needs a goodly amount of management itself. Add support for the A6XX core code, the GMU and the HFI (hardware firmware interface) queue that the CPU uses to communicate with the GMU. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/Makefile | 3 + drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1172 ++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 +++ drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 807 ++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 + drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 ++++++++ drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 +++ drivers/gpu/drm/msm/adreno/adreno_device.c | 12 + drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_gpu.h | 4 + drivers/gpu/drm/msm/msm_gpu.c | 2 +- 11 files changed, 2784 insertions(+), 2 deletions(-) create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index cd40c050b2d7..4affc665c0de 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -10,6 +10,9 @@ msm-y := \ adreno/a5xx_gpu.o \ adreno/a5xx_power.o \ adreno/a5xx_preempt.o \ + adreno/a6xx_gpu.o \ + adreno/a6xx_gmu.o \ + adreno/a6xx_hfi.o \ hdmi/hdmi.o \ hdmi/hdmi_audio.o \ hdmi/hdmi_bridge.o \ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c new file mode 100644 index 000000000000..9ecd7afe6321 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -0,0 +1,1172 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/clk.h> +#include <linux/iopoll.h> +#include <linux/pm_opp.h> +#include <soc/qcom/cmd-db.h> + +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static irqreturn_t a6xx_gmu_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_STATUS); + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, status); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE) { + dev_err_ratelimited(gmu->dev, "GMU watchdog expired\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR) + dev_err_ratelimited(gmu->dev, "GMU AHB bus error\n"); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + dev_err_ratelimited(gmu->dev, "GMU fence error: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_AHB_FENCE_STATUS)); + + return IRQ_HANDLED; +} + +static irqreturn_t a6xx_hfi_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, status); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ) + tasklet_schedule(&gmu->hfi_tasklet); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) { + dev_err_ratelimited(gmu->dev, "GMU firmware fault\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + return IRQ_HANDLED; +} + +/* Check to see if the GX rail is still powered */ +static bool a6xx_gmu_gx_is_on(struct a6xx_gmu *gmu) +{ + u32 val = gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS); + + return !(val & + (A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_GDSC_POWER_OFF | + A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_CLK_OFF)); +} + +static bool a6xx_gmu_check_idle_level(struct a6xx_gmu *gmu) +{ + u32 val; + int local = gmu->idle_level; + + /* SPTP and IFPC both report as IFPC */ + if (gmu->idle_level == GMU_IDLE_STATE_SPTP) + local = GMU_IDLE_STATE_IFPC; + + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val == local) { + if (gmu->idle_level != GMU_IDLE_STATE_IFPC || + !a6xx_gmu_gx_is_on(gmu)) + return true; + } + + return false; +} + +/* Wait for the GMU to get to its most idle state */ +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + return spin_until(a6xx_gmu_check_idle_level(gmu)); +} + +static int a6xx_gmu_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 1); + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 0); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, val, + val == 0xbabeface, 100, 10000); + + if (ret) + dev_err(gmu->dev, "GMU firmware initalization timed out\n"); + + return ret; +} + +static int a6xx_gmu_hfi_start(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + gmu_rmw(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ, 0); + + gmu_write(gmu, REG_A6XX_GMU_HFI_CTRL_INIT, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_HFI_CTRL_STATUS, val, + val & 1, 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to start the HFI queues\n"); + + return ret; +} + +/* Trigger a OOB (out of band) request to the GMU */ +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + int ret; + u32 val; + int request, ack; + const char *name; + + switch (state) { + case GMU_OOB_GPU_SET: + request = GMU_OOB_GPU_SET_REQUEST; + ack = GMU_OOB_GPU_SET_ACK; + name = "GPU_SET"; + break; + case GMU_OOB_BOOT_SLUMBER: + request = GMU_OOB_BOOT_SLUMBER_REQUEST; + ack = GMU_OOB_BOOT_SLUMBER_ACK; + name = "BOOT_SLUMBER"; + break; + case GMU_OOB_DCVS_SET: + request = GMU_OOB_DCVS_REQUEST; + ack = GMU_OOB_DCVS_ACK; + name = "GPU_DCVS"; + break; + default: + return -EINVAL; + } + + /* Trigger the equested OOB operation */ + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 1 << request); + + /* Wait for the acknowledge interrupt */ + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO, val, + val & (1 << ack), 100, 10000); + + if (ret) + dev_err(gmu->dev, + "Timeout waiting for GMU OOB set %s: 0x%x\n", + name, + gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO)); + + /* Clear the acknowledge interrupt */ + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, 1 << ack); + + return ret; +} + +/* Clear a pending OOB state in the GMU */ +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + switch (state) { + case GMU_OOB_GPU_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_GPU_SET_CLEAR); + break; + case GMU_OOB_BOOT_SLUMBER: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_BOOT_SLUMBER_CLEAR); + break; + case GMU_OOB_DCVS_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_DCVS_CLEAR); + break; + } +} + +/* Enable CPU control of SPTP power power collapse */ +static int a6xx_sptprac_enable(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778000); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x38) == 0x28, 1, 100); + + if (ret) { + dev_err(gmu->dev, "Unable to power on SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); + } + + return 0; +} + +/* Disable CPU control of SPTP power power collapse */ +static void a6xx_sptprac_disable(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + /* Make sure retention is on */ + gmu_rmw(gmu, REG_A6XX_GPU_CC_GX_GDSCR, 0, (1 << 11)); + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778001); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x04), 100, 10000); + + if (ret) + dev_err(gmu->dev, "failed to power off SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); +} + +/* Let the GMU know we are starting a boot sequence */ +static int a6xx_gmu_gfx_rail_on(struct a6xx_gmu *gmu) +{ + u32 vote; + + /* Let the GMU know we are getting ready for boot */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 0); + + /* Choose the "default" power level as the highest available */ + vote = gmu->gx_arc_votes[gmu->nr_gpu_freqs - 1]; + + gmu_write(gmu, REG_A6XX_GMU_GX_VOTE_IDX, vote & 0xff); + gmu_write(gmu, REG_A6XX_GMU_MX_VOTE_IDX, (vote >> 8) & 0xff); + + /* Let the GMU know the boot sequence has started */ + return a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); +} + +/* Let the GMU know that we are about to go into slumber */ +static int a6xx_gmu_notify_slumber(struct a6xx_gmu *gmu) +{ + int ret; + + /* Disable the power counter so the GMU isn't busy */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 0); + + /* Disable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) + a6xx_sptprac_disable(gmu); + + /* Tell the GMU to get ready to slumber */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 1); + + ret = a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + if (!ret) { + /* Check to see if the GMU really did slumber */ + if (gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE) + != 0x0f) { + dev_err(gmu->dev, "The GMU did not go into slumber\n"); + ret = -ETIMEDOUT; + } + } + + /* Put fence into allow mode */ + gmu_write(gmu, REG_A6XX_GMU_AO_AHB_FENCE_CTRL, 0); + return ret; +} + +static int a6xx_rpmh_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1 << 1); + /* Wait for the register to finish posting */ + wmb(); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_RSCC_CONTROL_ACK, val, + val & (1 << 1), 100, 10000); + if (ret) { + dev_err(gmu->dev, "Unable to power on the GPU RSC\n"); + return ret; + } + + ret = gmu_poll_timeout(gmu, REG_A6XX_RSCC_SEQ_BUSY_DRV0, val, + !val, 100, 10000); + + if (!ret) { + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); + + /* Re-enable the power counter */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 1); + return 0; + } + + dev_err(gmu->dev, "GPU RSC sequence stuck while waking up the GPU\n"); + return ret; +} + +static void a6xx_rpmh_stop(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, + val, val & (1 << 16), 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to power off the GPU RSC\n"); + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); +} + +static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) +{ + /* Disable SDE clock gating */ + gmu_write(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, BIT(24)); + + /* Setup RSC PDC handshake for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_PDC_SLAVE_ID_DRV0, 1); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 4, 0x80000000); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 4, 0); + gmu_write(gmu, REG_A6XX_RSCC_OVERRIDE_START_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_PDC_SEQ_START_ADDR, 0x4520); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_LO, 0x4510); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_HI, 0x4514); + + /* Load RSC sequencer uCode for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0, 0xa7a506a0); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 1, 0xa1e6a6e7); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 2, 0xa2e081e1); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 3, 0xe9a982e2); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 4, 0x0020e8a8); + + /* Load PDC sequencer uCode for power up and power down sequence */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0, 0xfebea1e1); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 1, 0xa5a4a3a2); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 2, 0x8382a6e0); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 3, 0xbce3e284); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 4, 0x002081fc); + + /* Set TCS commands used by PDC sequence for low power modes */ + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA, 1); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 4, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 8, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA, 2); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 4, 0x3); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 8, 0x3); + + /* Setup GPU PDC */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_START_ADDR, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_ENABLE_PDC, 0x80000001); + + /* ensure no writes happen before the uCode is fully written */ + wmb(); +} + +/* + * The lowest 16 bits of this value are the number of XO clock cycles for main + * hysteresis which is set at 0x1680 cycles (300 us). The higher 16 bits are + * for the shorter hysteresis that happens after main - this is 0xa (.5 us) + */ + +#define GMU_PWR_COL_HYST 0x000a1680 + +/* Set up the idle state for the GMU */ +static void a6xx_gmu_power_config(struct a6xx_gmu *gmu) +{ + /* Disable GMU WB/RB buffer */ + gmu_write(gmu, REG_A6XX_GMU_SYS_BUS_CONFIG, 0x1); + + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0x9c40400); + + switch (gmu->idle_level) { + case GMU_IDLE_STATE_IFPC: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_HM_POWER_COLLAPSE_ENABLE); + /* Fall through */ + case GMU_IDLE_STATE_SPTP: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_SPTPRAC_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_SPTPRAC_POWER_CONTROL_ENABLE); + } + + /* Enable RPMh GPU client */ + gmu_rmw(gmu, REG_A6XX_GMU_RPMH_CTRL, 0, + A6XX_GMU_RPMH_CTRL_RPMH_INTERFACE_ENABLE | + A6XX_GMU_RPMH_CTRL_LLC_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_DDR_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_MX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_CX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_GFX_VOTE_ENABLE); +} + +static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) +{ + static bool rpmh_init; + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + int i, ret; + u32 chipid; + u32 *image; + + if (state == GMU_WARM_BOOT) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } else { + if (WARN(!adreno_gpu->fw[ADRENO_FW_GMU], "GMU firmware is not loaded\n")) + return -ENOENT; + + /* Sanity check the size of the firmware that was loaded */ + if (adreno_gpu->fw[ADRENO_FW_GMU]->size > 0x8000) { + dev_err(gmu->dev, + "GMU firmware is bigger than the available region\n"); + return -EINVAL; + } + + /* Turn on register retention */ + gmu_write(gmu, REG_A6XX_GMU_GENERAL_7, 1); + + /* We only need to load the RPMh microcode once */ + if (!rpmh_init) { + a6xx_gmu_rpmh_init(gmu); + rpmh_init = true; + } else if (state != GMU_RESET) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } + + image = (u32 *) adreno_gpu->fw[ADRENO_FW_GMU]->data; + + for (i = 0; i < adreno_gpu->fw[ADRENO_FW_GMU]->size >> 2; i++) + gmu_write(gmu, REG_A6XX_GMU_CM3_ITCM_START + i, + image[i]); + } + + gmu_write(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, 0); + gmu_write(gmu, REG_A6XX_GMU_CM3_BOOT_CONFIG, 0x02); + + /* Write the iova of the HFI table */ + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_ADDR, gmu->hfi->iova); + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_INFO, 1); + + gmu_write(gmu, REG_A6XX_GMU_AHB_FENCE_RANGE_0, + (1 << 31) | (0xa << 18) | (0xa0)); + + chipid = adreno_gpu->rev.core << 24; + chipid |= adreno_gpu->rev.major << 16; + chipid |= adreno_gpu->rev.minor << 12; + chipid |= adreno_gpu->rev.patchid << 8; + + gmu_write(gmu, REG_A6XX_GMU_HFI_SFR_ADDR, chipid); + + /* Set up the lowest idle level on the GMU */ + a6xx_gmu_power_config(gmu); + + ret = a6xx_gmu_start(gmu); + if (ret) + return ret; + + ret = a6xx_gmu_gfx_rail_on(gmu); + if (ret) + return ret; + + /* Enable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) { + ret = a6xx_sptprac_enable(gmu); + if (ret) + return ret; + } + + ret = a6xx_gmu_hfi_start(gmu); + if (ret) + return ret; + + /* FIXME: Do we need this wmb() here? */ + wmb(); + + return 0; +} + +#define A6XX_HFI_IRQ_MASK \ + (A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ | \ + A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) + +#define A6XX_GMU_IRQ_MASK \ + (A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + +static void a6xx_gmu_irq_enable(struct a6xx_gmu *gmu) +{ + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, ~0); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, + ~A6XX_GMU_IRQ_MASK); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + ~A6XX_HFI_IRQ_MASK); + + enable_irq(gmu->gmu_irq); + enable_irq(gmu->hfi_irq); +} + +static void a6xx_gmu_irq_disable(struct a6xx_gmu *gmu) +{ + disable_irq(gmu->gmu_irq); + disable_irq(gmu->hfi_irq); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, ~0); +} + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int ret; + u32 val; + + /* Flush all the queues */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts */ + a6xx_gmu_irq_disable(gmu); + + /* Force off SPTP in case the GMU is managing it */ + a6xx_sptprac_disable(gmu); + + /* Make sure there are no outstanding RPMh votes */ + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS0_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS1_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS2_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS3_DRV0_STATUS, val, + (val & 1), 100, 1000); + + /* Force off the GX GSDC */ + regulator_force_disable(gmu->gx); + + /* Disable the resources */ + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + pm_runtime_put_sync(gmu->dev); + + /* Re-enable the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + + a6xx_gmu_irq_enable(gmu); + + ret = a6xx_gmu_fw_start(gmu, GMU_RESET); + if (!ret) + ret = a6xx_hfi_start(gmu, GMU_COLD_BOOT); + + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int status, ret; + + if (WARN(!gmu->mmio, "The GMU is not set up yet\n")) + return 0; + + /* Turn on the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + + a6xx_gmu_irq_enable(gmu); + + /* Check to see if we are doing a cold or warm boot */ + status = gmu_read(gmu, REG_A6XX_GMU_GENERAL_7) == 1 ? + GMU_WARM_BOOT : GMU_COLD_BOOT; + + ret = a6xx_gmu_fw_start(gmu, status); + if (ret) + goto out; + + ret = a6xx_hfi_start(gmu, status); + +out: + /* Make sure to turn off the boot OOB request on error */ + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu) +{ + u32 reg; + + if (!gmu->mmio) + return true; + + reg = gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS); + + if (reg & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB) + return false; + + return true; +} + +int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + u32 val; + + /* + * The GMU may still be in slumber unless the GPU started so check and + * skip putting it back into slumber if so + */ + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val != 0xf) { + int ret = a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Temporary until we can recover safely */ + BUG_ON(ret); + + /* tell the GMU we want to slumber */ + a6xx_gmu_notify_slumber(gmu); + + ret = gmu_poll_timeout(gmu, + REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS, val, + !(val & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB), + 100, 10000); + + /* + * Let the user know we failed to slumber but don't worry too + * much because we are powering down anyway + */ + + if (ret) + dev_err(gmu->dev, + "Unable to slumber GMU: status = 0%x/0%x\n", + gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS), + gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS2)); + } + + /* Turn off HFI */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts and mask the hardware */ + a6xx_gmu_irq_disable(gmu); + + /* Tell RPMh to power off the GPU */ + a6xx_rpmh_stop(gmu); + + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + + pm_runtime_put_sync(gmu->dev); + + return 0; +} + +static void a6xx_gmu_memory_free(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo) +{ + int count, i; + u64 iova; + + if (IS_ERR_OR_NULL(bo)) + return; + + count = bo->size >> PAGE_SHIFT; + iova = bo->iova; + + for (i = 0; i < count; i++, iova += PAGE_SIZE) { + iommu_unmap(gmu->domain, iova, PAGE_SIZE); + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); +} + +static struct a6xx_gmu_bo *a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, + size_t size) +{ + struct a6xx_gmu_bo *bo; + int ret, count, i; + + bo = kzalloc(sizeof(*bo), GFP_KERNEL); + if (!bo) + return ERR_PTR(-ENOMEM); + + bo->size = PAGE_ALIGN(size); + + count = bo->size >> PAGE_SHIFT; + + bo->pages = kcalloc(count, sizeof(struct page *), GFP_KERNEL); + if (!bo->pages) { + kfree(bo); + return ERR_PTR(-ENOMEM); + } + + for (i = 0; i < count; i++) { + bo->pages[i] = alloc_page(GFP_KERNEL); + if (!bo->pages[i]) + goto err; + } + + bo->iova = gmu->uncached_iova_base; + + for (i = 0; i < count; i++) { + ret = iommu_map(gmu->domain, + bo->iova + (PAGE_SIZE * i), + page_to_phys(bo->pages[i]), PAGE_SIZE, + IOMMU_READ | IOMMU_WRITE); + + if (ret) { + dev_err(gmu->dev, "Unable to map GMU buffer object\n"); + + for (i = i - 1 ; i >= 0; i--) + iommu_unmap(gmu->domain, + bo->iova + (PAGE_SIZE * i), + PAGE_SIZE); + + goto err; + } + } + + bo->virt = vmap(bo->pages, count, VM_IOREMAP, + pgprot_writecombine(PAGE_KERNEL)); + if (!bo->virt) + goto err; + + /* Align future IOVA addresses on 1MB boundaries */ + gmu->uncached_iova_base += ALIGN(size, SZ_1M); + + return bo; + +err: + for (i = 0; i < count; i++) { + if (bo->pages[i]) + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); + + return ERR_PTR(-ENOMEM); +} + +static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu) +{ + int ret; + + /* + * The GMU address space is hardcoded to treat the range + * 0x60000000 - 0x80000000 as un-cached memory. All buffers shared + * between the GMU and the CPU will live in this space + */ + gmu->uncached_iova_base = 0x60000000; + + + gmu->domain = iommu_domain_alloc(&platform_bus_type); + if (!gmu->domain) + return -ENODEV; + + ret = iommu_attach_device(gmu->domain, gmu->dev); + + if (ret) { + iommu_domain_free(gmu->domain); + gmu->domain = NULL; + } + + return ret; +} + +/* Get the list of RPMh voltage levels from cmd-db */ +static int a6xx_gmu_rpmh_arc_cmds(const char *id, void *vals, int size) +{ + u32 len = cmd_db_read_aux_data_len(id); + + if (!len) + return 0; + + if (WARN_ON(len > size)) + return -EINVAL; + + cmd_db_read_aux_data(id, vals, len); + + /* + * The data comes back as an array of unsigned shorts so adjust the + * count accordingly + */ + return len >> 1; +} + +/* Return the 'arc-level' for the given frequency */ +static u32 a6xx_gmu_get_arc_level(struct device *dev, unsigned long freq) +{ + struct dev_pm_opp *opp; + struct device_node *np; + u32 val = 0; + + if (!freq) + return 0; + + opp = dev_pm_opp_find_freq_exact(dev, freq, true); + if (IS_ERR(opp)) + return 0; + + np = dev_pm_opp_get_of_node(opp); + + if (np) { + of_property_read_u32(np, "qcom,level", &val); + of_node_put(np); + } + + dev_pm_opp_put(opp); + + return val; +} + +static int a6xx_gmu_rpmh_arc_votes_init(struct device *dev, u32 *votes, + unsigned long *freqs, int freqs_count, + u16 *pri, int pri_count, + u16 *sec, int sec_count) +{ + int i, j; + + /* Construct a vote for each frequency */ + for (i = 0; i < freqs_count; i++) { + u8 pindex = 0, sindex = 0; + u32 level = a6xx_gmu_get_arc_level(dev, freqs[i]); + + /* Get the primary index that matches the arc level */ + for (j = 0; j < pri_count; j++) { + if (pri[j] >= level) { + pindex = j; + break; + } + } + + if (j == pri_count) { + dev_err(dev, + "Level %u not found in in the RPMh list\n", + level); + dev_err(dev, "Available levels:\n"); + for (j = 0; j < pri_count; j++) + dev_err(dev, " %u\n", pri[j]); + + return -EINVAL; + } + + /* + * Look for a level in in the secondary list that matches. If + * nothing fits, use the maximum + */ + sindex = sec_count - 1; + + for (j = 0; j < sec_count - 1; j++) { + if (sec[j] >= level) { + sindex = j; + break; + } + } + + /* Construct the vote */ + votes[i] = ((pri[pindex] & 0xffff) << 16) | + (sindex << 8) | pindex; + } + + return 0; +} + +/* + * The GMU votes with the RPMh for itself and on behalf of the GPU but we need + * to construct the list of votes on the CPU and send it over. Query the RPMh + * voltage levels and build the votes + */ + +static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + u16 gx[16], cx[16], mx[16]; + u32 gxcount, cxcount, mxcount; + int ret; + + /* Get the list of available voltage levels for each component */ + gxcount = a6xx_gmu_rpmh_arc_cmds("gfx.lvl", gx, sizeof(gx)); + cxcount = a6xx_gmu_rpmh_arc_cmds("cx.lvl", cx, sizeof(cx)); + mxcount = a6xx_gmu_rpmh_arc_cmds("mx.lvl", mx, sizeof(mx)); + + /* Build the GX votes */ + ret = a6xx_gmu_rpmh_arc_votes_init(&gpu->pdev->dev, gmu->gx_arc_votes, + gmu->gpu_freqs, gmu->nr_gpu_freqs, + gx, gxcount, mx, mxcount); + + /* Build the CX votes */ + ret |= a6xx_gmu_rpmh_arc_votes_init(gmu->dev, gmu->cx_arc_votes, + gmu->gmu_freqs, gmu->nr_gmu_freqs, + cx, cxcount, mx, mxcount); + + return ret; +} + +static int a6xx_gmu_build_freq_table(struct device *dev, unsigned long *freqs, + u32 size) +{ + int count = dev_pm_opp_get_opp_count(dev); + struct dev_pm_opp *opp; + int i, index = 0; + unsigned long freq = 1; + + /* + * The OPP table doesn't contain the "off" frequency level so we need to + * add 1 to the table size to account for it + */ + + if (WARN(count + 1 > size, + "The GMU frequency table is being truncated\n")) + count = size - 1; + + /* Set the "off" frequency */ + freqs[index++] = 0; + + for (i = 0; i < count; i++) { + opp = dev_pm_opp_find_freq_ceil(dev, &freq); + if (IS_ERR(opp)) + break; + + dev_pm_opp_put(opp); + freqs[index++] = freq++; + } + + return index; +} + +static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + int ret = 0; + + /* + * The GMU handles its own frequency switching so build a list of + * available frequencies to send during initalization + */ + ret = dev_pm_opp_of_add_table(gmu->dev); + if (ret) { + dev_err(gmu->dev, "Unable to set the OPP table for the GMU\n"); + return ret; + } + + gmu->nr_gmu_freqs = a6xx_gmu_build_freq_table(gmu->dev, + gmu->gmu_freqs, ARRAY_SIZE(gmu->gmu_freqs)); + + /* + * The GMU also handles GPU frequency switching so build a list + * from the GPU OPP table + */ + gmu->nr_gpu_freqs = a6xx_gmu_build_freq_table(&gpu->pdev->dev, + gmu->gpu_freqs, ARRAY_SIZE(gmu->gpu_freqs)); + + /* Build the list of RPMh votes that we'll send to the GMU */ + return a6xx_gmu_rpmh_votes_init(gmu); +} + +static int a6xx_gmu_clocks_probe(struct a6xx_gmu *gmu) +{ + int ret = msm_clk_bulk_get(gmu->dev, &gmu->clocks); + + if (ret < 1) + return ret; + + gmu->nr_clocks = ret; + + gmu->core_clk = msm_clk_bulk_get_clock(gmu->clocks, + gmu->nr_clocks, "gmu"); + + return 0; +} + +static void __iomem *a6xx_gmu_get_mmio(struct platform_device *pdev, + const char *name) +{ + void __iomem *ret; + struct resource *res = platform_get_resource_byname(pdev, + IORESOURCE_MEM, name); + + if (!res) { + dev_err(&pdev->dev, "Unable to find the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + ret = devm_ioremap(&pdev->dev, res->start, resource_size(res)); + if (!ret) { + dev_err(&pdev->dev, "Unable to map the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + return ret; +} + +static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev, + const char *name, irq_handler_t handler) +{ + int irq, ret; + + irq = platform_get_irq_byname(pdev, name); + + ret = devm_request_irq(&pdev->dev, irq, handler, IRQF_TRIGGER_HIGH, + name, gmu); + if (ret) { + dev_err(&pdev->dev, "Unable to get interrupt %s\n", name); + return ret; + } + + disable_irq(irq); + + return irq; +} + +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + if (IS_ERR_OR_NULL(gmu->mmio)) + return; + + pm_runtime_disable(gmu->dev); + a6xx_gmu_stop(a6xx_gpu); + + a6xx_gmu_irq_disable(gmu); + a6xx_gmu_memory_free(gmu, gmu->hfi); + + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); +} + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + struct platform_device *pdev = of_find_device_by_node(node); + int ret; + + if (!pdev) + return -ENODEV; + + gmu->dev = &pdev->dev; + + of_dma_configure(gmu->dev, node); + + /* Fow now, don't do anything fancy until we get our feet under us */ + gmu->idle_level = GMU_IDLE_STATE_ACTIVE; + + pm_runtime_enable(gmu->dev); + gmu->gx = devm_regulator_get(gmu->dev, "vdd"); + + /* Get the list of clocks */ + ret = a6xx_gmu_clocks_probe(gmu); + if (ret) + return ret; + + /* Set up the IOMMU context bank */ + ret = a6xx_gmu_memory_probe(gmu); + if (ret) + return ret; + + /* Allocate memory for for the HFI queues */ + gmu->hfi = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->hfi)) + goto err; + + /* Allocate memory for the GMU debug region */ + gmu->debug = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->debug)) + goto err; + + /* Map the GMU registers */ + gmu->mmio = a6xx_gmu_get_mmio(pdev, "gmu"); + + /* Map the GPU power domain controller registers */ + gmu->pdc_mmio = a6xx_gmu_get_mmio(pdev, "gmu_pdc"); + + if (IS_ERR(gmu->mmio) || IS_ERR(gmu->pdc_mmio)) + goto err; + + /* Get the HFI and GMU interrupts */ + gmu->hfi_irq = a6xx_gmu_get_irq(gmu, pdev, "hfi", a6xx_hfi_irq); + gmu->gmu_irq = a6xx_gmu_get_irq(gmu, pdev, "gmu", a6xx_gmu_irq); + + if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) + goto err; + + /* Set up a tasklet to handle GMU HFI responses */ + tasklet_init(&gmu->hfi_tasklet, a6xx_hfi_task, (unsigned long) gmu); + + /* Get the power levels for the GMU and GPU */ + a6xx_gmu_pwrlevels_probe(gmu); + + /* Set up the HFI queues */ + a6xx_hfi_init(gmu); + + return 0; +err: + a6xx_gmu_memory_free(gmu, gmu->hfi); + + if (gmu->domain) { + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); + } + + return -ENODEV; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h new file mode 100644 index 000000000000..d9a386c18799 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_GMU_H_ +#define _A6XX_GMU_H_ + +#include <linux/interrupt.h> +#include "msm_drv.h" +#include "a6xx_hfi.h" + +struct a6xx_gmu_bo { + void *virt; + size_t size; + u64 iova; + struct page **pages; +}; + +/* + * These define the different GMU wake up options - these define how both the + * CPU and the GMU bring up the hardware + */ + +/* THe GMU has already been booted and the rentention registers are active */ +#define GMU_WARM_BOOT 0 + +/* the GMU is coming up for the first time or back from a power collapse */ +#define GMU_COLD_BOOT 1 + +/* The GMU is being soft reset after a fault */ +#define GMU_RESET 2 + +/* + * These define the level of control that the GMU has - the higher the number + * the more things that the GMU hardware controls on its own. + */ + +/* The GMU does not do any idle state management */ +#define GMU_IDLE_STATE_ACTIVE 0 + +/* The GMU manages SPTP power collapse */ +#define GMU_IDLE_STATE_SPTP 2 + +/* The GMU does automatic IFPC (intra-frame power collapse) */ +#define GMU_IDLE_STATE_IFPC 3 + +struct a6xx_gmu { + struct device *dev; + + void * __iomem mmio; + void * __iomem pdc_mmio; + + int hfi_irq; + int gmu_irq; + + struct regulator *gx; + + struct iommu_domain *domain; + u64 uncached_iova_base; + + int idle_level; + + struct a6xx_gmu_bo *hfi; + struct a6xx_gmu_bo *debug; + + int nr_clocks; + struct clk_bulk_data *clocks; + struct clk *core_clk; + + int nr_gpu_freqs; + unsigned long gpu_freqs[16]; + u32 gx_arc_votes[16]; + + int nr_gmu_freqs; + unsigned long gmu_freqs[4]; + u32 cx_arc_votes[4]; + + struct a6xx_hfi_queue queues[2]; + + struct tasklet_struct hfi_tasklet; +}; + +static inline u32 gmu_read(struct a6xx_gmu *gmu, u32 offset) +{ + return msm_readl(gmu->mmio + (offset << 2)); +} + +static inline void gmu_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->mmio + (offset << 2)); +} + +static inline void pdc_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->pdc_mmio + (offset << 2)); +} + +static inline void gmu_rmw(struct a6xx_gmu *gmu, u32 reg, u32 mask, u32 or) +{ + u32 val = gmu_read(gmu, reg); + + val &= ~mask; + + gmu_write(gmu, reg, val | or); +} + +#define gmu_poll_timeout(gmu, addr, val, cond, interval, timeout) \ + readl_poll_timeout((gmu)->mmio + ((addr) << 2), val, cond, \ + interval, timeout) + +/* + * These are the available OOB (out of band requests) to the GMU where "out of + * band" means that the CPU talks to the GMU directly and not through HFI. + * Normally this works by writing a ITCM/DTCM register and then triggering a + * interrupt (the "request" bit) and waiting for an acknowledgment (the "ack" + * bit). The state is cleared by writing the "clear' bit to the GMU interrupt. + * + * These are used to force the GMU/GPU to stay on during a critical sequence or + * for hardware workarounds. + */ + +enum a6xx_gmu_oob_state { + GMU_OOB_BOOT_SLUMBER = 0, + GMU_OOB_GPU_SET, + GMU_OOB_DCVS_SET, +}; + +/* These are the interrupt / ack bits for each OOB request that are set + * in a6xx_gmu_set_oob and a6xx_clear_oob + */ + +/* + * Let the GMU know that a boot or slumber operation has started. The value in + * REG_A6XX_GMU_BOOT_SLUMBER_OPTION lets the GMU know which operation we are + * doing + */ +#define GMU_OOB_BOOT_SLUMBER_REQUEST 22 +#define GMU_OOB_BOOT_SLUMBER_ACK 30 +#define GMU_OOB_BOOT_SLUMBER_CLEAR 30 + +/* + * Set a new power level for the GPU when the CPU is doing frequency scaling + */ +#define GMU_OOB_DCVS_REQUEST 23 +#define GMU_OOB_DCVS_ACK 31 +#define GMU_OOB_DCVS_CLEAR 31 + +/* + * Let the GMU know to not turn off any GPU registers while the CPU is in a + * critical section + */ +#define GMU_OOB_GPU_SET_REQUEST 16 +#define GMU_OOB_GPU_SET_ACK 24 +#define GMU_OOB_GPU_SET_CLEAR 24 + + +void a6xx_hfi_init(struct a6xx_gmu *gmu); +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state); +void a6xx_hfi_stop(struct a6xx_gmu *gmu); + +void a6xx_hfi_task(unsigned long data); + +#endif diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c new file mode 100644 index 000000000000..b3b85ebce88d --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -0,0 +1,807 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + + +#include "msm_gem.h" +#include "msm_mmu.h" +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static inline bool _a6xx_check_idle(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Check that the GMU is idle */ + if (!a6xx_gmu_isidle(&a6xx_gpu->gmu)) + return false; + + /* Check tha the CX master is idle */ + if (gpu_read(gpu, REG_A6XX_RBBM_STATUS) & + ~A6XX_RBBM_STATUS_CP_AHB_BUSY_CX_MASTER) + return false; + + return !(gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS) & + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT); +} + +bool a6xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + /* wait for CP to drain ringbuffer: */ + if (!adreno_idle(gpu, ring)) + return false; + + if (spin_until(_a6xx_check_idle(gpu))) { + DRM_ERROR("%s: %ps: timeout waiting for GPU to idle: status %8.8X irq %8.8X rptr/wptr %d/%d\n", + gpu->name, __builtin_return_address(0), + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR)); + return false; + } + + return true; +} + +static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + uint32_t wptr; + unsigned long flags; + + spin_lock_irqsave(&ring->lock, flags); + + /* Copy the shadow to the actual register */ + ring->cur = ring->next; + + /* Make sure to wrap wptr if we need to */ + wptr = get_wptr(ring); + + spin_unlock_irqrestore(&ring->lock, flags); + + /* Make sure everything is posted before making a decision */ + mb(); + + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); +} + +static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, + struct msm_file_private *ctx) +{ + struct msm_drm_private *priv = gpu->dev->dev_private; + struct msm_ringbuffer *ring = submit->ring; + unsigned int i; + + /* Invalidate CCU depth and color */ + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_DEPTH); + + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_COLOR); + + /* Submit the commands */ + for (i = 0; i < submit->nr_cmds; i++) { + switch (submit->cmd[i].type) { + case MSM_SUBMIT_CMD_IB_TARGET_BUF: + break; + case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: + if (priv->lastctx == ctx) + break; + case MSM_SUBMIT_CMD_BUF: + OUT_PKT7(ring, CP_INDIRECT_BUFFER_PFE, 3); + OUT_RING(ring, lower_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, upper_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, submit->cmd[i].size); + break; + } + } + + /* Write the fence to the scratch register */ + OUT_PKT4(ring, REG_A6XX_CP_SCRATCH_REG(2), 1); + OUT_RING(ring, submit->seqno); + + /* + * Execute a CACHE_FLUSH_TS event. This will ensure that the + * timestamp is written to the memory and then triggers the interrupt + */ + OUT_PKT7(ring, CP_EVENT_WRITE, 4); + OUT_RING(ring, CACHE_FLUSH_TS | (1 << 31)); + OUT_RING(ring, lower_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, upper_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, submit->seqno); + + a6xx_flush(gpu, ring); +} + +static const struct { + u32 offset; + u32 value; +} a6xx_hwcg[] = { + {REG_A6XX_RBBM_CLOCK_CNTL_SP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP0, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP1, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP2, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP3, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP0, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP1, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP2, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP3, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_HYST_SP0, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP1, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP2, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP3, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP0, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP1, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP2, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP3, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP0, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP1, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP2, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP3, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_HYST_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP0, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP1, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP2, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP3, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP0, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP1, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP2, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP3, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_CNTL_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_UCHE, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_HYST_UCHE, 0x00000004}, + {REG_A6XX_RBBM_CLOCK_DELAY_UCHE, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB0, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB1, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB2, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB3, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU0, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU1, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU2, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU3, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU0, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU1, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU2, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU3, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_CNTL_RAC, 0x05022022}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RAC, 0x00005555}, + {REG_A6XX_RBBM_CLOCK_DELAY_RAC, 0x00000011}, + {REG_A6XX_RBBM_CLOCK_HYST_RAC, 0x00445044}, + {REG_A6XX_RBBM_CLOCK_CNTL_TSE_RAS_RBBM, 0x04222222}, + {REG_A6XX_RBBM_CLOCK_MODE_GPC, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_MODE_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_HYST_TSE_RAS_RBBM, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_HYST_GPC, 0x04104004}, + {REG_A6XX_RBBM_CLOCK_HYST_VFD, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_TSE_RAS_RBBM, 0x00004000}, + {REG_A6XX_RBBM_CLOCK_DELAY_GPC, 0x00000200}, + {REG_A6XX_RBBM_CLOCK_DELAY_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ_2, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_MODE_HLSQ, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_GMU_GX, 0x00000222}, + {REG_A6XX_RBBM_CLOCK_DELAY_GMU_GX, 0x00000111}, + {REG_A6XX_RBBM_CLOCK_HYST_GMU_GX, 0x00000555} +}; + +static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + unsigned int i; + u32 val; + + val = gpu_read(gpu, REG_A6XX_RBBM_CLOCK_CNTL); + + /* Don't re-program the registers if they are already correct */ + if ((!state && !val) || (state && (val == 0x8aa8aa02))) + return; + + /* Disable SP clock before programming HWCG registers */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 1, 0); + + for (i = 0; i < ARRAY_SIZE(a6xx_hwcg); i++) + gpu_write(gpu, a6xx_hwcg[i].offset, + state ? a6xx_hwcg[i].value : 0); + + /* Enable SP clock */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 0, 1); + + gpu_write(gpu, REG_A6XX_RBBM_CLOCK_CNTL, state ? 0x8aa8aa02 : 0); +} + +static int a6xx_cp_init(struct msm_gpu *gpu) +{ + struct msm_ringbuffer *ring = gpu->rb[0]; + + OUT_PKT7(ring, CP_ME_INIT, 8); + + OUT_RING(ring, 0x0000002f); + + /* Enable multiple hardware contexts */ + OUT_RING(ring, 0x00000003); + + /* Enable error detection */ + OUT_RING(ring, 0x20000000); + + /* Don't enable header dump */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + /* No workarounds enabled */ + OUT_RING(ring, 0x00000000); + + /* Pad rest of the cmds with 0's */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + a6xx_flush(gpu, ring); + return a6xx_idle(gpu, ring) ? 0 : -EINVAL; +} + +static int a6xx_ucode_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (!a6xx_gpu->sqe_bo) { + a6xx_gpu->sqe_bo = adreno_fw_create_bo(gpu, + adreno_gpu->fw[ADRENO_FW_SQE], &a6xx_gpu->sqe_iova); + + if (IS_ERR(a6xx_gpu->sqe_bo)) { + int ret = PTR_ERR(a6xx_gpu->sqe_bo); + + a6xx_gpu->sqe_bo = NULL; + DRM_DEV_ERROR(&gpu->pdev->dev, + "Could not allocate SQE ucode: %d\n", ret); + + return ret; + } + } + + gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE_LO, + REG_A6XX_CP_SQE_INSTR_BASE_HI, a6xx_gpu->sqe_iova); + + return 0; +} + +#define A6XX_INT_MASK (A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_CP_HW_ERROR | \ + A6XX_RBBM_INT_0_MASK_CP_IB2 | \ + A6XX_RBBM_INT_0_MASK_CP_IB1 | \ + A6XX_RBBM_INT_0_MASK_CP_RB | \ + A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT | \ + A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS | \ + A6XX_RBBM_INT_0_MASK_UCHE_TRAP_INTR) + +static int a6xx_hw_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + /* Make sure the GMU keeps the GPU on while we set it up */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_CNTL, 0); + + /* + * Disable the trusted memory range - we don't actually supported secure + * memory rendering at this point in time and we don't want to block off + * part of the virtual memory space. + */ + gpu_write64(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO, + REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000); + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000); + + /* enable hardware clockgating */ + a6xx_set_hwcg(gpu, true); + + /* VBIF start */ + gpu_write(gpu, REG_A6XX_VBIF_GATE_OFF_WRREQ_EN, 0x00000009); + gpu_write(gpu, REG_A6XX_RBBM_VBIF_CLIENT_QOS_CNTL, 0x3); + + /* Make all blocks contribute to the GPU BUSY perf counter */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_GPU_BUSY_MASKED, 0xffffffff); + + /* Disable L2 bypass in the UCHE */ + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_LO, 0xffffffc0); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_HI, 0x0001ffff); + + /* Set the GMEM VA range [0x100000:0x100000 + gpu->gmem - 1] */ + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MIN_LO, + REG_A6XX_UCHE_GMEM_RANGE_MIN_HI, 0x00100000); + + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MAX_LO, + REG_A6XX_UCHE_GMEM_RANGE_MAX_HI, + 0x00100000 + adreno_gpu->gmem - 1); + + gpu_write(gpu, REG_A6XX_UCHE_FILTER_CNTL, 0x804); + gpu_write(gpu, REG_A6XX_UCHE_CACHE_WAYS, 0x4); + + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_2, 0x010000c0); + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_1, 0x8040362c); + + /* Setting the mem pool size */ + gpu_write(gpu, REG_A6XX_CP_MEM_POOL_SIZE, 128); + + /* Setting the primFifo thresholds default values */ + gpu_write(gpu, REG_A6XX_PC_DBG_ECO_CNTL, (0x300 << 11)); + + /* Set the AHB default slave response to "ERROR" */ + gpu_write(gpu, REG_A6XX_CP_AHB_CNTL, 0x1); + + /* Turn on performance counters */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_CNTL, 0x1); + + /* Select CP0 to always count cycles */ + gpu_write(gpu, REG_A6XX_CP_PERFCTR_CP_SEL_0, PERF_CP_ALWAYS_COUNT); + + /* FIXME: not sure if this should live here or in a6xx_gmu.c */ + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_MASK, + 0xff000000); + gmu_rmw(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_SELECT_0, + 0xff, 0x20); + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, + 0x01); + + gpu_write(gpu, REG_A6XX_RB_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_TPL1_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_SP_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_UCHE_MODE_CNTL, 2 << 21); + + /* Enable fault detection */ + gpu_write(gpu, REG_A6XX_RBBM_INTERFACE_HANG_INT_CNTL, + (1 << 30) | 0x1fffff); + + gpu_write(gpu, REG_A6XX_UCHE_CLIENT_PF, 1); + + /* Protect registers from the CP */ + gpu_write(gpu, REG_A6XX_CP_PROTECT_CNTL, 0x00000003); + + gpu_write(gpu, REG_A6XX_CP_PROTECT(0), + A6XX_PROTECT_RDONLY(0x600, 0x51)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(1), A6XX_PROTECT_RW(0xae50, 0x2)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(2), A6XX_PROTECT_RW(0x9624, 0x13)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(3), A6XX_PROTECT_RW(0x8630, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(4), A6XX_PROTECT_RW(0x9e70, 0x1)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(5), A6XX_PROTECT_RW(0x9e78, 0x187)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(6), A6XX_PROTECT_RW(0xf000, 0x810)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(7), + A6XX_PROTECT_RDONLY(0xfc00, 0x3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(8), A6XX_PROTECT_RW(0x50e, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(9), A6XX_PROTECT_RDONLY(0x50f, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(10), A6XX_PROTECT_RW(0x510, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(11), + A6XX_PROTECT_RDONLY(0x0, 0x4f9)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(12), + A6XX_PROTECT_RDONLY(0x501, 0xa)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(13), + A6XX_PROTECT_RDONLY(0x511, 0x44)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(14), A6XX_PROTECT_RW(0xe00, 0xe)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(15), A6XX_PROTECT_RW(0x8e00, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(16), A6XX_PROTECT_RW(0x8e50, 0xf)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(17), A6XX_PROTECT_RW(0xbe02, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(18), + A6XX_PROTECT_RW(0xbe20, 0x11f3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(19), A6XX_PROTECT_RW(0x800, 0x82)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(20), A6XX_PROTECT_RW(0x8a0, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(21), A6XX_PROTECT_RW(0x8ab, 0x19)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(22), A6XX_PROTECT_RW(0x900, 0x4d)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(23), A6XX_PROTECT_RW(0x98d, 0x76)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(24), + A6XX_PROTECT_RDONLY(0x8d0, 0x23)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(25), + A6XX_PROTECT_RDONLY(0x980, 0x4)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(26), A6XX_PROTECT_RW(0xa630, 0x0)); + + /* Enable interrupts */ + gpu_write(gpu, REG_A6XX_RBBM_INT_0_MASK, A6XX_INT_MASK); + + ret = adreno_hw_init(gpu); + if (ret) + goto out; + + ret = a6xx_ucode_init(gpu); + if (ret) + goto out; + + /* Always come up on rb 0 */ + a6xx_gpu->cur_ring = gpu->rb[0]; + + /* Enable the SQE_to start the CP engine */ + gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); + + ret = a6xx_cp_init(gpu); + if (ret) + goto out; + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TRUST_CNTL, 0x0); + +out: + /* + * Tell the GMU that we are done touching the GPU and it can start power + * management + */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + /* Take the GMU out of its special boot mode */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +static void a6xx_dump(struct msm_gpu *gpu) +{ + dev_info(&gpu->pdev->dev, "status: %08x\n", + gpu_read(gpu, REG_A6XX_RBBM_STATUS)); + adreno_dump(gpu); +} + +#define VBIF_RESET_ACK_TIMEOUT 100 +#define VBIF_RESET_ACK_MASK 0x00f0 + +static void a6xx_recover(struct msm_gpu *gpu) +{ + int i; + + adreno_dump_info(gpu); + + for (i = 0; i < 8; i++) + dev_info(&gpu->pdev->dev, "CP_SCRATCH_REG%d: %u\n", i, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(i))); + + if (hang_debug) + a6xx_dump(gpu); + + gpu->funcs->pm_suspend(gpu); + gpu->funcs->pm_resume(gpu); + + msm_gpu_hw_init(gpu); +} + +static int a6xx_fault_handler(void *arg, unsigned long iova, int flags) +{ + struct msm_gpu *gpu = arg; + + pr_warn_ratelimited("*** gpu fault: iova=%08lx, flags=%d (%u,%u,%u,%u)\n", + iova, flags, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(4)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(5)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(6)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(7))); + + return -EFAULT; +} + +static void a6xx_cp_hw_err_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_CP_INTERRUPT_STATUS); + + if (status & A6XX_CP_INT_CP_OPCODE_ERROR) { + u32 val; + + gpu_write(gpu, REG_A6XX_CP_SQE_STAT_ADDR, 1); + val = gpu_read(gpu, REG_A6XX_CP_SQE_STAT_DATA); + dev_err_ratelimited(&gpu->pdev->dev, + "CP | opcode error | possible opcode=0x%8.8X\n", + val); + } + + if (status & A6XX_CP_INT_CP_UCODE_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, + "CP ucode error interrupt\n"); + + if (status & A6XX_CP_INT_CP_HW_FAULT_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | HW fault | status=0x%8.8X\n", + gpu_read(gpu, REG_A6XX_CP_HW_FAULT)); + + if (status & A6XX_CP_INT_CP_REGISTER_PROTECTION_ERROR) { + u32 val = gpu_read(gpu, REG_A6XX_CP_PROTECT_STATUS); + + dev_err_ratelimited(&gpu->pdev->dev, + "CP | protected mode error | %s | addr=0x%8.8X | status=0x%8.8X\n", + val & (1 << 20) ? "READ" : "WRITE", + (val & 0x3ffff), val); + } + + if (status & A6XX_CP_INT_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP AHB error interrupt\n"); + + if (status & A6XX_CP_INT_CP_VSD_PARITY_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP VSD decoder parity error\n"); + + if (status & A6XX_CP_INT_CP_ILLEGAL_INSTR_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP illegal instruction error\n"); + +} + +static void a6xx_fault_detect_irq(struct msm_gpu *gpu) +{ + struct drm_device *dev = gpu->dev; + struct msm_drm_private *priv = dev->dev_private; + struct msm_ringbuffer *ring = gpu->funcs->active_ring(gpu); + + DRM_DEV_ERROR(&gpu->pdev->dev, + "gpu fault ring %d fence %x status %8.8X rb %4.4x/%4.4x ib1 %16.16llX/%4.4x ib2 %16.16llX/%4.4x\n", + ring ? ring->id : -1, ring ? ring->seqno : 0, + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR), + gpu_read64(gpu, REG_A6XX_CP_IB1_BASE, REG_A6XX_CP_IB1_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB1_REM_SIZE), + gpu_read64(gpu, REG_A6XX_CP_IB2_BASE, REG_A6XX_CP_IB2_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB2_REM_SIZE)); + + /* Turn off the hangcheck timer to keep it from bothering us */ + del_timer(&gpu->hangcheck_timer); + + queue_work(priv->wq, &gpu->recover_work); +} + +static irqreturn_t a6xx_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS); + + gpu_write(gpu, REG_A6XX_RBBM_INT_CLEAR_CMD, status); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT) + a6xx_fault_detect_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | AHB bus error\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_HW_ERROR) + a6xx_cp_hw_err_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB ASYNC overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB bus overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS) + dev_err_ratelimited(&gpu->pdev->dev, "UCHE | Out of bounds access\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) + msm_gpu_retire(gpu); + + return IRQ_HANDLED; +} + +static const u32 a6xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE, REG_A6XX_CP_RB_BASE), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE_HI, REG_A6XX_CP_RB_BASE_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR, + REG_A6XX_CP_RB_RPTR_ADDR_LO), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR_HI, + REG_A6XX_CP_RB_RPTR_ADDR_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR, REG_A6XX_CP_RB_RPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_WPTR, REG_A6XX_CP_RB_WPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_CNTL, REG_A6XX_CP_RB_CNTL), +}; + +static const u32 a6xx_registers[] = { + 0x0000, 0x0002, 0x0010, 0x0010, 0x0012, 0x0012, 0x0018, 0x001b, + 0x001e, 0x0032, 0x0038, 0x003c, 0x0042, 0x0042, 0x0044, 0x0044, + 0x0047, 0x0047, 0x0056, 0x0056, 0x00ad, 0x00ae, 0x00b0, 0x00fb, + 0x0100, 0x011d, 0x0200, 0x020d, 0x0210, 0x0213, 0x0218, 0x023d, + 0x0400, 0x04f9, 0x0500, 0x0500, 0x0505, 0x050b, 0x050e, 0x0511, + 0x0533, 0x0533, 0x0540, 0x0555, 0x0800, 0x0808, 0x0810, 0x0813, + 0x0820, 0x0821, 0x0823, 0x0827, 0x0830, 0x0833, 0x0840, 0x0843, + 0x084f, 0x086f, 0x0880, 0x088a, 0x08a0, 0x08ab, 0x08c0, 0x08c4, + 0x08d0, 0x08dd, 0x08f0, 0x08f3, 0x0900, 0x0903, 0x0908, 0x0911, + 0x0928, 0x093e, 0x0942, 0x094d, 0x0980, 0x0984, 0x098d, 0x0996, + 0x0998, 0x099e, 0x09a0, 0x09a6, 0x09a8, 0x09ae, 0x09b0, 0x09b1, + 0x09c2, 0x09c8, 0x0a00, 0x0a03, 0x0c00, 0x0c04, 0x0c06, 0x0c06, + 0x0c10, 0x0cd9, 0x0e00, 0x0e0e, 0x0e10, 0x0e13, 0x0e17, 0x0e19, + 0x0e1c, 0x0e2b, 0x0e30, 0x0e32, 0x0e38, 0x0e39, 0x8600, 0x8601, + 0x8610, 0x861b, 0x8620, 0x8620, 0x8628, 0x862b, 0x8630, 0x8637, + 0x8e01, 0x8e01, 0x8e04, 0x8e05, 0x8e07, 0x8e08, 0x8e0c, 0x8e0c, + 0x8e10, 0x8e1c, 0x8e20, 0x8e25, 0x8e28, 0x8e28, 0x8e2c, 0x8e2f, + 0x8e3b, 0x8e3e, 0x8e40, 0x8e43, 0x8e50, 0x8e5e, 0x8e70, 0x8e77, + 0x9600, 0x9604, 0x9624, 0x9637, 0x9e00, 0x9e01, 0x9e03, 0x9e0e, + 0x9e11, 0x9e16, 0x9e19, 0x9e19, 0x9e1c, 0x9e1c, 0x9e20, 0x9e23, + 0x9e30, 0x9e31, 0x9e34, 0x9e34, 0x9e70, 0x9e72, 0x9e78, 0x9e79, + 0x9e80, 0x9fff, 0xa600, 0xa601, 0xa603, 0xa603, 0xa60a, 0xa60a, + 0xa610, 0xa617, 0xa630, 0xa630, + ~0 +}; + +static int a6xx_pm_resume(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + ret = a6xx_gmu_resume(a6xx_gpu); + + gpu->needs_hw_init = true; + + return ret; +} + +static int a6xx_pm_suspend(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* + * Make sure the GMU is idle before continuing (because some transitions + * may use VBIF + */ + a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Clear the VBIF pipe before shutting down */ + /* FIXME: This accesses the GPU - do we need to make sure it is on? */ + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0xf); + spin_until((gpu_read(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL1) & 0xf) == 0xf); + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0); + + return a6xx_gmu_stop(a6xx_gpu); +} + +static int a6xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Force the GPU power on so we can read this register */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + *value = gpu_read64(gpu, REG_A6XX_RBBM_PERFCTR_CP_0_LO, + REG_A6XX_RBBM_PERFCTR_CP_0_HI); + + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + return 0; +} + +#ifdef CONFIG_DEBUG_FS +static void a6xx_show(struct msm_gpu *gpu, struct seq_file *m) +{ + gpu->funcs->pm_resume(gpu); + + seq_printf(m, "status: %08x\n", + gpu_read(gpu, REG_A6XX_RBBM_STATUS)); + gpu->funcs->pm_suspend(gpu); + + adreno_show(gpu, m); +} +#endif + +static struct msm_ringbuffer *a6xx_active_ring(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + return a6xx_gpu->cur_ring; +} + +static void a6xx_destroy(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (a6xx_gpu->sqe_bo) { + if (a6xx_gpu->sqe_iova) + msm_gem_put_iova(a6xx_gpu->sqe_bo, gpu->aspace); + drm_gem_object_unreference_unlocked(a6xx_gpu->sqe_bo); + } + + a6xx_gmu_remove(a6xx_gpu); + + adreno_gpu_cleanup(adreno_gpu); + kfree(a6xx_gpu); +} + +static const struct adreno_gpu_funcs funcs = { + .base = { + .get_param = adreno_get_param, + .hw_init = a6xx_hw_init, + .pm_suspend = a6xx_pm_suspend, + .pm_resume = a6xx_pm_resume, + .recover = a6xx_recover, + .submit = a6xx_submit, + .flush = a6xx_flush, + .active_ring = a6xx_active_ring, + .irq = a6xx_irq, + .destroy = a6xx_destroy, +#ifdef CONFIG_DEBUG_FS + .show = a6xx_show, +#endif + }, + .get_timestamp = a6xx_get_timestamp, +}; + +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) +{ + struct msm_drm_private *priv = dev->dev_private; + struct platform_device *pdev = priv->gpu_pdev; + struct device_node *node; + struct a6xx_gpu *a6xx_gpu; + struct adreno_gpu *adreno_gpu; + struct msm_gpu *gpu; + int ret; + + a6xx_gpu = kzalloc(sizeof(*a6xx_gpu), GFP_KERNEL); + if (!a6xx_gpu) + return ERR_PTR(-ENOMEM); + + adreno_gpu = &a6xx_gpu->base; + gpu = &adreno_gpu->base; + + adreno_gpu->registers = a6xx_registers; + adreno_gpu->reg_offsets = a6xx_register_offsets; + + ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + /* Check if there is a GMU phandle and set it up */ + node = of_parse_phandle(pdev->dev.of_node, "gmu", 0); + + /* FIXME: How do we gracefully handle this? */ + BUG_ON(!node); + + ret = a6xx_gmu_probe(a6xx_gpu, node); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + if (gpu->aspace) + msm_mmu_set_fault_handler(gpu->aspace->mmu, gpu, + a6xx_fault_handler); + + return gpu; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h new file mode 100644 index 000000000000..dd69e5b0e692 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef __A6XX_GPU_H__ +#define __A6XX_GPU_H__ + + +#include "adreno_gpu.h" +#include "a6xx.xml.h" + +#include "a6xx_gmu.h" + +extern bool hang_debug; + +struct a6xx_gpu { + struct adreno_gpu base; + + struct drm_gem_object *sqe_bo; + uint64_t sqe_iova; + + struct msm_ringbuffer *cur_ring; + + struct a6xx_gmu gmu; +}; + +#define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) + +/* + * Given a register and a count, return a value to program into + * REG_CP_PROTECT_REG(n) - this will block both reads and writes for _len + * registers starting at _reg. + */ +#define A6XX_PROTECT_RW(_reg, _len) \ + ((1 << 31) | \ + (((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + +/* + * Same as above, but allow reads over the range. For areas of mixed use (such + * as performance counters) this allows us to protect a much larger range with a + * single register + */ +#define A6XX_PROTECT_RDONLY(_reg, _len) \ + ((((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + + +int a6xx_gmu_resume(struct a6xx_gpu *gpu); +int a6xx_gmu_stop(struct a6xx_gpu *gpu); + +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *gpu); + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu); +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu); + +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node); +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); + +#endif /* __A6XX_GPU_H__ */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c new file mode 100644 index 000000000000..221eaafc8b5f --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c @@ -0,0 +1,435 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/completion.h> +#include <linux/circ_buf.h> +#include <linux/list.h> + +#include "a6xx_gmu.h" +#include "a6xx_gmu.xml.h" + +#define HFI_MSG_ID(val) [val] = #val + +static const char * const a6xx_hfi_msg_id[] = { + HFI_MSG_ID(HFI_H2F_MSG_INIT), + HFI_MSG_ID(HFI_H2F_MSG_FW_VERSION), + HFI_MSG_ID(HFI_H2F_MSG_BW_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_PERF_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_TEST), +}; + +static int a6xx_hfi_queue_read(struct a6xx_hfi_queue *queue, u32 *data, + u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, hdr, index = header->read_index; + + if (header->read_index == header->write_index) { + header->rx_request = 1; + return 0; + } + + hdr = queue->data[index]; + + /* + * If we are to assume that the GMU firmware is in fact a rational actor + * and is programmed to not send us a larger response than we expect + * then we can also assume that if the header size is unexpectedly large + * that it is due to memory corruption and/or hardware failure. In this + * case the only reasonable course of action is to BUG() to help harden + * the failure. + */ + + BUG_ON(HFI_HEADER_SIZE(hdr) > dwords); + + for (i = 0; i < HFI_HEADER_SIZE(hdr); i++) { + data[i] = queue->data[index]; + index = (index + 1) % header->size; + } + + header->read_index = index; + return HFI_HEADER_SIZE(hdr); +} + +static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu, + struct a6xx_hfi_queue *queue, u32 *data, u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, space, index = header->write_index; + + spin_lock(&queue->lock); + + space = CIRC_SPACE(header->write_index, header->read_index, + header->size); + if (space < dwords) { + header->dropped++; + spin_unlock(&queue->lock); + return -ENOSPC; + } + + for (i = 0; i < dwords; i++) { + queue->data[index] = data[i]; + index = (index + 1) % header->size; + } + + header->write_index = index; + spin_unlock(&queue->lock); + + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 0x01); + return 0; +} + +struct a6xx_hfi_response { + u32 id; + u32 seqnum; + struct list_head node; + struct completion complete; + + u32 error; + u32 payload[16]; +}; + +/* + * Incoming HFI ack messages can come in out of order so we need to store all + * the pending messages on a list until they are handled. + */ +static spinlock_t hfi_ack_lock = __SPIN_LOCK_UNLOCKED(message_lock); +static LIST_HEAD(hfi_ack_list); + +static void a6xx_hfi_handle_ack(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_response *resp; + u32 id, seqnum; + + /* msg->ret_header contains the header of the message being acked */ + id = HFI_HEADER_ID(msg->ret_header); + seqnum = HFI_HEADER_SEQNUM(msg->ret_header); + + spin_lock(&hfi_ack_lock); + list_for_each_entry(resp, &hfi_ack_list, node) { + if (resp->id == id && resp->seqnum == seqnum) { + resp->error = msg->error; + memcpy(resp->payload, msg->payload, + sizeof(resp->payload)); + + complete(&resp->complete); + spin_unlock(&hfi_ack_lock); + return; + } + } + spin_unlock(&hfi_ack_lock); + + dev_err(gmu->dev, "Nobody was waiting for HFI message %d\n", seqnum); +} + +static void a6xx_hfi_handle_error(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_msg_error *error = (struct a6xx_hfi_msg_error *) msg; + + dev_err(gmu->dev, "GMU firmware error %d\n", error->code); +} + +void a6xx_hfi_task(unsigned long data) +{ + struct a6xx_gmu *gmu = (struct a6xx_gmu *) data; + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_RESPONSE_QUEUE]; + struct a6xx_hfi_msg_response resp; + + for (;;) { + u32 id; + int ret = a6xx_hfi_queue_read(queue, (u32 *) &resp, + sizeof(resp) >> 2); + + /* Returns the number of bytes copied or negative on error */ + if (ret <= 0) { + if (ret < 0) + dev_err(gmu->dev, + "Unable to read the HFI message queue\n"); + break; + } + + id = HFI_HEADER_ID(resp.header); + + if (id == HFI_F2H_MSG_ACK) + a6xx_hfi_handle_ack(gmu, &resp); + else if (id == HFI_F2H_MSG_ERROR) + a6xx_hfi_handle_error(gmu, &resp); + } +} + +static int a6xx_hfi_send_msg(struct a6xx_gmu *gmu, int id, + void *data, u32 size, u32 *payload, u32 payload_size) +{ + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_COMMAND_QUEUE]; + struct a6xx_hfi_response resp = { 0 }; + int ret, dwords = size >> 2; + u32 seqnum; + + seqnum = atomic_inc_return(&queue->seqnum) % 0xfff; + + /* First dword of the message is the message header - fill it in */ + *((u32 *) data) = (seqnum << 20) | (HFI_MSG_CMD << 16) | + (dwords << 8) | id; + + init_completion(&resp.complete); + resp.id = id; + resp.seqnum = seqnum; + + spin_lock_bh(&hfi_ack_lock); + list_add_tail(&resp.node, &hfi_ack_list); + spin_unlock_bh(&hfi_ack_lock); + + ret = a6xx_hfi_queue_write(gmu, queue, data, dwords); + if (ret) { + dev_err(gmu->dev, "Unable to send message %s id %d\n", + a6xx_hfi_msg_id[id], seqnum); + goto out; + } + + /* Wait up to 5 seconds for the response */ + ret = wait_for_completion_timeout(&resp.complete, + msecs_to_jiffies(5000)); + if (!ret) { + dev_err(gmu->dev, + "Message %s id %d timed out waiting for response\n", + a6xx_hfi_msg_id[id], seqnum); + ret = -ETIMEDOUT; + } else + ret = 0; + +out: + spin_lock_bh(&hfi_ack_lock); + list_del(&resp.node); + spin_unlock_bh(&hfi_ack_lock); + + if (ret) + return ret; + + if (resp.error) { + dev_err(gmu->dev, "Message %s id %d returned error %d\n", + a6xx_hfi_msg_id[id], seqnum, resp.error); + return -EINVAL; + } + + if (payload && payload_size) { + int copy = min_t(u32, payload_size, sizeof(resp.payload)); + + memcpy(payload, resp.payload, copy); + } + + return 0; +} + +static int a6xx_hfi_send_gmu_init(struct a6xx_gmu *gmu, int boot_state) +{ + struct a6xx_hfi_msg_gmu_init_cmd msg = { 0 }; + + msg.dbg_buffer_addr = (u32) gmu->debug->iova; + msg.dbg_buffer_size = (u32) gmu->debug->size; + msg.boot_state = boot_state; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_INIT, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_get_fw_version(struct a6xx_gmu *gmu, u32 *version) +{ + struct a6xx_hfi_msg_fw_version msg = { 0 }; + + /* Currently supporting version 1.1 */ + msg.supported_version = (1 << 28) | (1 << 16); + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_FW_VERSION, &msg, sizeof(msg), + version, sizeof(*version)); +} + +static int a6xx_hfi_send_perf_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_perf_table msg = { 0 }; + int i; + + msg.num_gpu_levels = gmu->nr_gpu_freqs; + msg.num_gmu_levels = gmu->nr_gmu_freqs; + + for (i = 0; i < gmu->nr_gpu_freqs; i++) { + msg.gx_votes[i].vote = gmu->gx_arc_votes[i]; + msg.gx_votes[i].vote = gmu->gpu_freqs[i] / 1000; + } + + for (i = 0; i < gmu->nr_gmu_freqs; i++) { + msg.cx_votes[i].vote = gmu->cx_arc_votes[i]; + msg.cx_votes[i].vote = gmu->gmu_freqs[i] / 1000; + } + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_PERF_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_bw_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_bw_table msg = { 0 }; + + /* + * The sdm845 GMU doesn't do bus frequency scaling on its own but it + * does need at least one entry in the list because it might be accessed + * when the GMU is shutting down. Send a single "off" entry. + */ + + msg.bw_level_num = 1; + + msg.ddr_cmds_num = 3; + msg.ddr_wait_bitmask = 0x07; + + msg.ddr_cmds_addrs[0] = 0x50000; + msg.ddr_cmds_addrs[1] = 0x5005c; + msg.ddr_cmds_addrs[2] = 0x5000c; + + msg.ddr_cmds_data[0][0] = 0x40000000; + msg.ddr_cmds_data[0][1] = 0x40000000; + msg.ddr_cmds_data[0][2] = 0x40000000; + + /* + * These are the CX (CNOC) votes. This is used but the values for the + * sdm845 GMU are known and fixed so we can hard code them. + */ + + msg.cnoc_cmds_num = 3; + msg.cnoc_wait_bitmask = 0x05; + + msg.cnoc_cmds_addrs[0] = 0x50034; + msg.cnoc_cmds_addrs[1] = 0x5007c; + msg.cnoc_cmds_addrs[2] = 0x5004c; + + msg.cnoc_cmds_data[0][0] = 0x40000000; + msg.cnoc_cmds_data[0][1] = 0x00000000; + msg.cnoc_cmds_data[0][2] = 0x40000000; + + msg.cnoc_cmds_data[1][0] = 0x60000001; + msg.cnoc_cmds_data[1][1] = 0x20000001; + msg.cnoc_cmds_data[1][2] = 0x60000001; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_BW_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_test(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_test msg = { 0 }; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_TEST, &msg, sizeof(msg), + NULL, 0); +} + +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state) +{ + int ret; + + ret = a6xx_hfi_send_gmu_init(gmu, boot_state); + if (ret) + return ret; + + ret = a6xx_hfi_get_fw_version(gmu, NULL); + if (ret) + return ret; + + /* + * We have to get exchange version numbers per the sequence but at this + * point th kernel driver doesn't need to know the exact version of + * the GMU firmware + */ + + ret = a6xx_hfi_send_perf_table(gmu); + if (ret) + return ret; + + ret = a6xx_hfi_send_bw_table(gmu); + if (ret) + return ret; + + /* + * Let the GMU know that there won't be any more HFI messages until next + * boot + */ + a6xx_hfi_send_test(gmu); + + return 0; +} + +void a6xx_hfi_stop(struct a6xx_gmu *gmu) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(gmu->queues); i++) { + struct a6xx_hfi_queue *queue = &gmu->queues[i]; + + if (!queue->header) + continue; + + if (queue->header->read_index != queue->header->write_index) + dev_err(gmu->dev, "HFI queue %d is not empty\n", i); + + queue->header->read_index = 0; + queue->header->write_index = 0; + } +} + +static void a6xx_hfi_queue_init(struct a6xx_hfi_queue *queue, + struct a6xx_hfi_queue_header *header, void *virt, u64 iova, + u32 id) +{ + spin_lock_init(&queue->lock); + queue->header = header; + queue->data = virt; + atomic_set(&queue->seqnum, 0); + + /* Set up the shared memory header */ + header->iova = iova; + header->type = 10 << 8 | id; + header->status = 1; + header->size = SZ_4K >> 2; + header->msg_size = 0; + header->dropped = 0; + header->rx_watermark = 1; + header->tx_watermark = 1; + header->rx_request = 1; + header->tx_request = 0; + header->read_index = 0; + header->write_index = 0; +} + +void a6xx_hfi_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gmu_bo *hfi = gmu->hfi; + struct a6xx_hfi_queue_table_header *table = hfi->virt; + struct a6xx_hfi_queue_header *headers = hfi->virt + sizeof(*table); + u64 offset; + int table_size; + + /* + * The table size is the size of the table header plus all of the queue + * headers + */ + table_size = sizeof(*table); + table_size += (ARRAY_SIZE(gmu->queues) * + sizeof(struct a6xx_hfi_queue_header)); + + table->version = 0; + table->size = table_size; + /* First queue header is located immediately after the table header */ + table->qhdr0_offset = sizeof(*table) >> 2; + table->qhdr_size = sizeof(struct a6xx_hfi_queue_header) >> 2; + table->num_queues = ARRAY_SIZE(gmu->queues); + table->active_queues = ARRAY_SIZE(gmu->queues); + + /* Command queue */ + offset = SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[0], &headers[0], hfi->virt + offset, + hfi->iova + offset, 0); + + /* GMU response queue */ + offset += SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[1], &headers[1], hfi->virt + offset, + hfi->iova + offset, 4); +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.h b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h new file mode 100644 index 000000000000..60d1319fa44f --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_HFI_H_ +#define _A6XX_HFI_H_ + +struct a6xx_hfi_queue_table_header { + u32 version; + u32 size; /* Size of the queue table in dwords */ + u32 qhdr0_offset; /* Offset of the first queue header */ + u32 qhdr_size; /* Size of the queue headers */ + u32 num_queues; /* Number of total queues */ + u32 active_queues; /* Number of active queues */ +}; + +struct a6xx_hfi_queue_header { + u32 status; + u32 iova; + u32 type; + u32 size; + u32 msg_size; + u32 dropped; + u32 rx_watermark; + u32 tx_watermark; + u32 rx_request; + u32 tx_request; + u32 read_index; + u32 write_index; +}; + +struct a6xx_hfi_queue { + struct a6xx_hfi_queue_header *header; + spinlock_t lock; + u32 *data; + atomic_t seqnum; +}; + +/* This is the outgoing queue to the GMU */ +#define HFI_COMMAND_QUEUE 0 + +/* THis is the incoming response queue from the GMU */ +#define HFI_RESPONSE_QUEUE 1 + +#define HFI_HEADER_ID(msg) ((msg) & 0xff) +#define HFI_HEADER_SIZE(msg) (((msg) >> 8) & 0xff) +#define HFI_HEADER_SEQNUM(msg) (((msg) >> 20) & 0xfff) + +/* FIXME: Do we need this or can we use ARRAY_SIZE? */ +#define HFI_RESPONSE_PAYLOAD_SIZE 16 + +/* HFI message types */ + +#define HFI_MSG_CMD 0 +#define HFI_MSG_ACK 2 + +#define HFI_F2H_MSG_ACK 126 + +struct a6xx_hfi_msg_response { + u32 header; + u32 ret_header; + u32 error; + u32 payload[HFI_RESPONSE_PAYLOAD_SIZE]; +}; + +#define HFI_F2H_MSG_ERROR 100 + +struct a6xx_hfi_msg_error { + u32 header; + u32 code; + u32 payload[2]; +}; + +#define HFI_H2F_MSG_INIT 0 + +struct a6xx_hfi_msg_gmu_init_cmd { + u32 header; + u32 seg_id; + u32 dbg_buffer_addr; + u32 dbg_buffer_size; + u32 boot_state; +}; + +#define HFI_H2F_MSG_FW_VERSION 1 + +struct a6xx_hfi_msg_fw_version { + u32 header; + u32 supported_version; +}; + +#define HFI_H2F_MSG_PERF_TABLE 4 + +struct perf_level { + u32 vote; + u32 freq; +}; + +struct a6xx_hfi_msg_perf_table { + u32 header; + u32 num_gpu_levels; + u32 num_gmu_levels; + + struct perf_level gx_votes[16]; + struct perf_level cx_votes[4]; +}; + +#define HFI_H2F_MSG_BW_TABLE 3 + +struct a6xx_hfi_msg_bw_table { + u32 header; + u32 bw_level_num; + u32 cnoc_cmds_num; + u32 ddr_cmds_num; + u32 cnoc_wait_bitmask; + u32 ddr_wait_bitmask; + u32 cnoc_cmds_addrs[6]; + u32 cnoc_cmds_data[2][6]; + u32 ddr_cmds_addrs[8]; + u32 ddr_cmds_data[16][8]; +}; + +#define HFI_H2F_MSG_TEST 5 + +struct a6xx_hfi_msg_test { + u32 header; +}; + +#endif diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index d70e7d145dae..a6a9721d82fe 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -111,6 +111,16 @@ static const struct adreno_info gpulist[] = { ADRENO_QUIRK_FAULT_DETECT_MASK, .init = a5xx_gpu_init, .zapfw = "a530_zap.mdt", + }, { + .rev = ADRENO_REV(6, 3, 0, ANY_ID), + .revn = 630, + .name = "A630", + .fw = { + [ADRENO_FW_SQE] = "a630_sqe.fw", + [ADRENO_FW_GMU] = "a630_gmu.bin", + }, + .gmem = SZ_1M, + .init = a6xx_gpu_init, }, }; @@ -127,6 +137,8 @@ MODULE_FIRMWARE("qcom/a530_zap.mdt"); MODULE_FIRMWARE("qcom/a530_zap.b00"); MODULE_FIRMWARE("qcom/a530_zap.b01"); MODULE_FIRMWARE("qcom/a530_zap.b02"); +MODULE_FIRMWARE("qcom/a630_sqe.fw"); +MODULE_FIRMWARE("qcom/a630_gmu.bin"); static inline bool _rev_match(uint8_t entry, uint8_t id) { diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c index bcbf9f2a29f9..ae693c28d66c 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c @@ -138,7 +138,7 @@ adreno_request_fw(struct adreno_gpu *adreno_gpu, const char *fwname) return ERR_PTR(-ENOENT); } -static int adreno_load_fw(struct adreno_gpu *adreno_gpu) +int adreno_load_fw(struct adreno_gpu *adreno_gpu) { int i; diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index bc9ec27e9ed8..3e0c03d2e91a 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -50,7 +50,9 @@ enum adreno_regs { enum { ADRENO_FW_PM4 = 0, + ADRENO_FW_SQE = 0, /* a6xx */ ADRENO_FW_PFP = 1, + ADRENO_FW_GMU = 1, /* a6xx */ ADRENO_FW_GPMU = 2, ADRENO_FW_MAX, }; @@ -227,6 +229,7 @@ int adreno_gpu_init(struct drm_device *drm, struct platform_device *pdev, struct adreno_gpu *gpu, const struct adreno_gpu_funcs *funcs, int nr_rings); void adreno_gpu_cleanup(struct adreno_gpu *gpu); +int adreno_load_fw(struct adreno_gpu *adreno_gpu); /* ringbuffer helpers (the parts that are adreno specific) */ @@ -329,6 +332,7 @@ static inline void adreno_gpu_write(struct adreno_gpu *gpu, struct msm_gpu *a3xx_gpu_init(struct drm_device *dev); struct msm_gpu *a4xx_gpu_init(struct drm_device *dev); struct msm_gpu *a5xx_gpu_init(struct drm_device *dev); +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev); static inline void adreno_gpu_write64(struct adreno_gpu *gpu, enum adreno_regs lo, enum adreno_regs hi, u64 data) diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index d4d5a68e468d..b66d45c41443 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -87,7 +87,7 @@ static struct devfreq_dev_profile msm_devfreq_profile = { static void msm_devfreq_init(struct msm_gpu *gpu) { /* We need target support to do devfreq */ - if (!gpu->funcs->gpu_busy) + if (!gpu->funcs->gpu_busy || !gpu->core_clk) return; msm_devfreq_profile.initial_freq = gpu->fast_rate; -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [v6 PATCH 0/5] drm/msm: Add support for Adreno a6xx
@ 2018-06-19 22:30 Jordan Crouse
[not found] ` <20180619223006.31519-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Jordan Crouse @ 2018-06-19 22:30 UTC (permalink / raw)
To: freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
This is an initial version of support for the Adreno a6xx GPU family starting
with the a630 from the sdm845 SoC. This code is ahead of much of the sdm845
code that would be needed to actually bring up a device and it is also in
advance of any user side support for the a6xx GPU so this is mainly just a
chance to look over the code structure and get a feel for the
direction that the hardware is going in.
The a6xx GPU is an iteration of a5xx so most of the GPU side code looks pretty
identical except for the usual register differences. The big different is in
power control. On the a5xx there was a rudimentary device called the GPMU that
did some basic power stuff but left most of the complexity to the kernel.
On the a6xx the power complexity is being moved to a component
called the GMU (graphics management unit) which handles the power control of the
GPU and shuts it down quickly after it goes idle.
This stack has a number of dependencies that are in various stages of being
merged:
https://patchwork.kernel.org/patch/10286375/ - dev_pm_opp_get_of_node()
https://patchwork.kernel.org/patch/10333575/ - command DB
https://patchwork.kernel.org/patch/10301163/ - pm_runtime ops for arm-smmu
[v6 - set nr_clocks to 0 when parsing fails and minor cleanup in msm_iommu for
Vivek. Fix compile error in the load firmware patch ]
[v5 - Move firmware loading to adreno_load_gpu() so it is easier to fail if the
firmware isn't found ]
[v4 - Add clock parsing helper function and use it for GPU and GMU. Update
recommended clock gating settings. Fix a bug in the CMD DB parser. Update
register values from updated database ]
[v3 - fix inverted register definition for GMU_SPTPRAC_CLK_STATUS; fix incorrect
register check in a5xx_gmu_gx_is_on(), use dev_pm_opp_get_of_node() from
Rajendra and Viresh to read the qcom,level from the device tree; read qcom,level
from the DT to get the voltage level to pass to the GMU, fix issues
identified by smatch]
[v2 - addressed comments from Lucas Stach; added pm_runtime_get_supplier calls
for accesses to the GMU IOMMU; moved to SPDX headers for new files]
Jordan Crouse (4):
drm/msm: Remove pm_runtime operations from msm_iommu
drm/msm: Add a helper function to parse clock names
drm/msm/adreno: Load the firmware before bringing up the hardware
drm/msm: Add A6XX device support
Sharat Masetty (1):
drm/msm: Add generated headers for A6XX
drivers/gpu/drm/msm/Makefile | 3 +
drivers/gpu/drm/msm/adreno/a6xx.xml.h | 1784 ++++++++++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1172 +++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 ++
drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h | 382 +++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 807 +++++++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 +
drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 +++++
drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 ++
drivers/gpu/drm/msm/adreno/adreno_device.c | 35 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 +-
drivers/gpu/drm/msm/msm_drv.c | 57 +
drivers/gpu/drm/msm/msm_drv.h | 4 +
drivers/gpu/drm/msm/msm_gpu.c | 60 +-
drivers/gpu/drm/msm/msm_gpu.h | 2 +-
drivers/gpu/drm/msm/msm_iommu.c | 13 +-
17 files changed, 5045 insertions(+), 65 deletions(-)
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h
--
2.17.1
_______________________________________________
Freedreno mailing list
Freedreno@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/freedreno
^ permalink raw reply [flat|nested] 12+ messages in thread[parent not found: <20180619223006.31519-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>]
* [PATCH 5/5] drm/msm: Add A6XX device support [not found] ` <20180619223006.31519-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> @ 2018-06-19 22:30 ` Jordan Crouse 0 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-06-19 22:30 UTC (permalink / raw) To: freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Add support for the A6XX family of Adreno GPUs. The biggest addition is the GMU (Graphics Management Unit) which takes over most of the power management of the GPU itself but in a ironic twist of fate needs a goodly amount of management itself. Add support for the A6XX core code, the GMU and the HFI (hardware firmware interface) queue that the CPU uses to communicate with the GMU. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/Makefile | 3 + drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1172 ++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 +++ drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 807 ++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 + drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 ++++++++ drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 +++ drivers/gpu/drm/msm/adreno/adreno_device.c | 12 + drivers/gpu/drm/msm/adreno/adreno_gpu.h | 3 + drivers/gpu/drm/msm/msm_gpu.c | 2 +- 10 files changed, 2782 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index cd40c050b2d7..4affc665c0de 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -10,6 +10,9 @@ msm-y := \ adreno/a5xx_gpu.o \ adreno/a5xx_power.o \ adreno/a5xx_preempt.o \ + adreno/a6xx_gpu.o \ + adreno/a6xx_gmu.o \ + adreno/a6xx_hfi.o \ hdmi/hdmi.o \ hdmi/hdmi_audio.o \ hdmi/hdmi_bridge.o \ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c new file mode 100644 index 000000000000..9ecd7afe6321 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -0,0 +1,1172 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/clk.h> +#include <linux/iopoll.h> +#include <linux/pm_opp.h> +#include <soc/qcom/cmd-db.h> + +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static irqreturn_t a6xx_gmu_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_STATUS); + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, status); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE) { + dev_err_ratelimited(gmu->dev, "GMU watchdog expired\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR) + dev_err_ratelimited(gmu->dev, "GMU AHB bus error\n"); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + dev_err_ratelimited(gmu->dev, "GMU fence error: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_AHB_FENCE_STATUS)); + + return IRQ_HANDLED; +} + +static irqreturn_t a6xx_hfi_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, status); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ) + tasklet_schedule(&gmu->hfi_tasklet); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) { + dev_err_ratelimited(gmu->dev, "GMU firmware fault\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + return IRQ_HANDLED; +} + +/* Check to see if the GX rail is still powered */ +static bool a6xx_gmu_gx_is_on(struct a6xx_gmu *gmu) +{ + u32 val = gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS); + + return !(val & + (A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_GDSC_POWER_OFF | + A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_CLK_OFF)); +} + +static bool a6xx_gmu_check_idle_level(struct a6xx_gmu *gmu) +{ + u32 val; + int local = gmu->idle_level; + + /* SPTP and IFPC both report as IFPC */ + if (gmu->idle_level == GMU_IDLE_STATE_SPTP) + local = GMU_IDLE_STATE_IFPC; + + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val == local) { + if (gmu->idle_level != GMU_IDLE_STATE_IFPC || + !a6xx_gmu_gx_is_on(gmu)) + return true; + } + + return false; +} + +/* Wait for the GMU to get to its most idle state */ +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + return spin_until(a6xx_gmu_check_idle_level(gmu)); +} + +static int a6xx_gmu_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 1); + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 0); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, val, + val == 0xbabeface, 100, 10000); + + if (ret) + dev_err(gmu->dev, "GMU firmware initalization timed out\n"); + + return ret; +} + +static int a6xx_gmu_hfi_start(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + gmu_rmw(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ, 0); + + gmu_write(gmu, REG_A6XX_GMU_HFI_CTRL_INIT, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_HFI_CTRL_STATUS, val, + val & 1, 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to start the HFI queues\n"); + + return ret; +} + +/* Trigger a OOB (out of band) request to the GMU */ +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + int ret; + u32 val; + int request, ack; + const char *name; + + switch (state) { + case GMU_OOB_GPU_SET: + request = GMU_OOB_GPU_SET_REQUEST; + ack = GMU_OOB_GPU_SET_ACK; + name = "GPU_SET"; + break; + case GMU_OOB_BOOT_SLUMBER: + request = GMU_OOB_BOOT_SLUMBER_REQUEST; + ack = GMU_OOB_BOOT_SLUMBER_ACK; + name = "BOOT_SLUMBER"; + break; + case GMU_OOB_DCVS_SET: + request = GMU_OOB_DCVS_REQUEST; + ack = GMU_OOB_DCVS_ACK; + name = "GPU_DCVS"; + break; + default: + return -EINVAL; + } + + /* Trigger the equested OOB operation */ + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 1 << request); + + /* Wait for the acknowledge interrupt */ + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO, val, + val & (1 << ack), 100, 10000); + + if (ret) + dev_err(gmu->dev, + "Timeout waiting for GMU OOB set %s: 0x%x\n", + name, + gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO)); + + /* Clear the acknowledge interrupt */ + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, 1 << ack); + + return ret; +} + +/* Clear a pending OOB state in the GMU */ +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + switch (state) { + case GMU_OOB_GPU_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_GPU_SET_CLEAR); + break; + case GMU_OOB_BOOT_SLUMBER: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_BOOT_SLUMBER_CLEAR); + break; + case GMU_OOB_DCVS_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_DCVS_CLEAR); + break; + } +} + +/* Enable CPU control of SPTP power power collapse */ +static int a6xx_sptprac_enable(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778000); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x38) == 0x28, 1, 100); + + if (ret) { + dev_err(gmu->dev, "Unable to power on SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); + } + + return 0; +} + +/* Disable CPU control of SPTP power power collapse */ +static void a6xx_sptprac_disable(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + /* Make sure retention is on */ + gmu_rmw(gmu, REG_A6XX_GPU_CC_GX_GDSCR, 0, (1 << 11)); + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778001); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x04), 100, 10000); + + if (ret) + dev_err(gmu->dev, "failed to power off SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); +} + +/* Let the GMU know we are starting a boot sequence */ +static int a6xx_gmu_gfx_rail_on(struct a6xx_gmu *gmu) +{ + u32 vote; + + /* Let the GMU know we are getting ready for boot */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 0); + + /* Choose the "default" power level as the highest available */ + vote = gmu->gx_arc_votes[gmu->nr_gpu_freqs - 1]; + + gmu_write(gmu, REG_A6XX_GMU_GX_VOTE_IDX, vote & 0xff); + gmu_write(gmu, REG_A6XX_GMU_MX_VOTE_IDX, (vote >> 8) & 0xff); + + /* Let the GMU know the boot sequence has started */ + return a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); +} + +/* Let the GMU know that we are about to go into slumber */ +static int a6xx_gmu_notify_slumber(struct a6xx_gmu *gmu) +{ + int ret; + + /* Disable the power counter so the GMU isn't busy */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 0); + + /* Disable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) + a6xx_sptprac_disable(gmu); + + /* Tell the GMU to get ready to slumber */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 1); + + ret = a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + if (!ret) { + /* Check to see if the GMU really did slumber */ + if (gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE) + != 0x0f) { + dev_err(gmu->dev, "The GMU did not go into slumber\n"); + ret = -ETIMEDOUT; + } + } + + /* Put fence into allow mode */ + gmu_write(gmu, REG_A6XX_GMU_AO_AHB_FENCE_CTRL, 0); + return ret; +} + +static int a6xx_rpmh_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1 << 1); + /* Wait for the register to finish posting */ + wmb(); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_RSCC_CONTROL_ACK, val, + val & (1 << 1), 100, 10000); + if (ret) { + dev_err(gmu->dev, "Unable to power on the GPU RSC\n"); + return ret; + } + + ret = gmu_poll_timeout(gmu, REG_A6XX_RSCC_SEQ_BUSY_DRV0, val, + !val, 100, 10000); + + if (!ret) { + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); + + /* Re-enable the power counter */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 1); + return 0; + } + + dev_err(gmu->dev, "GPU RSC sequence stuck while waking up the GPU\n"); + return ret; +} + +static void a6xx_rpmh_stop(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, + val, val & (1 << 16), 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to power off the GPU RSC\n"); + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); +} + +static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) +{ + /* Disable SDE clock gating */ + gmu_write(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, BIT(24)); + + /* Setup RSC PDC handshake for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_PDC_SLAVE_ID_DRV0, 1); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 4, 0x80000000); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 4, 0); + gmu_write(gmu, REG_A6XX_RSCC_OVERRIDE_START_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_PDC_SEQ_START_ADDR, 0x4520); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_LO, 0x4510); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_HI, 0x4514); + + /* Load RSC sequencer uCode for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0, 0xa7a506a0); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 1, 0xa1e6a6e7); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 2, 0xa2e081e1); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 3, 0xe9a982e2); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 4, 0x0020e8a8); + + /* Load PDC sequencer uCode for power up and power down sequence */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0, 0xfebea1e1); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 1, 0xa5a4a3a2); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 2, 0x8382a6e0); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 3, 0xbce3e284); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 4, 0x002081fc); + + /* Set TCS commands used by PDC sequence for low power modes */ + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA, 1); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 4, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 8, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA, 2); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 4, 0x3); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 8, 0x3); + + /* Setup GPU PDC */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_START_ADDR, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_ENABLE_PDC, 0x80000001); + + /* ensure no writes happen before the uCode is fully written */ + wmb(); +} + +/* + * The lowest 16 bits of this value are the number of XO clock cycles for main + * hysteresis which is set at 0x1680 cycles (300 us). The higher 16 bits are + * for the shorter hysteresis that happens after main - this is 0xa (.5 us) + */ + +#define GMU_PWR_COL_HYST 0x000a1680 + +/* Set up the idle state for the GMU */ +static void a6xx_gmu_power_config(struct a6xx_gmu *gmu) +{ + /* Disable GMU WB/RB buffer */ + gmu_write(gmu, REG_A6XX_GMU_SYS_BUS_CONFIG, 0x1); + + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0x9c40400); + + switch (gmu->idle_level) { + case GMU_IDLE_STATE_IFPC: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_HM_POWER_COLLAPSE_ENABLE); + /* Fall through */ + case GMU_IDLE_STATE_SPTP: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_SPTPRAC_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_SPTPRAC_POWER_CONTROL_ENABLE); + } + + /* Enable RPMh GPU client */ + gmu_rmw(gmu, REG_A6XX_GMU_RPMH_CTRL, 0, + A6XX_GMU_RPMH_CTRL_RPMH_INTERFACE_ENABLE | + A6XX_GMU_RPMH_CTRL_LLC_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_DDR_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_MX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_CX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_GFX_VOTE_ENABLE); +} + +static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) +{ + static bool rpmh_init; + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + int i, ret; + u32 chipid; + u32 *image; + + if (state == GMU_WARM_BOOT) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } else { + if (WARN(!adreno_gpu->fw[ADRENO_FW_GMU], "GMU firmware is not loaded\n")) + return -ENOENT; + + /* Sanity check the size of the firmware that was loaded */ + if (adreno_gpu->fw[ADRENO_FW_GMU]->size > 0x8000) { + dev_err(gmu->dev, + "GMU firmware is bigger than the available region\n"); + return -EINVAL; + } + + /* Turn on register retention */ + gmu_write(gmu, REG_A6XX_GMU_GENERAL_7, 1); + + /* We only need to load the RPMh microcode once */ + if (!rpmh_init) { + a6xx_gmu_rpmh_init(gmu); + rpmh_init = true; + } else if (state != GMU_RESET) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } + + image = (u32 *) adreno_gpu->fw[ADRENO_FW_GMU]->data; + + for (i = 0; i < adreno_gpu->fw[ADRENO_FW_GMU]->size >> 2; i++) + gmu_write(gmu, REG_A6XX_GMU_CM3_ITCM_START + i, + image[i]); + } + + gmu_write(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, 0); + gmu_write(gmu, REG_A6XX_GMU_CM3_BOOT_CONFIG, 0x02); + + /* Write the iova of the HFI table */ + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_ADDR, gmu->hfi->iova); + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_INFO, 1); + + gmu_write(gmu, REG_A6XX_GMU_AHB_FENCE_RANGE_0, + (1 << 31) | (0xa << 18) | (0xa0)); + + chipid = adreno_gpu->rev.core << 24; + chipid |= adreno_gpu->rev.major << 16; + chipid |= adreno_gpu->rev.minor << 12; + chipid |= adreno_gpu->rev.patchid << 8; + + gmu_write(gmu, REG_A6XX_GMU_HFI_SFR_ADDR, chipid); + + /* Set up the lowest idle level on the GMU */ + a6xx_gmu_power_config(gmu); + + ret = a6xx_gmu_start(gmu); + if (ret) + return ret; + + ret = a6xx_gmu_gfx_rail_on(gmu); + if (ret) + return ret; + + /* Enable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) { + ret = a6xx_sptprac_enable(gmu); + if (ret) + return ret; + } + + ret = a6xx_gmu_hfi_start(gmu); + if (ret) + return ret; + + /* FIXME: Do we need this wmb() here? */ + wmb(); + + return 0; +} + +#define A6XX_HFI_IRQ_MASK \ + (A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ | \ + A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) + +#define A6XX_GMU_IRQ_MASK \ + (A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + +static void a6xx_gmu_irq_enable(struct a6xx_gmu *gmu) +{ + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, ~0); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, + ~A6XX_GMU_IRQ_MASK); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + ~A6XX_HFI_IRQ_MASK); + + enable_irq(gmu->gmu_irq); + enable_irq(gmu->hfi_irq); +} + +static void a6xx_gmu_irq_disable(struct a6xx_gmu *gmu) +{ + disable_irq(gmu->gmu_irq); + disable_irq(gmu->hfi_irq); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, ~0); +} + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int ret; + u32 val; + + /* Flush all the queues */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts */ + a6xx_gmu_irq_disable(gmu); + + /* Force off SPTP in case the GMU is managing it */ + a6xx_sptprac_disable(gmu); + + /* Make sure there are no outstanding RPMh votes */ + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS0_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS1_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS2_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS3_DRV0_STATUS, val, + (val & 1), 100, 1000); + + /* Force off the GX GSDC */ + regulator_force_disable(gmu->gx); + + /* Disable the resources */ + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + pm_runtime_put_sync(gmu->dev); + + /* Re-enable the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + + a6xx_gmu_irq_enable(gmu); + + ret = a6xx_gmu_fw_start(gmu, GMU_RESET); + if (!ret) + ret = a6xx_hfi_start(gmu, GMU_COLD_BOOT); + + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int status, ret; + + if (WARN(!gmu->mmio, "The GMU is not set up yet\n")) + return 0; + + /* Turn on the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + + a6xx_gmu_irq_enable(gmu); + + /* Check to see if we are doing a cold or warm boot */ + status = gmu_read(gmu, REG_A6XX_GMU_GENERAL_7) == 1 ? + GMU_WARM_BOOT : GMU_COLD_BOOT; + + ret = a6xx_gmu_fw_start(gmu, status); + if (ret) + goto out; + + ret = a6xx_hfi_start(gmu, status); + +out: + /* Make sure to turn off the boot OOB request on error */ + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu) +{ + u32 reg; + + if (!gmu->mmio) + return true; + + reg = gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS); + + if (reg & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB) + return false; + + return true; +} + +int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + u32 val; + + /* + * The GMU may still be in slumber unless the GPU started so check and + * skip putting it back into slumber if so + */ + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val != 0xf) { + int ret = a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Temporary until we can recover safely */ + BUG_ON(ret); + + /* tell the GMU we want to slumber */ + a6xx_gmu_notify_slumber(gmu); + + ret = gmu_poll_timeout(gmu, + REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS, val, + !(val & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB), + 100, 10000); + + /* + * Let the user know we failed to slumber but don't worry too + * much because we are powering down anyway + */ + + if (ret) + dev_err(gmu->dev, + "Unable to slumber GMU: status = 0%x/0%x\n", + gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS), + gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS2)); + } + + /* Turn off HFI */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts and mask the hardware */ + a6xx_gmu_irq_disable(gmu); + + /* Tell RPMh to power off the GPU */ + a6xx_rpmh_stop(gmu); + + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + + pm_runtime_put_sync(gmu->dev); + + return 0; +} + +static void a6xx_gmu_memory_free(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo) +{ + int count, i; + u64 iova; + + if (IS_ERR_OR_NULL(bo)) + return; + + count = bo->size >> PAGE_SHIFT; + iova = bo->iova; + + for (i = 0; i < count; i++, iova += PAGE_SIZE) { + iommu_unmap(gmu->domain, iova, PAGE_SIZE); + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); +} + +static struct a6xx_gmu_bo *a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, + size_t size) +{ + struct a6xx_gmu_bo *bo; + int ret, count, i; + + bo = kzalloc(sizeof(*bo), GFP_KERNEL); + if (!bo) + return ERR_PTR(-ENOMEM); + + bo->size = PAGE_ALIGN(size); + + count = bo->size >> PAGE_SHIFT; + + bo->pages = kcalloc(count, sizeof(struct page *), GFP_KERNEL); + if (!bo->pages) { + kfree(bo); + return ERR_PTR(-ENOMEM); + } + + for (i = 0; i < count; i++) { + bo->pages[i] = alloc_page(GFP_KERNEL); + if (!bo->pages[i]) + goto err; + } + + bo->iova = gmu->uncached_iova_base; + + for (i = 0; i < count; i++) { + ret = iommu_map(gmu->domain, + bo->iova + (PAGE_SIZE * i), + page_to_phys(bo->pages[i]), PAGE_SIZE, + IOMMU_READ | IOMMU_WRITE); + + if (ret) { + dev_err(gmu->dev, "Unable to map GMU buffer object\n"); + + for (i = i - 1 ; i >= 0; i--) + iommu_unmap(gmu->domain, + bo->iova + (PAGE_SIZE * i), + PAGE_SIZE); + + goto err; + } + } + + bo->virt = vmap(bo->pages, count, VM_IOREMAP, + pgprot_writecombine(PAGE_KERNEL)); + if (!bo->virt) + goto err; + + /* Align future IOVA addresses on 1MB boundaries */ + gmu->uncached_iova_base += ALIGN(size, SZ_1M); + + return bo; + +err: + for (i = 0; i < count; i++) { + if (bo->pages[i]) + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); + + return ERR_PTR(-ENOMEM); +} + +static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu) +{ + int ret; + + /* + * The GMU address space is hardcoded to treat the range + * 0x60000000 - 0x80000000 as un-cached memory. All buffers shared + * between the GMU and the CPU will live in this space + */ + gmu->uncached_iova_base = 0x60000000; + + + gmu->domain = iommu_domain_alloc(&platform_bus_type); + if (!gmu->domain) + return -ENODEV; + + ret = iommu_attach_device(gmu->domain, gmu->dev); + + if (ret) { + iommu_domain_free(gmu->domain); + gmu->domain = NULL; + } + + return ret; +} + +/* Get the list of RPMh voltage levels from cmd-db */ +static int a6xx_gmu_rpmh_arc_cmds(const char *id, void *vals, int size) +{ + u32 len = cmd_db_read_aux_data_len(id); + + if (!len) + return 0; + + if (WARN_ON(len > size)) + return -EINVAL; + + cmd_db_read_aux_data(id, vals, len); + + /* + * The data comes back as an array of unsigned shorts so adjust the + * count accordingly + */ + return len >> 1; +} + +/* Return the 'arc-level' for the given frequency */ +static u32 a6xx_gmu_get_arc_level(struct device *dev, unsigned long freq) +{ + struct dev_pm_opp *opp; + struct device_node *np; + u32 val = 0; + + if (!freq) + return 0; + + opp = dev_pm_opp_find_freq_exact(dev, freq, true); + if (IS_ERR(opp)) + return 0; + + np = dev_pm_opp_get_of_node(opp); + + if (np) { + of_property_read_u32(np, "qcom,level", &val); + of_node_put(np); + } + + dev_pm_opp_put(opp); + + return val; +} + +static int a6xx_gmu_rpmh_arc_votes_init(struct device *dev, u32 *votes, + unsigned long *freqs, int freqs_count, + u16 *pri, int pri_count, + u16 *sec, int sec_count) +{ + int i, j; + + /* Construct a vote for each frequency */ + for (i = 0; i < freqs_count; i++) { + u8 pindex = 0, sindex = 0; + u32 level = a6xx_gmu_get_arc_level(dev, freqs[i]); + + /* Get the primary index that matches the arc level */ + for (j = 0; j < pri_count; j++) { + if (pri[j] >= level) { + pindex = j; + break; + } + } + + if (j == pri_count) { + dev_err(dev, + "Level %u not found in in the RPMh list\n", + level); + dev_err(dev, "Available levels:\n"); + for (j = 0; j < pri_count; j++) + dev_err(dev, " %u\n", pri[j]); + + return -EINVAL; + } + + /* + * Look for a level in in the secondary list that matches. If + * nothing fits, use the maximum + */ + sindex = sec_count - 1; + + for (j = 0; j < sec_count - 1; j++) { + if (sec[j] >= level) { + sindex = j; + break; + } + } + + /* Construct the vote */ + votes[i] = ((pri[pindex] & 0xffff) << 16) | + (sindex << 8) | pindex; + } + + return 0; +} + +/* + * The GMU votes with the RPMh for itself and on behalf of the GPU but we need + * to construct the list of votes on the CPU and send it over. Query the RPMh + * voltage levels and build the votes + */ + +static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + u16 gx[16], cx[16], mx[16]; + u32 gxcount, cxcount, mxcount; + int ret; + + /* Get the list of available voltage levels for each component */ + gxcount = a6xx_gmu_rpmh_arc_cmds("gfx.lvl", gx, sizeof(gx)); + cxcount = a6xx_gmu_rpmh_arc_cmds("cx.lvl", cx, sizeof(cx)); + mxcount = a6xx_gmu_rpmh_arc_cmds("mx.lvl", mx, sizeof(mx)); + + /* Build the GX votes */ + ret = a6xx_gmu_rpmh_arc_votes_init(&gpu->pdev->dev, gmu->gx_arc_votes, + gmu->gpu_freqs, gmu->nr_gpu_freqs, + gx, gxcount, mx, mxcount); + + /* Build the CX votes */ + ret |= a6xx_gmu_rpmh_arc_votes_init(gmu->dev, gmu->cx_arc_votes, + gmu->gmu_freqs, gmu->nr_gmu_freqs, + cx, cxcount, mx, mxcount); + + return ret; +} + +static int a6xx_gmu_build_freq_table(struct device *dev, unsigned long *freqs, + u32 size) +{ + int count = dev_pm_opp_get_opp_count(dev); + struct dev_pm_opp *opp; + int i, index = 0; + unsigned long freq = 1; + + /* + * The OPP table doesn't contain the "off" frequency level so we need to + * add 1 to the table size to account for it + */ + + if (WARN(count + 1 > size, + "The GMU frequency table is being truncated\n")) + count = size - 1; + + /* Set the "off" frequency */ + freqs[index++] = 0; + + for (i = 0; i < count; i++) { + opp = dev_pm_opp_find_freq_ceil(dev, &freq); + if (IS_ERR(opp)) + break; + + dev_pm_opp_put(opp); + freqs[index++] = freq++; + } + + return index; +} + +static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + int ret = 0; + + /* + * The GMU handles its own frequency switching so build a list of + * available frequencies to send during initalization + */ + ret = dev_pm_opp_of_add_table(gmu->dev); + if (ret) { + dev_err(gmu->dev, "Unable to set the OPP table for the GMU\n"); + return ret; + } + + gmu->nr_gmu_freqs = a6xx_gmu_build_freq_table(gmu->dev, + gmu->gmu_freqs, ARRAY_SIZE(gmu->gmu_freqs)); + + /* + * The GMU also handles GPU frequency switching so build a list + * from the GPU OPP table + */ + gmu->nr_gpu_freqs = a6xx_gmu_build_freq_table(&gpu->pdev->dev, + gmu->gpu_freqs, ARRAY_SIZE(gmu->gpu_freqs)); + + /* Build the list of RPMh votes that we'll send to the GMU */ + return a6xx_gmu_rpmh_votes_init(gmu); +} + +static int a6xx_gmu_clocks_probe(struct a6xx_gmu *gmu) +{ + int ret = msm_clk_bulk_get(gmu->dev, &gmu->clocks); + + if (ret < 1) + return ret; + + gmu->nr_clocks = ret; + + gmu->core_clk = msm_clk_bulk_get_clock(gmu->clocks, + gmu->nr_clocks, "gmu"); + + return 0; +} + +static void __iomem *a6xx_gmu_get_mmio(struct platform_device *pdev, + const char *name) +{ + void __iomem *ret; + struct resource *res = platform_get_resource_byname(pdev, + IORESOURCE_MEM, name); + + if (!res) { + dev_err(&pdev->dev, "Unable to find the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + ret = devm_ioremap(&pdev->dev, res->start, resource_size(res)); + if (!ret) { + dev_err(&pdev->dev, "Unable to map the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + return ret; +} + +static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev, + const char *name, irq_handler_t handler) +{ + int irq, ret; + + irq = platform_get_irq_byname(pdev, name); + + ret = devm_request_irq(&pdev->dev, irq, handler, IRQF_TRIGGER_HIGH, + name, gmu); + if (ret) { + dev_err(&pdev->dev, "Unable to get interrupt %s\n", name); + return ret; + } + + disable_irq(irq); + + return irq; +} + +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + if (IS_ERR_OR_NULL(gmu->mmio)) + return; + + pm_runtime_disable(gmu->dev); + a6xx_gmu_stop(a6xx_gpu); + + a6xx_gmu_irq_disable(gmu); + a6xx_gmu_memory_free(gmu, gmu->hfi); + + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); +} + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + struct platform_device *pdev = of_find_device_by_node(node); + int ret; + + if (!pdev) + return -ENODEV; + + gmu->dev = &pdev->dev; + + of_dma_configure(gmu->dev, node); + + /* Fow now, don't do anything fancy until we get our feet under us */ + gmu->idle_level = GMU_IDLE_STATE_ACTIVE; + + pm_runtime_enable(gmu->dev); + gmu->gx = devm_regulator_get(gmu->dev, "vdd"); + + /* Get the list of clocks */ + ret = a6xx_gmu_clocks_probe(gmu); + if (ret) + return ret; + + /* Set up the IOMMU context bank */ + ret = a6xx_gmu_memory_probe(gmu); + if (ret) + return ret; + + /* Allocate memory for for the HFI queues */ + gmu->hfi = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->hfi)) + goto err; + + /* Allocate memory for the GMU debug region */ + gmu->debug = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->debug)) + goto err; + + /* Map the GMU registers */ + gmu->mmio = a6xx_gmu_get_mmio(pdev, "gmu"); + + /* Map the GPU power domain controller registers */ + gmu->pdc_mmio = a6xx_gmu_get_mmio(pdev, "gmu_pdc"); + + if (IS_ERR(gmu->mmio) || IS_ERR(gmu->pdc_mmio)) + goto err; + + /* Get the HFI and GMU interrupts */ + gmu->hfi_irq = a6xx_gmu_get_irq(gmu, pdev, "hfi", a6xx_hfi_irq); + gmu->gmu_irq = a6xx_gmu_get_irq(gmu, pdev, "gmu", a6xx_gmu_irq); + + if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) + goto err; + + /* Set up a tasklet to handle GMU HFI responses */ + tasklet_init(&gmu->hfi_tasklet, a6xx_hfi_task, (unsigned long) gmu); + + /* Get the power levels for the GMU and GPU */ + a6xx_gmu_pwrlevels_probe(gmu); + + /* Set up the HFI queues */ + a6xx_hfi_init(gmu); + + return 0; +err: + a6xx_gmu_memory_free(gmu, gmu->hfi); + + if (gmu->domain) { + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); + } + + return -ENODEV; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h new file mode 100644 index 000000000000..d9a386c18799 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_GMU_H_ +#define _A6XX_GMU_H_ + +#include <linux/interrupt.h> +#include "msm_drv.h" +#include "a6xx_hfi.h" + +struct a6xx_gmu_bo { + void *virt; + size_t size; + u64 iova; + struct page **pages; +}; + +/* + * These define the different GMU wake up options - these define how both the + * CPU and the GMU bring up the hardware + */ + +/* THe GMU has already been booted and the rentention registers are active */ +#define GMU_WARM_BOOT 0 + +/* the GMU is coming up for the first time or back from a power collapse */ +#define GMU_COLD_BOOT 1 + +/* The GMU is being soft reset after a fault */ +#define GMU_RESET 2 + +/* + * These define the level of control that the GMU has - the higher the number + * the more things that the GMU hardware controls on its own. + */ + +/* The GMU does not do any idle state management */ +#define GMU_IDLE_STATE_ACTIVE 0 + +/* The GMU manages SPTP power collapse */ +#define GMU_IDLE_STATE_SPTP 2 + +/* The GMU does automatic IFPC (intra-frame power collapse) */ +#define GMU_IDLE_STATE_IFPC 3 + +struct a6xx_gmu { + struct device *dev; + + void * __iomem mmio; + void * __iomem pdc_mmio; + + int hfi_irq; + int gmu_irq; + + struct regulator *gx; + + struct iommu_domain *domain; + u64 uncached_iova_base; + + int idle_level; + + struct a6xx_gmu_bo *hfi; + struct a6xx_gmu_bo *debug; + + int nr_clocks; + struct clk_bulk_data *clocks; + struct clk *core_clk; + + int nr_gpu_freqs; + unsigned long gpu_freqs[16]; + u32 gx_arc_votes[16]; + + int nr_gmu_freqs; + unsigned long gmu_freqs[4]; + u32 cx_arc_votes[4]; + + struct a6xx_hfi_queue queues[2]; + + struct tasklet_struct hfi_tasklet; +}; + +static inline u32 gmu_read(struct a6xx_gmu *gmu, u32 offset) +{ + return msm_readl(gmu->mmio + (offset << 2)); +} + +static inline void gmu_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->mmio + (offset << 2)); +} + +static inline void pdc_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->pdc_mmio + (offset << 2)); +} + +static inline void gmu_rmw(struct a6xx_gmu *gmu, u32 reg, u32 mask, u32 or) +{ + u32 val = gmu_read(gmu, reg); + + val &= ~mask; + + gmu_write(gmu, reg, val | or); +} + +#define gmu_poll_timeout(gmu, addr, val, cond, interval, timeout) \ + readl_poll_timeout((gmu)->mmio + ((addr) << 2), val, cond, \ + interval, timeout) + +/* + * These are the available OOB (out of band requests) to the GMU where "out of + * band" means that the CPU talks to the GMU directly and not through HFI. + * Normally this works by writing a ITCM/DTCM register and then triggering a + * interrupt (the "request" bit) and waiting for an acknowledgment (the "ack" + * bit). The state is cleared by writing the "clear' bit to the GMU interrupt. + * + * These are used to force the GMU/GPU to stay on during a critical sequence or + * for hardware workarounds. + */ + +enum a6xx_gmu_oob_state { + GMU_OOB_BOOT_SLUMBER = 0, + GMU_OOB_GPU_SET, + GMU_OOB_DCVS_SET, +}; + +/* These are the interrupt / ack bits for each OOB request that are set + * in a6xx_gmu_set_oob and a6xx_clear_oob + */ + +/* + * Let the GMU know that a boot or slumber operation has started. The value in + * REG_A6XX_GMU_BOOT_SLUMBER_OPTION lets the GMU know which operation we are + * doing + */ +#define GMU_OOB_BOOT_SLUMBER_REQUEST 22 +#define GMU_OOB_BOOT_SLUMBER_ACK 30 +#define GMU_OOB_BOOT_SLUMBER_CLEAR 30 + +/* + * Set a new power level for the GPU when the CPU is doing frequency scaling + */ +#define GMU_OOB_DCVS_REQUEST 23 +#define GMU_OOB_DCVS_ACK 31 +#define GMU_OOB_DCVS_CLEAR 31 + +/* + * Let the GMU know to not turn off any GPU registers while the CPU is in a + * critical section + */ +#define GMU_OOB_GPU_SET_REQUEST 16 +#define GMU_OOB_GPU_SET_ACK 24 +#define GMU_OOB_GPU_SET_CLEAR 24 + + +void a6xx_hfi_init(struct a6xx_gmu *gmu); +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state); +void a6xx_hfi_stop(struct a6xx_gmu *gmu); + +void a6xx_hfi_task(unsigned long data); + +#endif diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c new file mode 100644 index 000000000000..b3b85ebce88d --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -0,0 +1,807 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + + +#include "msm_gem.h" +#include "msm_mmu.h" +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static inline bool _a6xx_check_idle(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Check that the GMU is idle */ + if (!a6xx_gmu_isidle(&a6xx_gpu->gmu)) + return false; + + /* Check tha the CX master is idle */ + if (gpu_read(gpu, REG_A6XX_RBBM_STATUS) & + ~A6XX_RBBM_STATUS_CP_AHB_BUSY_CX_MASTER) + return false; + + return !(gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS) & + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT); +} + +bool a6xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + /* wait for CP to drain ringbuffer: */ + if (!adreno_idle(gpu, ring)) + return false; + + if (spin_until(_a6xx_check_idle(gpu))) { + DRM_ERROR("%s: %ps: timeout waiting for GPU to idle: status %8.8X irq %8.8X rptr/wptr %d/%d\n", + gpu->name, __builtin_return_address(0), + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR)); + return false; + } + + return true; +} + +static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + uint32_t wptr; + unsigned long flags; + + spin_lock_irqsave(&ring->lock, flags); + + /* Copy the shadow to the actual register */ + ring->cur = ring->next; + + /* Make sure to wrap wptr if we need to */ + wptr = get_wptr(ring); + + spin_unlock_irqrestore(&ring->lock, flags); + + /* Make sure everything is posted before making a decision */ + mb(); + + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); +} + +static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, + struct msm_file_private *ctx) +{ + struct msm_drm_private *priv = gpu->dev->dev_private; + struct msm_ringbuffer *ring = submit->ring; + unsigned int i; + + /* Invalidate CCU depth and color */ + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_DEPTH); + + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_COLOR); + + /* Submit the commands */ + for (i = 0; i < submit->nr_cmds; i++) { + switch (submit->cmd[i].type) { + case MSM_SUBMIT_CMD_IB_TARGET_BUF: + break; + case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: + if (priv->lastctx == ctx) + break; + case MSM_SUBMIT_CMD_BUF: + OUT_PKT7(ring, CP_INDIRECT_BUFFER_PFE, 3); + OUT_RING(ring, lower_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, upper_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, submit->cmd[i].size); + break; + } + } + + /* Write the fence to the scratch register */ + OUT_PKT4(ring, REG_A6XX_CP_SCRATCH_REG(2), 1); + OUT_RING(ring, submit->seqno); + + /* + * Execute a CACHE_FLUSH_TS event. This will ensure that the + * timestamp is written to the memory and then triggers the interrupt + */ + OUT_PKT7(ring, CP_EVENT_WRITE, 4); + OUT_RING(ring, CACHE_FLUSH_TS | (1 << 31)); + OUT_RING(ring, lower_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, upper_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, submit->seqno); + + a6xx_flush(gpu, ring); +} + +static const struct { + u32 offset; + u32 value; +} a6xx_hwcg[] = { + {REG_A6XX_RBBM_CLOCK_CNTL_SP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP0, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP1, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP2, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP3, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP0, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP1, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP2, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP3, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_HYST_SP0, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP1, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP2, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP3, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP0, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP1, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP2, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP3, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP0, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP1, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP2, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP3, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_HYST_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP0, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP1, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP2, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP3, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP0, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP1, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP2, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP3, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_CNTL_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_UCHE, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_HYST_UCHE, 0x00000004}, + {REG_A6XX_RBBM_CLOCK_DELAY_UCHE, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB0, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB1, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB2, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB3, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU0, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU1, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU2, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU3, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU0, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU1, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU2, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU3, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_CNTL_RAC, 0x05022022}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RAC, 0x00005555}, + {REG_A6XX_RBBM_CLOCK_DELAY_RAC, 0x00000011}, + {REG_A6XX_RBBM_CLOCK_HYST_RAC, 0x00445044}, + {REG_A6XX_RBBM_CLOCK_CNTL_TSE_RAS_RBBM, 0x04222222}, + {REG_A6XX_RBBM_CLOCK_MODE_GPC, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_MODE_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_HYST_TSE_RAS_RBBM, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_HYST_GPC, 0x04104004}, + {REG_A6XX_RBBM_CLOCK_HYST_VFD, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_TSE_RAS_RBBM, 0x00004000}, + {REG_A6XX_RBBM_CLOCK_DELAY_GPC, 0x00000200}, + {REG_A6XX_RBBM_CLOCK_DELAY_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ_2, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_MODE_HLSQ, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_GMU_GX, 0x00000222}, + {REG_A6XX_RBBM_CLOCK_DELAY_GMU_GX, 0x00000111}, + {REG_A6XX_RBBM_CLOCK_HYST_GMU_GX, 0x00000555} +}; + +static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + unsigned int i; + u32 val; + + val = gpu_read(gpu, REG_A6XX_RBBM_CLOCK_CNTL); + + /* Don't re-program the registers if they are already correct */ + if ((!state && !val) || (state && (val == 0x8aa8aa02))) + return; + + /* Disable SP clock before programming HWCG registers */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 1, 0); + + for (i = 0; i < ARRAY_SIZE(a6xx_hwcg); i++) + gpu_write(gpu, a6xx_hwcg[i].offset, + state ? a6xx_hwcg[i].value : 0); + + /* Enable SP clock */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 0, 1); + + gpu_write(gpu, REG_A6XX_RBBM_CLOCK_CNTL, state ? 0x8aa8aa02 : 0); +} + +static int a6xx_cp_init(struct msm_gpu *gpu) +{ + struct msm_ringbuffer *ring = gpu->rb[0]; + + OUT_PKT7(ring, CP_ME_INIT, 8); + + OUT_RING(ring, 0x0000002f); + + /* Enable multiple hardware contexts */ + OUT_RING(ring, 0x00000003); + + /* Enable error detection */ + OUT_RING(ring, 0x20000000); + + /* Don't enable header dump */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + /* No workarounds enabled */ + OUT_RING(ring, 0x00000000); + + /* Pad rest of the cmds with 0's */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + a6xx_flush(gpu, ring); + return a6xx_idle(gpu, ring) ? 0 : -EINVAL; +} + +static int a6xx_ucode_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (!a6xx_gpu->sqe_bo) { + a6xx_gpu->sqe_bo = adreno_fw_create_bo(gpu, + adreno_gpu->fw[ADRENO_FW_SQE], &a6xx_gpu->sqe_iova); + + if (IS_ERR(a6xx_gpu->sqe_bo)) { + int ret = PTR_ERR(a6xx_gpu->sqe_bo); + + a6xx_gpu->sqe_bo = NULL; + DRM_DEV_ERROR(&gpu->pdev->dev, + "Could not allocate SQE ucode: %d\n", ret); + + return ret; + } + } + + gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE_LO, + REG_A6XX_CP_SQE_INSTR_BASE_HI, a6xx_gpu->sqe_iova); + + return 0; +} + +#define A6XX_INT_MASK (A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_CP_HW_ERROR | \ + A6XX_RBBM_INT_0_MASK_CP_IB2 | \ + A6XX_RBBM_INT_0_MASK_CP_IB1 | \ + A6XX_RBBM_INT_0_MASK_CP_RB | \ + A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT | \ + A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS | \ + A6XX_RBBM_INT_0_MASK_UCHE_TRAP_INTR) + +static int a6xx_hw_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + /* Make sure the GMU keeps the GPU on while we set it up */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_CNTL, 0); + + /* + * Disable the trusted memory range - we don't actually supported secure + * memory rendering at this point in time and we don't want to block off + * part of the virtual memory space. + */ + gpu_write64(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO, + REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000); + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000); + + /* enable hardware clockgating */ + a6xx_set_hwcg(gpu, true); + + /* VBIF start */ + gpu_write(gpu, REG_A6XX_VBIF_GATE_OFF_WRREQ_EN, 0x00000009); + gpu_write(gpu, REG_A6XX_RBBM_VBIF_CLIENT_QOS_CNTL, 0x3); + + /* Make all blocks contribute to the GPU BUSY perf counter */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_GPU_BUSY_MASKED, 0xffffffff); + + /* Disable L2 bypass in the UCHE */ + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_LO, 0xffffffc0); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_HI, 0x0001ffff); + + /* Set the GMEM VA range [0x100000:0x100000 + gpu->gmem - 1] */ + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MIN_LO, + REG_A6XX_UCHE_GMEM_RANGE_MIN_HI, 0x00100000); + + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MAX_LO, + REG_A6XX_UCHE_GMEM_RANGE_MAX_HI, + 0x00100000 + adreno_gpu->gmem - 1); + + gpu_write(gpu, REG_A6XX_UCHE_FILTER_CNTL, 0x804); + gpu_write(gpu, REG_A6XX_UCHE_CACHE_WAYS, 0x4); + + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_2, 0x010000c0); + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_1, 0x8040362c); + + /* Setting the mem pool size */ + gpu_write(gpu, REG_A6XX_CP_MEM_POOL_SIZE, 128); + + /* Setting the primFifo thresholds default values */ + gpu_write(gpu, REG_A6XX_PC_DBG_ECO_CNTL, (0x300 << 11)); + + /* Set the AHB default slave response to "ERROR" */ + gpu_write(gpu, REG_A6XX_CP_AHB_CNTL, 0x1); + + /* Turn on performance counters */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_CNTL, 0x1); + + /* Select CP0 to always count cycles */ + gpu_write(gpu, REG_A6XX_CP_PERFCTR_CP_SEL_0, PERF_CP_ALWAYS_COUNT); + + /* FIXME: not sure if this should live here or in a6xx_gmu.c */ + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_MASK, + 0xff000000); + gmu_rmw(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_SELECT_0, + 0xff, 0x20); + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, + 0x01); + + gpu_write(gpu, REG_A6XX_RB_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_TPL1_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_SP_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_UCHE_MODE_CNTL, 2 << 21); + + /* Enable fault detection */ + gpu_write(gpu, REG_A6XX_RBBM_INTERFACE_HANG_INT_CNTL, + (1 << 30) | 0x1fffff); + + gpu_write(gpu, REG_A6XX_UCHE_CLIENT_PF, 1); + + /* Protect registers from the CP */ + gpu_write(gpu, REG_A6XX_CP_PROTECT_CNTL, 0x00000003); + + gpu_write(gpu, REG_A6XX_CP_PROTECT(0), + A6XX_PROTECT_RDONLY(0x600, 0x51)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(1), A6XX_PROTECT_RW(0xae50, 0x2)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(2), A6XX_PROTECT_RW(0x9624, 0x13)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(3), A6XX_PROTECT_RW(0x8630, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(4), A6XX_PROTECT_RW(0x9e70, 0x1)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(5), A6XX_PROTECT_RW(0x9e78, 0x187)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(6), A6XX_PROTECT_RW(0xf000, 0x810)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(7), + A6XX_PROTECT_RDONLY(0xfc00, 0x3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(8), A6XX_PROTECT_RW(0x50e, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(9), A6XX_PROTECT_RDONLY(0x50f, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(10), A6XX_PROTECT_RW(0x510, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(11), + A6XX_PROTECT_RDONLY(0x0, 0x4f9)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(12), + A6XX_PROTECT_RDONLY(0x501, 0xa)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(13), + A6XX_PROTECT_RDONLY(0x511, 0x44)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(14), A6XX_PROTECT_RW(0xe00, 0xe)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(15), A6XX_PROTECT_RW(0x8e00, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(16), A6XX_PROTECT_RW(0x8e50, 0xf)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(17), A6XX_PROTECT_RW(0xbe02, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(18), + A6XX_PROTECT_RW(0xbe20, 0x11f3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(19), A6XX_PROTECT_RW(0x800, 0x82)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(20), A6XX_PROTECT_RW(0x8a0, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(21), A6XX_PROTECT_RW(0x8ab, 0x19)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(22), A6XX_PROTECT_RW(0x900, 0x4d)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(23), A6XX_PROTECT_RW(0x98d, 0x76)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(24), + A6XX_PROTECT_RDONLY(0x8d0, 0x23)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(25), + A6XX_PROTECT_RDONLY(0x980, 0x4)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(26), A6XX_PROTECT_RW(0xa630, 0x0)); + + /* Enable interrupts */ + gpu_write(gpu, REG_A6XX_RBBM_INT_0_MASK, A6XX_INT_MASK); + + ret = adreno_hw_init(gpu); + if (ret) + goto out; + + ret = a6xx_ucode_init(gpu); + if (ret) + goto out; + + /* Always come up on rb 0 */ + a6xx_gpu->cur_ring = gpu->rb[0]; + + /* Enable the SQE_to start the CP engine */ + gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); + + ret = a6xx_cp_init(gpu); + if (ret) + goto out; + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TRUST_CNTL, 0x0); + +out: + /* + * Tell the GMU that we are done touching the GPU and it can start power + * management + */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + /* Take the GMU out of its special boot mode */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +static void a6xx_dump(struct msm_gpu *gpu) +{ + dev_info(&gpu->pdev->dev, "status: %08x\n", + gpu_read(gpu, REG_A6XX_RBBM_STATUS)); + adreno_dump(gpu); +} + +#define VBIF_RESET_ACK_TIMEOUT 100 +#define VBIF_RESET_ACK_MASK 0x00f0 + +static void a6xx_recover(struct msm_gpu *gpu) +{ + int i; + + adreno_dump_info(gpu); + + for (i = 0; i < 8; i++) + dev_info(&gpu->pdev->dev, "CP_SCRATCH_REG%d: %u\n", i, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(i))); + + if (hang_debug) + a6xx_dump(gpu); + + gpu->funcs->pm_suspend(gpu); + gpu->funcs->pm_resume(gpu); + + msm_gpu_hw_init(gpu); +} + +static int a6xx_fault_handler(void *arg, unsigned long iova, int flags) +{ + struct msm_gpu *gpu = arg; + + pr_warn_ratelimited("*** gpu fault: iova=%08lx, flags=%d (%u,%u,%u,%u)\n", + iova, flags, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(4)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(5)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(6)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(7))); + + return -EFAULT; +} + +static void a6xx_cp_hw_err_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_CP_INTERRUPT_STATUS); + + if (status & A6XX_CP_INT_CP_OPCODE_ERROR) { + u32 val; + + gpu_write(gpu, REG_A6XX_CP_SQE_STAT_ADDR, 1); + val = gpu_read(gpu, REG_A6XX_CP_SQE_STAT_DATA); + dev_err_ratelimited(&gpu->pdev->dev, + "CP | opcode error | possible opcode=0x%8.8X\n", + val); + } + + if (status & A6XX_CP_INT_CP_UCODE_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, + "CP ucode error interrupt\n"); + + if (status & A6XX_CP_INT_CP_HW_FAULT_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | HW fault | status=0x%8.8X\n", + gpu_read(gpu, REG_A6XX_CP_HW_FAULT)); + + if (status & A6XX_CP_INT_CP_REGISTER_PROTECTION_ERROR) { + u32 val = gpu_read(gpu, REG_A6XX_CP_PROTECT_STATUS); + + dev_err_ratelimited(&gpu->pdev->dev, + "CP | protected mode error | %s | addr=0x%8.8X | status=0x%8.8X\n", + val & (1 << 20) ? "READ" : "WRITE", + (val & 0x3ffff), val); + } + + if (status & A6XX_CP_INT_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP AHB error interrupt\n"); + + if (status & A6XX_CP_INT_CP_VSD_PARITY_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP VSD decoder parity error\n"); + + if (status & A6XX_CP_INT_CP_ILLEGAL_INSTR_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP illegal instruction error\n"); + +} + +static void a6xx_fault_detect_irq(struct msm_gpu *gpu) +{ + struct drm_device *dev = gpu->dev; + struct msm_drm_private *priv = dev->dev_private; + struct msm_ringbuffer *ring = gpu->funcs->active_ring(gpu); + + DRM_DEV_ERROR(&gpu->pdev->dev, + "gpu fault ring %d fence %x status %8.8X rb %4.4x/%4.4x ib1 %16.16llX/%4.4x ib2 %16.16llX/%4.4x\n", + ring ? ring->id : -1, ring ? ring->seqno : 0, + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR), + gpu_read64(gpu, REG_A6XX_CP_IB1_BASE, REG_A6XX_CP_IB1_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB1_REM_SIZE), + gpu_read64(gpu, REG_A6XX_CP_IB2_BASE, REG_A6XX_CP_IB2_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB2_REM_SIZE)); + + /* Turn off the hangcheck timer to keep it from bothering us */ + del_timer(&gpu->hangcheck_timer); + + queue_work(priv->wq, &gpu->recover_work); +} + +static irqreturn_t a6xx_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS); + + gpu_write(gpu, REG_A6XX_RBBM_INT_CLEAR_CMD, status); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT) + a6xx_fault_detect_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | AHB bus error\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_HW_ERROR) + a6xx_cp_hw_err_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB ASYNC overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB bus overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS) + dev_err_ratelimited(&gpu->pdev->dev, "UCHE | Out of bounds access\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) + msm_gpu_retire(gpu); + + return IRQ_HANDLED; +} + +static const u32 a6xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE, REG_A6XX_CP_RB_BASE), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE_HI, REG_A6XX_CP_RB_BASE_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR, + REG_A6XX_CP_RB_RPTR_ADDR_LO), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR_HI, + REG_A6XX_CP_RB_RPTR_ADDR_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR, REG_A6XX_CP_RB_RPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_WPTR, REG_A6XX_CP_RB_WPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_CNTL, REG_A6XX_CP_RB_CNTL), +}; + +static const u32 a6xx_registers[] = { + 0x0000, 0x0002, 0x0010, 0x0010, 0x0012, 0x0012, 0x0018, 0x001b, + 0x001e, 0x0032, 0x0038, 0x003c, 0x0042, 0x0042, 0x0044, 0x0044, + 0x0047, 0x0047, 0x0056, 0x0056, 0x00ad, 0x00ae, 0x00b0, 0x00fb, + 0x0100, 0x011d, 0x0200, 0x020d, 0x0210, 0x0213, 0x0218, 0x023d, + 0x0400, 0x04f9, 0x0500, 0x0500, 0x0505, 0x050b, 0x050e, 0x0511, + 0x0533, 0x0533, 0x0540, 0x0555, 0x0800, 0x0808, 0x0810, 0x0813, + 0x0820, 0x0821, 0x0823, 0x0827, 0x0830, 0x0833, 0x0840, 0x0843, + 0x084f, 0x086f, 0x0880, 0x088a, 0x08a0, 0x08ab, 0x08c0, 0x08c4, + 0x08d0, 0x08dd, 0x08f0, 0x08f3, 0x0900, 0x0903, 0x0908, 0x0911, + 0x0928, 0x093e, 0x0942, 0x094d, 0x0980, 0x0984, 0x098d, 0x0996, + 0x0998, 0x099e, 0x09a0, 0x09a6, 0x09a8, 0x09ae, 0x09b0, 0x09b1, + 0x09c2, 0x09c8, 0x0a00, 0x0a03, 0x0c00, 0x0c04, 0x0c06, 0x0c06, + 0x0c10, 0x0cd9, 0x0e00, 0x0e0e, 0x0e10, 0x0e13, 0x0e17, 0x0e19, + 0x0e1c, 0x0e2b, 0x0e30, 0x0e32, 0x0e38, 0x0e39, 0x8600, 0x8601, + 0x8610, 0x861b, 0x8620, 0x8620, 0x8628, 0x862b, 0x8630, 0x8637, + 0x8e01, 0x8e01, 0x8e04, 0x8e05, 0x8e07, 0x8e08, 0x8e0c, 0x8e0c, + 0x8e10, 0x8e1c, 0x8e20, 0x8e25, 0x8e28, 0x8e28, 0x8e2c, 0x8e2f, + 0x8e3b, 0x8e3e, 0x8e40, 0x8e43, 0x8e50, 0x8e5e, 0x8e70, 0x8e77, + 0x9600, 0x9604, 0x9624, 0x9637, 0x9e00, 0x9e01, 0x9e03, 0x9e0e, + 0x9e11, 0x9e16, 0x9e19, 0x9e19, 0x9e1c, 0x9e1c, 0x9e20, 0x9e23, + 0x9e30, 0x9e31, 0x9e34, 0x9e34, 0x9e70, 0x9e72, 0x9e78, 0x9e79, + 0x9e80, 0x9fff, 0xa600, 0xa601, 0xa603, 0xa603, 0xa60a, 0xa60a, + 0xa610, 0xa617, 0xa630, 0xa630, + ~0 +}; + +static int a6xx_pm_resume(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + ret = a6xx_gmu_resume(a6xx_gpu); + + gpu->needs_hw_init = true; + + return ret; +} + +static int a6xx_pm_suspend(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* + * Make sure the GMU is idle before continuing (because some transitions + * may use VBIF + */ + a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Clear the VBIF pipe before shutting down */ + /* FIXME: This accesses the GPU - do we need to make sure it is on? */ + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0xf); + spin_until((gpu_read(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL1) & 0xf) == 0xf); + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0); + + return a6xx_gmu_stop(a6xx_gpu); +} + +static int a6xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Force the GPU power on so we can read this register */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + *value = gpu_read64(gpu, REG_A6XX_RBBM_PERFCTR_CP_0_LO, + REG_A6XX_RBBM_PERFCTR_CP_0_HI); + + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + return 0; +} + +#ifdef CONFIG_DEBUG_FS +static void a6xx_show(struct msm_gpu *gpu, struct seq_file *m) +{ + gpu->funcs->pm_resume(gpu); + + seq_printf(m, "status: %08x\n", + gpu_read(gpu, REG_A6XX_RBBM_STATUS)); + gpu->funcs->pm_suspend(gpu); + + adreno_show(gpu, m); +} +#endif + +static struct msm_ringbuffer *a6xx_active_ring(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + return a6xx_gpu->cur_ring; +} + +static void a6xx_destroy(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (a6xx_gpu->sqe_bo) { + if (a6xx_gpu->sqe_iova) + msm_gem_put_iova(a6xx_gpu->sqe_bo, gpu->aspace); + drm_gem_object_unreference_unlocked(a6xx_gpu->sqe_bo); + } + + a6xx_gmu_remove(a6xx_gpu); + + adreno_gpu_cleanup(adreno_gpu); + kfree(a6xx_gpu); +} + +static const struct adreno_gpu_funcs funcs = { + .base = { + .get_param = adreno_get_param, + .hw_init = a6xx_hw_init, + .pm_suspend = a6xx_pm_suspend, + .pm_resume = a6xx_pm_resume, + .recover = a6xx_recover, + .submit = a6xx_submit, + .flush = a6xx_flush, + .active_ring = a6xx_active_ring, + .irq = a6xx_irq, + .destroy = a6xx_destroy, +#ifdef CONFIG_DEBUG_FS + .show = a6xx_show, +#endif + }, + .get_timestamp = a6xx_get_timestamp, +}; + +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) +{ + struct msm_drm_private *priv = dev->dev_private; + struct platform_device *pdev = priv->gpu_pdev; + struct device_node *node; + struct a6xx_gpu *a6xx_gpu; + struct adreno_gpu *adreno_gpu; + struct msm_gpu *gpu; + int ret; + + a6xx_gpu = kzalloc(sizeof(*a6xx_gpu), GFP_KERNEL); + if (!a6xx_gpu) + return ERR_PTR(-ENOMEM); + + adreno_gpu = &a6xx_gpu->base; + gpu = &adreno_gpu->base; + + adreno_gpu->registers = a6xx_registers; + adreno_gpu->reg_offsets = a6xx_register_offsets; + + ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + /* Check if there is a GMU phandle and set it up */ + node = of_parse_phandle(pdev->dev.of_node, "gmu", 0); + + /* FIXME: How do we gracefully handle this? */ + BUG_ON(!node); + + ret = a6xx_gmu_probe(a6xx_gpu, node); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + if (gpu->aspace) + msm_mmu_set_fault_handler(gpu->aspace->mmu, gpu, + a6xx_fault_handler); + + return gpu; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h new file mode 100644 index 000000000000..dd69e5b0e692 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef __A6XX_GPU_H__ +#define __A6XX_GPU_H__ + + +#include "adreno_gpu.h" +#include "a6xx.xml.h" + +#include "a6xx_gmu.h" + +extern bool hang_debug; + +struct a6xx_gpu { + struct adreno_gpu base; + + struct drm_gem_object *sqe_bo; + uint64_t sqe_iova; + + struct msm_ringbuffer *cur_ring; + + struct a6xx_gmu gmu; +}; + +#define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) + +/* + * Given a register and a count, return a value to program into + * REG_CP_PROTECT_REG(n) - this will block both reads and writes for _len + * registers starting at _reg. + */ +#define A6XX_PROTECT_RW(_reg, _len) \ + ((1 << 31) | \ + (((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + +/* + * Same as above, but allow reads over the range. For areas of mixed use (such + * as performance counters) this allows us to protect a much larger range with a + * single register + */ +#define A6XX_PROTECT_RDONLY(_reg, _len) \ + ((((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + + +int a6xx_gmu_resume(struct a6xx_gpu *gpu); +int a6xx_gmu_stop(struct a6xx_gpu *gpu); + +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *gpu); + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu); +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu); + +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node); +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); + +#endif /* __A6XX_GPU_H__ */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c new file mode 100644 index 000000000000..221eaafc8b5f --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c @@ -0,0 +1,435 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/completion.h> +#include <linux/circ_buf.h> +#include <linux/list.h> + +#include "a6xx_gmu.h" +#include "a6xx_gmu.xml.h" + +#define HFI_MSG_ID(val) [val] = #val + +static const char * const a6xx_hfi_msg_id[] = { + HFI_MSG_ID(HFI_H2F_MSG_INIT), + HFI_MSG_ID(HFI_H2F_MSG_FW_VERSION), + HFI_MSG_ID(HFI_H2F_MSG_BW_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_PERF_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_TEST), +}; + +static int a6xx_hfi_queue_read(struct a6xx_hfi_queue *queue, u32 *data, + u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, hdr, index = header->read_index; + + if (header->read_index == header->write_index) { + header->rx_request = 1; + return 0; + } + + hdr = queue->data[index]; + + /* + * If we are to assume that the GMU firmware is in fact a rational actor + * and is programmed to not send us a larger response than we expect + * then we can also assume that if the header size is unexpectedly large + * that it is due to memory corruption and/or hardware failure. In this + * case the only reasonable course of action is to BUG() to help harden + * the failure. + */ + + BUG_ON(HFI_HEADER_SIZE(hdr) > dwords); + + for (i = 0; i < HFI_HEADER_SIZE(hdr); i++) { + data[i] = queue->data[index]; + index = (index + 1) % header->size; + } + + header->read_index = index; + return HFI_HEADER_SIZE(hdr); +} + +static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu, + struct a6xx_hfi_queue *queue, u32 *data, u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, space, index = header->write_index; + + spin_lock(&queue->lock); + + space = CIRC_SPACE(header->write_index, header->read_index, + header->size); + if (space < dwords) { + header->dropped++; + spin_unlock(&queue->lock); + return -ENOSPC; + } + + for (i = 0; i < dwords; i++) { + queue->data[index] = data[i]; + index = (index + 1) % header->size; + } + + header->write_index = index; + spin_unlock(&queue->lock); + + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 0x01); + return 0; +} + +struct a6xx_hfi_response { + u32 id; + u32 seqnum; + struct list_head node; + struct completion complete; + + u32 error; + u32 payload[16]; +}; + +/* + * Incoming HFI ack messages can come in out of order so we need to store all + * the pending messages on a list until they are handled. + */ +static spinlock_t hfi_ack_lock = __SPIN_LOCK_UNLOCKED(message_lock); +static LIST_HEAD(hfi_ack_list); + +static void a6xx_hfi_handle_ack(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_response *resp; + u32 id, seqnum; + + /* msg->ret_header contains the header of the message being acked */ + id = HFI_HEADER_ID(msg->ret_header); + seqnum = HFI_HEADER_SEQNUM(msg->ret_header); + + spin_lock(&hfi_ack_lock); + list_for_each_entry(resp, &hfi_ack_list, node) { + if (resp->id == id && resp->seqnum == seqnum) { + resp->error = msg->error; + memcpy(resp->payload, msg->payload, + sizeof(resp->payload)); + + complete(&resp->complete); + spin_unlock(&hfi_ack_lock); + return; + } + } + spin_unlock(&hfi_ack_lock); + + dev_err(gmu->dev, "Nobody was waiting for HFI message %d\n", seqnum); +} + +static void a6xx_hfi_handle_error(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_msg_error *error = (struct a6xx_hfi_msg_error *) msg; + + dev_err(gmu->dev, "GMU firmware error %d\n", error->code); +} + +void a6xx_hfi_task(unsigned long data) +{ + struct a6xx_gmu *gmu = (struct a6xx_gmu *) data; + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_RESPONSE_QUEUE]; + struct a6xx_hfi_msg_response resp; + + for (;;) { + u32 id; + int ret = a6xx_hfi_queue_read(queue, (u32 *) &resp, + sizeof(resp) >> 2); + + /* Returns the number of bytes copied or negative on error */ + if (ret <= 0) { + if (ret < 0) + dev_err(gmu->dev, + "Unable to read the HFI message queue\n"); + break; + } + + id = HFI_HEADER_ID(resp.header); + + if (id == HFI_F2H_MSG_ACK) + a6xx_hfi_handle_ack(gmu, &resp); + else if (id == HFI_F2H_MSG_ERROR) + a6xx_hfi_handle_error(gmu, &resp); + } +} + +static int a6xx_hfi_send_msg(struct a6xx_gmu *gmu, int id, + void *data, u32 size, u32 *payload, u32 payload_size) +{ + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_COMMAND_QUEUE]; + struct a6xx_hfi_response resp = { 0 }; + int ret, dwords = size >> 2; + u32 seqnum; + + seqnum = atomic_inc_return(&queue->seqnum) % 0xfff; + + /* First dword of the message is the message header - fill it in */ + *((u32 *) data) = (seqnum << 20) | (HFI_MSG_CMD << 16) | + (dwords << 8) | id; + + init_completion(&resp.complete); + resp.id = id; + resp.seqnum = seqnum; + + spin_lock_bh(&hfi_ack_lock); + list_add_tail(&resp.node, &hfi_ack_list); + spin_unlock_bh(&hfi_ack_lock); + + ret = a6xx_hfi_queue_write(gmu, queue, data, dwords); + if (ret) { + dev_err(gmu->dev, "Unable to send message %s id %d\n", + a6xx_hfi_msg_id[id], seqnum); + goto out; + } + + /* Wait up to 5 seconds for the response */ + ret = wait_for_completion_timeout(&resp.complete, + msecs_to_jiffies(5000)); + if (!ret) { + dev_err(gmu->dev, + "Message %s id %d timed out waiting for response\n", + a6xx_hfi_msg_id[id], seqnum); + ret = -ETIMEDOUT; + } else + ret = 0; + +out: + spin_lock_bh(&hfi_ack_lock); + list_del(&resp.node); + spin_unlock_bh(&hfi_ack_lock); + + if (ret) + return ret; + + if (resp.error) { + dev_err(gmu->dev, "Message %s id %d returned error %d\n", + a6xx_hfi_msg_id[id], seqnum, resp.error); + return -EINVAL; + } + + if (payload && payload_size) { + int copy = min_t(u32, payload_size, sizeof(resp.payload)); + + memcpy(payload, resp.payload, copy); + } + + return 0; +} + +static int a6xx_hfi_send_gmu_init(struct a6xx_gmu *gmu, int boot_state) +{ + struct a6xx_hfi_msg_gmu_init_cmd msg = { 0 }; + + msg.dbg_buffer_addr = (u32) gmu->debug->iova; + msg.dbg_buffer_size = (u32) gmu->debug->size; + msg.boot_state = boot_state; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_INIT, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_get_fw_version(struct a6xx_gmu *gmu, u32 *version) +{ + struct a6xx_hfi_msg_fw_version msg = { 0 }; + + /* Currently supporting version 1.1 */ + msg.supported_version = (1 << 28) | (1 << 16); + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_FW_VERSION, &msg, sizeof(msg), + version, sizeof(*version)); +} + +static int a6xx_hfi_send_perf_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_perf_table msg = { 0 }; + int i; + + msg.num_gpu_levels = gmu->nr_gpu_freqs; + msg.num_gmu_levels = gmu->nr_gmu_freqs; + + for (i = 0; i < gmu->nr_gpu_freqs; i++) { + msg.gx_votes[i].vote = gmu->gx_arc_votes[i]; + msg.gx_votes[i].vote = gmu->gpu_freqs[i] / 1000; + } + + for (i = 0; i < gmu->nr_gmu_freqs; i++) { + msg.cx_votes[i].vote = gmu->cx_arc_votes[i]; + msg.cx_votes[i].vote = gmu->gmu_freqs[i] / 1000; + } + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_PERF_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_bw_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_bw_table msg = { 0 }; + + /* + * The sdm845 GMU doesn't do bus frequency scaling on its own but it + * does need at least one entry in the list because it might be accessed + * when the GMU is shutting down. Send a single "off" entry. + */ + + msg.bw_level_num = 1; + + msg.ddr_cmds_num = 3; + msg.ddr_wait_bitmask = 0x07; + + msg.ddr_cmds_addrs[0] = 0x50000; + msg.ddr_cmds_addrs[1] = 0x5005c; + msg.ddr_cmds_addrs[2] = 0x5000c; + + msg.ddr_cmds_data[0][0] = 0x40000000; + msg.ddr_cmds_data[0][1] = 0x40000000; + msg.ddr_cmds_data[0][2] = 0x40000000; + + /* + * These are the CX (CNOC) votes. This is used but the values for the + * sdm845 GMU are known and fixed so we can hard code them. + */ + + msg.cnoc_cmds_num = 3; + msg.cnoc_wait_bitmask = 0x05; + + msg.cnoc_cmds_addrs[0] = 0x50034; + msg.cnoc_cmds_addrs[1] = 0x5007c; + msg.cnoc_cmds_addrs[2] = 0x5004c; + + msg.cnoc_cmds_data[0][0] = 0x40000000; + msg.cnoc_cmds_data[0][1] = 0x00000000; + msg.cnoc_cmds_data[0][2] = 0x40000000; + + msg.cnoc_cmds_data[1][0] = 0x60000001; + msg.cnoc_cmds_data[1][1] = 0x20000001; + msg.cnoc_cmds_data[1][2] = 0x60000001; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_BW_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_test(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_test msg = { 0 }; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_TEST, &msg, sizeof(msg), + NULL, 0); +} + +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state) +{ + int ret; + + ret = a6xx_hfi_send_gmu_init(gmu, boot_state); + if (ret) + return ret; + + ret = a6xx_hfi_get_fw_version(gmu, NULL); + if (ret) + return ret; + + /* + * We have to get exchange version numbers per the sequence but at this + * point th kernel driver doesn't need to know the exact version of + * the GMU firmware + */ + + ret = a6xx_hfi_send_perf_table(gmu); + if (ret) + return ret; + + ret = a6xx_hfi_send_bw_table(gmu); + if (ret) + return ret; + + /* + * Let the GMU know that there won't be any more HFI messages until next + * boot + */ + a6xx_hfi_send_test(gmu); + + return 0; +} + +void a6xx_hfi_stop(struct a6xx_gmu *gmu) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(gmu->queues); i++) { + struct a6xx_hfi_queue *queue = &gmu->queues[i]; + + if (!queue->header) + continue; + + if (queue->header->read_index != queue->header->write_index) + dev_err(gmu->dev, "HFI queue %d is not empty\n", i); + + queue->header->read_index = 0; + queue->header->write_index = 0; + } +} + +static void a6xx_hfi_queue_init(struct a6xx_hfi_queue *queue, + struct a6xx_hfi_queue_header *header, void *virt, u64 iova, + u32 id) +{ + spin_lock_init(&queue->lock); + queue->header = header; + queue->data = virt; + atomic_set(&queue->seqnum, 0); + + /* Set up the shared memory header */ + header->iova = iova; + header->type = 10 << 8 | id; + header->status = 1; + header->size = SZ_4K >> 2; + header->msg_size = 0; + header->dropped = 0; + header->rx_watermark = 1; + header->tx_watermark = 1; + header->rx_request = 1; + header->tx_request = 0; + header->read_index = 0; + header->write_index = 0; +} + +void a6xx_hfi_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gmu_bo *hfi = gmu->hfi; + struct a6xx_hfi_queue_table_header *table = hfi->virt; + struct a6xx_hfi_queue_header *headers = hfi->virt + sizeof(*table); + u64 offset; + int table_size; + + /* + * The table size is the size of the table header plus all of the queue + * headers + */ + table_size = sizeof(*table); + table_size += (ARRAY_SIZE(gmu->queues) * + sizeof(struct a6xx_hfi_queue_header)); + + table->version = 0; + table->size = table_size; + /* First queue header is located immediately after the table header */ + table->qhdr0_offset = sizeof(*table) >> 2; + table->qhdr_size = sizeof(struct a6xx_hfi_queue_header) >> 2; + table->num_queues = ARRAY_SIZE(gmu->queues); + table->active_queues = ARRAY_SIZE(gmu->queues); + + /* Command queue */ + offset = SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[0], &headers[0], hfi->virt + offset, + hfi->iova + offset, 0); + + /* GMU response queue */ + offset += SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[1], &headers[1], hfi->virt + offset, + hfi->iova + offset, 4); +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.h b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h new file mode 100644 index 000000000000..60d1319fa44f --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_HFI_H_ +#define _A6XX_HFI_H_ + +struct a6xx_hfi_queue_table_header { + u32 version; + u32 size; /* Size of the queue table in dwords */ + u32 qhdr0_offset; /* Offset of the first queue header */ + u32 qhdr_size; /* Size of the queue headers */ + u32 num_queues; /* Number of total queues */ + u32 active_queues; /* Number of active queues */ +}; + +struct a6xx_hfi_queue_header { + u32 status; + u32 iova; + u32 type; + u32 size; + u32 msg_size; + u32 dropped; + u32 rx_watermark; + u32 tx_watermark; + u32 rx_request; + u32 tx_request; + u32 read_index; + u32 write_index; +}; + +struct a6xx_hfi_queue { + struct a6xx_hfi_queue_header *header; + spinlock_t lock; + u32 *data; + atomic_t seqnum; +}; + +/* This is the outgoing queue to the GMU */ +#define HFI_COMMAND_QUEUE 0 + +/* THis is the incoming response queue from the GMU */ +#define HFI_RESPONSE_QUEUE 1 + +#define HFI_HEADER_ID(msg) ((msg) & 0xff) +#define HFI_HEADER_SIZE(msg) (((msg) >> 8) & 0xff) +#define HFI_HEADER_SEQNUM(msg) (((msg) >> 20) & 0xfff) + +/* FIXME: Do we need this or can we use ARRAY_SIZE? */ +#define HFI_RESPONSE_PAYLOAD_SIZE 16 + +/* HFI message types */ + +#define HFI_MSG_CMD 0 +#define HFI_MSG_ACK 2 + +#define HFI_F2H_MSG_ACK 126 + +struct a6xx_hfi_msg_response { + u32 header; + u32 ret_header; + u32 error; + u32 payload[HFI_RESPONSE_PAYLOAD_SIZE]; +}; + +#define HFI_F2H_MSG_ERROR 100 + +struct a6xx_hfi_msg_error { + u32 header; + u32 code; + u32 payload[2]; +}; + +#define HFI_H2F_MSG_INIT 0 + +struct a6xx_hfi_msg_gmu_init_cmd { + u32 header; + u32 seg_id; + u32 dbg_buffer_addr; + u32 dbg_buffer_size; + u32 boot_state; +}; + +#define HFI_H2F_MSG_FW_VERSION 1 + +struct a6xx_hfi_msg_fw_version { + u32 header; + u32 supported_version; +}; + +#define HFI_H2F_MSG_PERF_TABLE 4 + +struct perf_level { + u32 vote; + u32 freq; +}; + +struct a6xx_hfi_msg_perf_table { + u32 header; + u32 num_gpu_levels; + u32 num_gmu_levels; + + struct perf_level gx_votes[16]; + struct perf_level cx_votes[4]; +}; + +#define HFI_H2F_MSG_BW_TABLE 3 + +struct a6xx_hfi_msg_bw_table { + u32 header; + u32 bw_level_num; + u32 cnoc_cmds_num; + u32 ddr_cmds_num; + u32 cnoc_wait_bitmask; + u32 ddr_wait_bitmask; + u32 cnoc_cmds_addrs[6]; + u32 cnoc_cmds_data[2][6]; + u32 ddr_cmds_addrs[8]; + u32 ddr_cmds_data[16][8]; +}; + +#define HFI_H2F_MSG_TEST 5 + +struct a6xx_hfi_msg_test { + u32 header; +}; + +#endif diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index 37746f1d54cf..7d3e9a129ac7 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -111,6 +111,16 @@ static const struct adreno_info gpulist[] = { ADRENO_QUIRK_FAULT_DETECT_MASK, .init = a5xx_gpu_init, .zapfw = "a530_zap.mdt", + }, { + .rev = ADRENO_REV(6, 3, 0, ANY_ID), + .revn = 630, + .name = "A630", + .fw = { + [ADRENO_FW_SQE] = "a630_sqe.fw", + [ADRENO_FW_GMU] = "a630_gmu.bin", + }, + .gmem = SZ_1M, + .init = a6xx_gpu_init, }, }; @@ -127,6 +137,8 @@ MODULE_FIRMWARE("qcom/a530_zap.mdt"); MODULE_FIRMWARE("qcom/a530_zap.b00"); MODULE_FIRMWARE("qcom/a530_zap.b01"); MODULE_FIRMWARE("qcom/a530_zap.b02"); +MODULE_FIRMWARE("qcom/a630_sqe.fw"); +MODULE_FIRMWARE("qcom/a630_gmu.bin"); static inline bool _rev_match(uint8_t entry, uint8_t id) { diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index 81397935d847..1a8a1d6ac99b 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -50,7 +50,9 @@ enum adreno_regs { enum { ADRENO_FW_PM4 = 0, + ADRENO_FW_SQE = 0, /* a6xx */ ADRENO_FW_PFP = 1, + ADRENO_FW_GMU = 1, /* a6xx */ ADRENO_FW_GPMU = 2, ADRENO_FW_MAX, }; @@ -329,6 +331,7 @@ static inline void adreno_gpu_write(struct adreno_gpu *gpu, struct msm_gpu *a3xx_gpu_init(struct drm_device *dev); struct msm_gpu *a4xx_gpu_init(struct drm_device *dev); struct msm_gpu *a5xx_gpu_init(struct drm_device *dev); +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev); static inline void adreno_gpu_write64(struct adreno_gpu *gpu, enum adreno_regs lo, enum adreno_regs hi, u64 data) diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index 827644f0e1a7..853eb5f161ee 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -87,7 +87,7 @@ static struct devfreq_dev_profile msm_devfreq_profile = { static void msm_devfreq_init(struct msm_gpu *gpu) { /* We need target support to do devfreq */ - if (!gpu->funcs->gpu_busy) + if (!gpu->funcs->gpu_busy || !gpu->core_clk) return; msm_devfreq_profile.initial_freq = gpu->fast_rate; -- 2.17.1 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
* [v7 PATCH 0/5] Add support for Adreno a6xx
@ 2018-08-06 17:33 Jordan Crouse
[not found] ` <20180806173324.16074-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
0 siblings, 1 reply; 12+ messages in thread
From: Jordan Crouse @ 2018-08-06 17:33 UTC (permalink / raw)
To: freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA,
hoegsberg-Re5JQEeQqe8AvxtiuMwx3w,
dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW
This is an initial version of support for the Adreno a6xx GPU family starting
with the a630 from the sdm845 SoC. This code is ahead of much of the sdm845
code that would be needed to actually bring up a device and it is also in
advance of any user side support for the a6xx GPU so this is mainly just a
chance to look over the code structure and get a feel for the
direction that the hardware is going in.
The a6xx GPU is an iteration of a5xx so most of the GPU side code looks pretty
identical except for the usual register differences. The big different is in
power control. On the a5xx there was a rudimentary device called the GPMU that
did some basic power stuff but left most of the complexity to the kernel.
On the a6xx the power complexity is being moved to a component
called the GMU (graphics management unit) which handles the power control of the
GPU and shuts it down quickly after it goes idle.
This stack has one run time dependency that shifts the power responsibility
for the SMMU to the SMMU driver instead of the current workaround of implicitly
powering the SMMU by powering the GPU:
https://patchwork.kernel.org/patch/10301163/ - pm_runtime ops for arm-smmu
[v7 - Make sure the GMU sets the GPU clock to something reasonable. Force the
GPU/GMU on during state capture to avoid errors. Fix checkpatch errors]
[v6 - set nr_clocks to 0 when parsing fails and minor cleanup in msm_iommu for
Vivek. Fix compile error in the load firmware patch ]
[v5 - Move firmware loading to adreno_load_gpu() so it is easier to fail if the
firmware isn't found ]
[v4 - Add clock parsing helper function and use it for GPU and GMU. Update
recommended clock gating settings. Fix a bug in the CMD DB parser. Update
register values from updated database ]
[v3 - fix inverted register definition for GMU_SPTPRAC_CLK_STATUS; fix incorrect
register check in a5xx_gmu_gx_is_on(), use dev_pm_opp_get_of_node() from
Rajendra and Viresh to read the qcom,level from the device tree; read qcom,level
from the DT to get the voltage level to pass to the GMU, fix issues
identified by smatch]
[v2 - addressed comments from Lucas Stach; added pm_runtime_get_supplier calls
for accesses to the GMU IOMMU; moved to SPDX headers for new files]
Jordan Crouse (4):
drm/msm: Remove pm_runtime operations from msm_iommu
drm/msm: Add a helper function to parse clock names
drm/msm/adreno: Load the firmware before bringing up the hardware
drm/msm: Add A6XX device support
Sharat Masetty (1):
drm/msm: Add generated headers for A6XX
drivers/gpu/drm/msm/Makefile | 3 +
drivers/gpu/drm/msm/adreno/a6xx.xml.h | 1784 ++++++++++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1207 +++++++++++++
drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 ++
drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h | 382 +++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 818 +++++++++
drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 +
drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 +++++
drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 ++
drivers/gpu/drm/msm/adreno/adreno_device.c | 35 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 2 +-
drivers/gpu/drm/msm/adreno/adreno_gpu.h | 5 +-
drivers/gpu/drm/msm/msm_drv.c | 57 +
drivers/gpu/drm/msm/msm_drv.h | 4 +
drivers/gpu/drm/msm/msm_gpu.c | 62 +-
drivers/gpu/drm/msm/msm_gpu.h | 2 +-
drivers/gpu/drm/msm/msm_iommu.c | 13 +-
17 files changed, 5091 insertions(+), 67 deletions(-)
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.xml.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c
create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h
--
2.18.0
_______________________________________________
Freedreno mailing list
Freedreno@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/freedreno
^ permalink raw reply [flat|nested] 12+ messages in thread[parent not found: <20180806173324.16074-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>]
* [PATCH 5/5] drm/msm: Add A6XX device support [not found] ` <20180806173324.16074-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org> @ 2018-08-06 17:33 ` Jordan Crouse 0 siblings, 0 replies; 12+ messages in thread From: Jordan Crouse @ 2018-08-06 17:33 UTC (permalink / raw) To: freedreno-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Cc: linux-arm-msm-u79uwXL29TY76Z2rM5mHXA, hoegsberg-Re5JQEeQqe8AvxtiuMwx3w, dri-devel-PD4FTy7X32lNgt0PjOBp9y5qC8QIuHrW Add support for the A6XX family of Adreno GPUs. The biggest addition is the GMU (Graphics Management Unit) which takes over most of the power management of the GPU itself but in a ironic twist of fate needs a goodly amount of management itself. Add support for the A6XX core code, the GMU and the HFI (hardware firmware interface) queue that the CPU uses to communicate with the GMU. Signed-off-by: Jordan Crouse <jcrouse@codeaurora.org> --- drivers/gpu/drm/msm/Makefile | 3 + drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 1207 ++++++++++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gmu.h | 162 +++ drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 818 +++++++++++++ drivers/gpu/drm/msm/adreno/a6xx_gpu.h | 60 + drivers/gpu/drm/msm/adreno/a6xx_hfi.c | 435 +++++++ drivers/gpu/drm/msm/adreno/a6xx_hfi.h | 127 ++ drivers/gpu/drm/msm/adreno/adreno_device.c | 12 + drivers/gpu/drm/msm/adreno/adreno_gpu.h | 3 + drivers/gpu/drm/msm/msm_gpu.c | 2 +- 10 files changed, 2828 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gmu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_gpu.h create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.c create mode 100644 drivers/gpu/drm/msm/adreno/a6xx_hfi.h diff --git a/drivers/gpu/drm/msm/Makefile b/drivers/gpu/drm/msm/Makefile index 7c773e003663..261fa79d456d 100644 --- a/drivers/gpu/drm/msm/Makefile +++ b/drivers/gpu/drm/msm/Makefile @@ -11,6 +11,9 @@ msm-y := \ adreno/a5xx_gpu.o \ adreno/a5xx_power.o \ adreno/a5xx_preempt.o \ + adreno/a6xx_gpu.o \ + adreno/a6xx_gmu.o \ + adreno/a6xx_hfi.o \ hdmi/hdmi.o \ hdmi/hdmi_audio.o \ hdmi/hdmi_bridge.o \ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c new file mode 100644 index 000000000000..fbb501986720 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c @@ -0,0 +1,1207 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/clk.h> +#include <linux/iopoll.h> +#include <linux/pm_opp.h> +#include <soc/qcom/cmd-db.h> + +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static irqreturn_t a6xx_gmu_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_STATUS); + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, status); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE) { + dev_err_ratelimited(gmu->dev, "GMU watchdog expired\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR) + dev_err_ratelimited(gmu->dev, "GMU AHB bus error\n"); + + if (status & A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + dev_err_ratelimited(gmu->dev, "GMU fence error: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_AHB_FENCE_STATUS)); + + return IRQ_HANDLED; +} + +static irqreturn_t a6xx_hfi_irq(int irq, void *data) +{ + struct a6xx_gmu *gmu = data; + u32 status; + + status = gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, status); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ) + tasklet_schedule(&gmu->hfi_tasklet); + + if (status & A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) { + dev_err_ratelimited(gmu->dev, "GMU firmware fault\n"); + + /* Temporary until we can recover safely */ + BUG(); + } + + return IRQ_HANDLED; +} + +/* Check to see if the GX rail is still powered */ +static bool a6xx_gmu_gx_is_on(struct a6xx_gmu *gmu) +{ + u32 val = gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS); + + return !(val & + (A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_GDSC_POWER_OFF | + A6XX_GMU_SPTPRAC_PWR_CLK_STATUS_GX_HM_CLK_OFF)); +} + +static int a6xx_gmu_set_freq(struct a6xx_gmu *gmu, int index) +{ + gmu_write(gmu, REG_A6XX_GMU_DCVS_ACK_OPTION, 0); + + gmu_write(gmu, REG_A6XX_GMU_DCVS_PERF_SETTING, + ((index << 24) & 0xff) | (3 & 0xf)); + + /* + * Send an invalid index as a vote for the bus bandwidth and let the + * firmware decide on the right vote + */ + gmu_write(gmu, REG_A6XX_GMU_DCVS_BW_SETTING, 0xff); + + /* Set and clear the OOB for DCVS to trigger the GMU */ + a6xx_gmu_set_oob(gmu, GMU_OOB_DCVS_SET); + a6xx_gmu_clear_oob(gmu, GMU_OOB_DCVS_SET); + + return gmu_read(gmu, REG_A6XX_GMU_DCVS_RETURN); +} + +static bool a6xx_gmu_check_idle_level(struct a6xx_gmu *gmu) +{ + u32 val; + int local = gmu->idle_level; + + /* SPTP and IFPC both report as IFPC */ + if (gmu->idle_level == GMU_IDLE_STATE_SPTP) + local = GMU_IDLE_STATE_IFPC; + + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val == local) { + if (gmu->idle_level != GMU_IDLE_STATE_IFPC || + !a6xx_gmu_gx_is_on(gmu)) + return true; + } + + return false; +} + +/* Wait for the GMU to get to its most idle state */ +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + return spin_until(a6xx_gmu_check_idle_level(gmu)); +} + +static int a6xx_gmu_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 1); + gmu_write(gmu, REG_A6XX_GMU_CM3_SYSRESET, 0); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, val, + val == 0xbabeface, 100, 10000); + + if (ret) + dev_err(gmu->dev, "GMU firmware initalization timed out\n"); + + return ret; +} + +static int a6xx_gmu_hfi_start(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + gmu_rmw(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ, 0); + + gmu_write(gmu, REG_A6XX_GMU_HFI_CTRL_INIT, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_HFI_CTRL_STATUS, val, + val & 1, 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to start the HFI queues\n"); + + return ret; +} + +/* Trigger a OOB (out of band) request to the GMU */ +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + int ret; + u32 val; + int request, ack; + const char *name; + + switch (state) { + case GMU_OOB_GPU_SET: + request = GMU_OOB_GPU_SET_REQUEST; + ack = GMU_OOB_GPU_SET_ACK; + name = "GPU_SET"; + break; + case GMU_OOB_BOOT_SLUMBER: + request = GMU_OOB_BOOT_SLUMBER_REQUEST; + ack = GMU_OOB_BOOT_SLUMBER_ACK; + name = "BOOT_SLUMBER"; + break; + case GMU_OOB_DCVS_SET: + request = GMU_OOB_DCVS_REQUEST; + ack = GMU_OOB_DCVS_ACK; + name = "GPU_DCVS"; + break; + default: + return -EINVAL; + } + + /* Trigger the equested OOB operation */ + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 1 << request); + + /* Wait for the acknowledge interrupt */ + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO, val, + val & (1 << ack), 100, 10000); + + if (ret) + dev_err(gmu->dev, + "Timeout waiting for GMU OOB set %s: 0x%x\n", + name, + gmu_read(gmu, REG_A6XX_GMU_GMU2HOST_INTR_INFO)); + + /* Clear the acknowledge interrupt */ + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, 1 << ack); + + return ret; +} + +/* Clear a pending OOB state in the GMU */ +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state) +{ + switch (state) { + case GMU_OOB_GPU_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_GPU_SET_CLEAR); + break; + case GMU_OOB_BOOT_SLUMBER: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_BOOT_SLUMBER_CLEAR); + break; + case GMU_OOB_DCVS_SET: + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, + 1 << GMU_OOB_DCVS_CLEAR); + break; + } +} + +/* Enable CPU control of SPTP power power collapse */ +static int a6xx_sptprac_enable(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778000); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x38) == 0x28, 1, 100); + + if (ret) { + dev_err(gmu->dev, "Unable to power on SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); + } + + return 0; +} + +/* Disable CPU control of SPTP power power collapse */ +static void a6xx_sptprac_disable(struct a6xx_gmu *gmu) +{ + u32 val; + int ret; + + /* Make sure retention is on */ + gmu_rmw(gmu, REG_A6XX_GPU_CC_GX_GDSCR, 0, (1 << 11)); + + gmu_write(gmu, REG_A6XX_GMU_GX_SPTPRAC_POWER_CONTROL, 0x778001); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS, val, + (val & 0x04), 100, 10000); + + if (ret) + dev_err(gmu->dev, "failed to power off SPTPRAC: 0x%x\n", + gmu_read(gmu, REG_A6XX_GMU_SPTPRAC_PWR_CLK_STATUS)); +} + +/* Let the GMU know we are starting a boot sequence */ +static int a6xx_gmu_gfx_rail_on(struct a6xx_gmu *gmu) +{ + u32 vote; + + /* Let the GMU know we are getting ready for boot */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 0); + + /* Choose the "default" power level as the highest available */ + vote = gmu->gx_arc_votes[gmu->nr_gpu_freqs - 1]; + + gmu_write(gmu, REG_A6XX_GMU_GX_VOTE_IDX, vote & 0xff); + gmu_write(gmu, REG_A6XX_GMU_MX_VOTE_IDX, (vote >> 8) & 0xff); + + /* Let the GMU know the boot sequence has started */ + return a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); +} + +/* Let the GMU know that we are about to go into slumber */ +static int a6xx_gmu_notify_slumber(struct a6xx_gmu *gmu) +{ + int ret; + + /* Disable the power counter so the GMU isn't busy */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 0); + + /* Disable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) + a6xx_sptprac_disable(gmu); + + /* Tell the GMU to get ready to slumber */ + gmu_write(gmu, REG_A6XX_GMU_BOOT_SLUMBER_OPTION, 1); + + ret = a6xx_gmu_set_oob(gmu, GMU_OOB_BOOT_SLUMBER); + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + if (!ret) { + /* Check to see if the GMU really did slumber */ + if (gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE) + != 0x0f) { + dev_err(gmu->dev, "The GMU did not go into slumber\n"); + ret = -ETIMEDOUT; + } + } + + /* Put fence into allow mode */ + gmu_write(gmu, REG_A6XX_GMU_AO_AHB_FENCE_CTRL, 0); + return ret; +} + +static int a6xx_rpmh_start(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1 << 1); + /* Wait for the register to finish posting */ + wmb(); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GMU_RSCC_CONTROL_ACK, val, + val & (1 << 1), 100, 10000); + if (ret) { + dev_err(gmu->dev, "Unable to power on the GPU RSC\n"); + return ret; + } + + ret = gmu_poll_timeout(gmu, REG_A6XX_RSCC_SEQ_BUSY_DRV0, val, + !val, 100, 10000); + + if (!ret) { + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); + + /* Re-enable the power counter */ + gmu_write(gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, 1); + return 0; + } + + dev_err(gmu->dev, "GPU RSC sequence stuck while waking up the GPU\n"); + return ret; +} + +static void a6xx_rpmh_stop(struct a6xx_gmu *gmu) +{ + int ret; + u32 val; + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 1); + + ret = gmu_poll_timeout(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, + val, val & (1 << 16), 100, 10000); + if (ret) + dev_err(gmu->dev, "Unable to power off the GPU RSC\n"); + + gmu_write(gmu, REG_A6XX_GMU_RSCC_CONTROL_REQ, 0); +} + +static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu) +{ + /* Disable SDE clock gating */ + gmu_write(gmu, REG_A6XX_GPU_RSCC_RSC_STATUS0_DRV0, BIT(24)); + + /* Setup RSC PDC handshake for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_PDC_SLAVE_ID_DRV0, 1); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 2, 0); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_DATA + 4, 0x80000000); + gmu_write(gmu, REG_A6XX_RSCC_HIDDEN_TCS_CMD0_ADDR + 4, 0); + gmu_write(gmu, REG_A6XX_RSCC_OVERRIDE_START_ADDR, 0); + gmu_write(gmu, REG_A6XX_RSCC_PDC_SEQ_START_ADDR, 0x4520); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_LO, 0x4510); + gmu_write(gmu, REG_A6XX_RSCC_PDC_MATCH_VALUE_HI, 0x4514); + + /* Load RSC sequencer uCode for sleep and wakeup */ + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0, 0xa7a506a0); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 1, 0xa1e6a6e7); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 2, 0xa2e081e1); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 3, 0xe9a982e2); + gmu_write(gmu, REG_A6XX_RSCC_SEQ_MEM_0_DRV0 + 4, 0x0020e8a8); + + /* Load PDC sequencer uCode for power up and power down sequence */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0, 0xfebea1e1); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 1, 0xa5a4a3a2); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 2, 0x8382a6e0); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 3, 0xbce3e284); + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_MEM_0 + 4, 0x002081fc); + + /* Set TCS commands used by PDC sequence for low power modes */ + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA, 1); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 4, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS1_CMD0_DATA + 8, 0x0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_ENABLE_BANK, 7); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD_WAIT_FOR_CMPL_BANK, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CONTROL, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR, 0x30010); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA, 2); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 4, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 4, 0x30000); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 4, 0x3); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_MSGID + 8, 0x10108); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_ADDR + 8, 0x30080); + pdc_write(gmu, REG_A6XX_PDC_GPU_TCS3_CMD0_DATA + 8, 0x3); + + /* Setup GPU PDC */ + pdc_write(gmu, REG_A6XX_PDC_GPU_SEQ_START_ADDR, 0); + pdc_write(gmu, REG_A6XX_PDC_GPU_ENABLE_PDC, 0x80000001); + + /* ensure no writes happen before the uCode is fully written */ + wmb(); +} + +/* + * The lowest 16 bits of this value are the number of XO clock cycles for main + * hysteresis which is set at 0x1680 cycles (300 us). The higher 16 bits are + * for the shorter hysteresis that happens after main - this is 0xa (.5 us) + */ + +#define GMU_PWR_COL_HYST 0x000a1680 + +/* Set up the idle state for the GMU */ +static void a6xx_gmu_power_config(struct a6xx_gmu *gmu) +{ + /* Disable GMU WB/RB buffer */ + gmu_write(gmu, REG_A6XX_GMU_SYS_BUS_CONFIG, 0x1); + + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0x9c40400); + + switch (gmu->idle_level) { + case GMU_IDLE_STATE_IFPC: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_HM_POWER_COLLAPSE_ENABLE); + /* Fall through */ + case GMU_IDLE_STATE_SPTP: + gmu_write(gmu, REG_A6XX_GMU_PWR_COL_SPTPRAC_HYST, + GMU_PWR_COL_HYST); + gmu_rmw(gmu, REG_A6XX_GMU_PWR_COL_INTER_FRAME_CTRL, 0, + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_IFPC_ENABLE | + A6XX_GMU_PWR_COL_INTER_FRAME_CTRL_SPTPRAC_POWER_CONTROL_ENABLE); + } + + /* Enable RPMh GPU client */ + gmu_rmw(gmu, REG_A6XX_GMU_RPMH_CTRL, 0, + A6XX_GMU_RPMH_CTRL_RPMH_INTERFACE_ENABLE | + A6XX_GMU_RPMH_CTRL_LLC_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_DDR_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_MX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_CX_VOTE_ENABLE | + A6XX_GMU_RPMH_CTRL_GFX_VOTE_ENABLE); +} + +static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state) +{ + static bool rpmh_init; + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + int i, ret; + u32 chipid; + u32 *image; + + if (state == GMU_WARM_BOOT) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } else { + if (WARN(!adreno_gpu->fw[ADRENO_FW_GMU], + "GMU firmware is not loaded\n")) + return -ENOENT; + + /* Sanity check the size of the firmware that was loaded */ + if (adreno_gpu->fw[ADRENO_FW_GMU]->size > 0x8000) { + dev_err(gmu->dev, + "GMU firmware is bigger than the available region\n"); + return -EINVAL; + } + + /* Turn on register retention */ + gmu_write(gmu, REG_A6XX_GMU_GENERAL_7, 1); + + /* We only need to load the RPMh microcode once */ + if (!rpmh_init) { + a6xx_gmu_rpmh_init(gmu); + rpmh_init = true; + } else if (state != GMU_RESET) { + ret = a6xx_rpmh_start(gmu); + if (ret) + return ret; + } + + image = (u32 *) adreno_gpu->fw[ADRENO_FW_GMU]->data; + + for (i = 0; i < adreno_gpu->fw[ADRENO_FW_GMU]->size >> 2; i++) + gmu_write(gmu, REG_A6XX_GMU_CM3_ITCM_START + i, + image[i]); + } + + gmu_write(gmu, REG_A6XX_GMU_CM3_FW_INIT_RESULT, 0); + gmu_write(gmu, REG_A6XX_GMU_CM3_BOOT_CONFIG, 0x02); + + /* Write the iova of the HFI table */ + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_ADDR, gmu->hfi->iova); + gmu_write(gmu, REG_A6XX_GMU_HFI_QTBL_INFO, 1); + + gmu_write(gmu, REG_A6XX_GMU_AHB_FENCE_RANGE_0, + (1 << 31) | (0xa << 18) | (0xa0)); + + chipid = adreno_gpu->rev.core << 24; + chipid |= adreno_gpu->rev.major << 16; + chipid |= adreno_gpu->rev.minor << 12; + chipid |= adreno_gpu->rev.patchid << 8; + + gmu_write(gmu, REG_A6XX_GMU_HFI_SFR_ADDR, chipid); + + /* Set up the lowest idle level on the GMU */ + a6xx_gmu_power_config(gmu); + + ret = a6xx_gmu_start(gmu); + if (ret) + return ret; + + ret = a6xx_gmu_gfx_rail_on(gmu); + if (ret) + return ret; + + /* Enable SPTP_PC if the CPU is responsible for it */ + if (gmu->idle_level < GMU_IDLE_STATE_SPTP) { + ret = a6xx_sptprac_enable(gmu); + if (ret) + return ret; + } + + ret = a6xx_gmu_hfi_start(gmu); + if (ret) + return ret; + + /* FIXME: Do we need this wmb() here? */ + wmb(); + + return 0; +} + +#define A6XX_HFI_IRQ_MASK \ + (A6XX_GMU_GMU2HOST_INTR_INFO_MSGQ | \ + A6XX_GMU_GMU2HOST_INTR_INFO_CM3_FAULT) + +#define A6XX_GMU_IRQ_MASK \ + (A6XX_GMU_AO_HOST_INTERRUPT_STATUS_WDOG_BITE | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_HOST_AHB_BUS_ERROR | \ + A6XX_GMU_AO_HOST_INTERRUPT_STATUS_FENCE_ERR) + +static void a6xx_gmu_irq_enable(struct a6xx_gmu *gmu) +{ + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_CLR, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_CLR, ~0); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, + ~A6XX_GMU_IRQ_MASK); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, + ~A6XX_HFI_IRQ_MASK); + + enable_irq(gmu->gmu_irq); + enable_irq(gmu->hfi_irq); +} + +static void a6xx_gmu_irq_disable(struct a6xx_gmu *gmu) +{ + disable_irq(gmu->gmu_irq); + disable_irq(gmu->hfi_irq); + + gmu_write(gmu, REG_A6XX_GMU_AO_HOST_INTERRUPT_MASK, ~0); + gmu_write(gmu, REG_A6XX_GMU_GMU2HOST_INTR_MASK, ~0); +} + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int ret; + u32 val; + + /* Flush all the queues */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts */ + a6xx_gmu_irq_disable(gmu); + + /* Force off SPTP in case the GMU is managing it */ + a6xx_sptprac_disable(gmu); + + /* Make sure there are no outstanding RPMh votes */ + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS0_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS1_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS2_DRV0_STATUS, val, + (val & 1), 100, 10000); + gmu_poll_timeout(gmu, REG_A6XX_RSCC_TCS3_DRV0_STATUS, val, + (val & 1), 100, 1000); + + /* Force off the GX GSDC */ + regulator_force_disable(gmu->gx); + + /* Disable the resources */ + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + pm_runtime_put_sync(gmu->dev); + + /* Re-enable the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + ret = clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + if (ret) + goto out; + + a6xx_gmu_irq_enable(gmu); + + ret = a6xx_gmu_fw_start(gmu, GMU_RESET); + if (!ret) + ret = a6xx_hfi_start(gmu, GMU_COLD_BOOT); + + /* Set the GPU back to the highest power frequency */ + a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); + +out: + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +int a6xx_gmu_resume(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + int status, ret; + + if (WARN(!gmu->mmio, "The GMU is not set up yet\n")) + return 0; + + /* Turn on the resources */ + pm_runtime_get_sync(gmu->dev); + + /* Use a known rate to bring up the GMU */ + clk_set_rate(gmu->core_clk, 200000000); + ret = clk_bulk_prepare_enable(gmu->nr_clocks, gmu->clocks); + if (ret) + goto out; + + a6xx_gmu_irq_enable(gmu); + + /* Check to see if we are doing a cold or warm boot */ + status = gmu_read(gmu, REG_A6XX_GMU_GENERAL_7) == 1 ? + GMU_WARM_BOOT : GMU_COLD_BOOT; + + ret = a6xx_gmu_fw_start(gmu, status); + if (ret) + goto out; + + ret = a6xx_hfi_start(gmu, status); + + /* Set the GPU to the highest power frequency */ + a6xx_gmu_set_freq(gmu, gmu->nr_gpu_freqs - 1); + +out: + /* Make sure to turn off the boot OOB request on error */ + if (ret) + a6xx_gmu_clear_oob(gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu) +{ + u32 reg; + + if (!gmu->mmio) + return true; + + reg = gmu_read(gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS); + + if (reg & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB) + return false; + + return true; +} + +int a6xx_gmu_stop(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + u32 val; + + /* + * The GMU may still be in slumber unless the GPU started so check and + * skip putting it back into slumber if so + */ + val = gmu_read(gmu, REG_A6XX_GPU_GMU_CX_GMU_RPMH_POWER_STATE); + + if (val != 0xf) { + int ret = a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Temporary until we can recover safely */ + BUG_ON(ret); + + /* tell the GMU we want to slumber */ + a6xx_gmu_notify_slumber(gmu); + + ret = gmu_poll_timeout(gmu, + REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS, val, + !(val & A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS_GPUBUSYIGNAHB), + 100, 10000); + + /* + * Let the user know we failed to slumber but don't worry too + * much because we are powering down anyway + */ + + if (ret) + dev_err(gmu->dev, + "Unable to slumber GMU: status = 0%x/0%x\n", + gmu_read(gmu, + REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS), + gmu_read(gmu, + REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_STATUS2)); + } + + /* Turn off HFI */ + a6xx_hfi_stop(gmu); + + /* Stop the interrupts and mask the hardware */ + a6xx_gmu_irq_disable(gmu); + + /* Tell RPMh to power off the GPU */ + a6xx_rpmh_stop(gmu); + + clk_bulk_disable_unprepare(gmu->nr_clocks, gmu->clocks); + + pm_runtime_put_sync(gmu->dev); + + return 0; +} + +static void a6xx_gmu_memory_free(struct a6xx_gmu *gmu, struct a6xx_gmu_bo *bo) +{ + int count, i; + u64 iova; + + if (IS_ERR_OR_NULL(bo)) + return; + + count = bo->size >> PAGE_SHIFT; + iova = bo->iova; + + for (i = 0; i < count; i++, iova += PAGE_SIZE) { + iommu_unmap(gmu->domain, iova, PAGE_SIZE); + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); +} + +static struct a6xx_gmu_bo *a6xx_gmu_memory_alloc(struct a6xx_gmu *gmu, + size_t size) +{ + struct a6xx_gmu_bo *bo; + int ret, count, i; + + bo = kzalloc(sizeof(*bo), GFP_KERNEL); + if (!bo) + return ERR_PTR(-ENOMEM); + + bo->size = PAGE_ALIGN(size); + + count = bo->size >> PAGE_SHIFT; + + bo->pages = kcalloc(count, sizeof(struct page *), GFP_KERNEL); + if (!bo->pages) { + kfree(bo); + return ERR_PTR(-ENOMEM); + } + + for (i = 0; i < count; i++) { + bo->pages[i] = alloc_page(GFP_KERNEL); + if (!bo->pages[i]) + goto err; + } + + bo->iova = gmu->uncached_iova_base; + + for (i = 0; i < count; i++) { + ret = iommu_map(gmu->domain, + bo->iova + (PAGE_SIZE * i), + page_to_phys(bo->pages[i]), PAGE_SIZE, + IOMMU_READ | IOMMU_WRITE); + + if (ret) { + dev_err(gmu->dev, "Unable to map GMU buffer object\n"); + + for (i = i - 1 ; i >= 0; i--) + iommu_unmap(gmu->domain, + bo->iova + (PAGE_SIZE * i), + PAGE_SIZE); + + goto err; + } + } + + bo->virt = vmap(bo->pages, count, VM_IOREMAP, + pgprot_writecombine(PAGE_KERNEL)); + if (!bo->virt) + goto err; + + /* Align future IOVA addresses on 1MB boundaries */ + gmu->uncached_iova_base += ALIGN(size, SZ_1M); + + return bo; + +err: + for (i = 0; i < count; i++) { + if (bo->pages[i]) + __free_pages(bo->pages[i], 0); + } + + kfree(bo->pages); + kfree(bo); + + return ERR_PTR(-ENOMEM); +} + +static int a6xx_gmu_memory_probe(struct a6xx_gmu *gmu) +{ + int ret; + + /* + * The GMU address space is hardcoded to treat the range + * 0x60000000 - 0x80000000 as un-cached memory. All buffers shared + * between the GMU and the CPU will live in this space + */ + gmu->uncached_iova_base = 0x60000000; + + + gmu->domain = iommu_domain_alloc(&platform_bus_type); + if (!gmu->domain) + return -ENODEV; + + ret = iommu_attach_device(gmu->domain, gmu->dev); + + if (ret) { + iommu_domain_free(gmu->domain); + gmu->domain = NULL; + } + + return ret; +} + +/* Get the list of RPMh voltage levels from cmd-db */ +static int a6xx_gmu_rpmh_arc_cmds(const char *id, void *vals, int size) +{ + u32 len = cmd_db_read_aux_data_len(id); + + if (!len) + return 0; + + if (WARN_ON(len > size)) + return -EINVAL; + + cmd_db_read_aux_data(id, vals, len); + + /* + * The data comes back as an array of unsigned shorts so adjust the + * count accordingly + */ + return len >> 1; +} + +/* Return the 'arc-level' for the given frequency */ +static u32 a6xx_gmu_get_arc_level(struct device *dev, unsigned long freq) +{ + struct dev_pm_opp *opp; + struct device_node *np; + u32 val = 0; + + if (!freq) + return 0; + + opp = dev_pm_opp_find_freq_exact(dev, freq, true); + if (IS_ERR(opp)) + return 0; + + np = dev_pm_opp_get_of_node(opp); + + if (np) { + of_property_read_u32(np, "qcom,level", &val); + of_node_put(np); + } + + dev_pm_opp_put(opp); + + return val; +} + +static int a6xx_gmu_rpmh_arc_votes_init(struct device *dev, u32 *votes, + unsigned long *freqs, int freqs_count, + u16 *pri, int pri_count, + u16 *sec, int sec_count) +{ + int i, j; + + /* Construct a vote for each frequency */ + for (i = 0; i < freqs_count; i++) { + u8 pindex = 0, sindex = 0; + u32 level = a6xx_gmu_get_arc_level(dev, freqs[i]); + + /* Get the primary index that matches the arc level */ + for (j = 0; j < pri_count; j++) { + if (pri[j] >= level) { + pindex = j; + break; + } + } + + if (j == pri_count) { + dev_err(dev, + "Level %u not found in in the RPMh list\n", + level); + dev_err(dev, "Available levels:\n"); + for (j = 0; j < pri_count; j++) + dev_err(dev, " %u\n", pri[j]); + + return -EINVAL; + } + + /* + * Look for a level in in the secondary list that matches. If + * nothing fits, use the maximum non zero vote + */ + + for (j = 0; j < sec_count; j++) { + if (sec[j] >= level) { + sindex = j; + break; + } else if (sec[j]) { + sindex = j; + } + } + + /* Construct the vote */ + votes[i] = ((pri[pindex] & 0xffff) << 16) | + (sindex << 8) | pindex; + } + + return 0; +} + +/* + * The GMU votes with the RPMh for itself and on behalf of the GPU but we need + * to construct the list of votes on the CPU and send it over. Query the RPMh + * voltage levels and build the votes + */ + +static int a6xx_gmu_rpmh_votes_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + u16 gx[16], cx[16], mx[16]; + u32 gxcount, cxcount, mxcount; + int ret; + + /* Get the list of available voltage levels for each component */ + gxcount = a6xx_gmu_rpmh_arc_cmds("gfx.lvl", gx, sizeof(gx)); + cxcount = a6xx_gmu_rpmh_arc_cmds("cx.lvl", cx, sizeof(cx)); + mxcount = a6xx_gmu_rpmh_arc_cmds("mx.lvl", mx, sizeof(mx)); + + /* Build the GX votes */ + ret = a6xx_gmu_rpmh_arc_votes_init(&gpu->pdev->dev, gmu->gx_arc_votes, + gmu->gpu_freqs, gmu->nr_gpu_freqs, + gx, gxcount, mx, mxcount); + + /* Build the CX votes */ + ret |= a6xx_gmu_rpmh_arc_votes_init(gmu->dev, gmu->cx_arc_votes, + gmu->gmu_freqs, gmu->nr_gmu_freqs, + cx, cxcount, mx, mxcount); + + return ret; +} + +static int a6xx_gmu_build_freq_table(struct device *dev, unsigned long *freqs, + u32 size) +{ + int count = dev_pm_opp_get_opp_count(dev); + struct dev_pm_opp *opp; + int i, index = 0; + unsigned long freq = 1; + + /* + * The OPP table doesn't contain the "off" frequency level so we need to + * add 1 to the table size to account for it + */ + + if (WARN(count + 1 > size, + "The GMU frequency table is being truncated\n")) + count = size - 1; + + /* Set the "off" frequency */ + freqs[index++] = 0; + + for (i = 0; i < count; i++) { + opp = dev_pm_opp_find_freq_ceil(dev, &freq); + if (IS_ERR(opp)) + break; + + dev_pm_opp_put(opp); + freqs[index++] = freq++; + } + + return index; +} + +static int a6xx_gmu_pwrlevels_probe(struct a6xx_gmu *gmu) +{ + struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu); + struct adreno_gpu *adreno_gpu = &a6xx_gpu->base; + struct msm_gpu *gpu = &adreno_gpu->base; + + int ret = 0; + + /* + * The GMU handles its own frequency switching so build a list of + * available frequencies to send during initalization + */ + ret = dev_pm_opp_of_add_table(gmu->dev); + if (ret) { + dev_err(gmu->dev, "Unable to set the OPP table for the GMU\n"); + return ret; + } + + gmu->nr_gmu_freqs = a6xx_gmu_build_freq_table(gmu->dev, + gmu->gmu_freqs, ARRAY_SIZE(gmu->gmu_freqs)); + + /* + * The GMU also handles GPU frequency switching so build a list + * from the GPU OPP table + */ + gmu->nr_gpu_freqs = a6xx_gmu_build_freq_table(&gpu->pdev->dev, + gmu->gpu_freqs, ARRAY_SIZE(gmu->gpu_freqs)); + + /* Build the list of RPMh votes that we'll send to the GMU */ + return a6xx_gmu_rpmh_votes_init(gmu); +} + +static int a6xx_gmu_clocks_probe(struct a6xx_gmu *gmu) +{ + int ret = msm_clk_bulk_get(gmu->dev, &gmu->clocks); + + if (ret < 1) + return ret; + + gmu->nr_clocks = ret; + + gmu->core_clk = msm_clk_bulk_get_clock(gmu->clocks, + gmu->nr_clocks, "gmu"); + + return 0; +} + +static void __iomem *a6xx_gmu_get_mmio(struct platform_device *pdev, + const char *name) +{ + void __iomem *ret; + struct resource *res = platform_get_resource_byname(pdev, + IORESOURCE_MEM, name); + + if (!res) { + dev_err(&pdev->dev, "Unable to find the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + ret = devm_ioremap(&pdev->dev, res->start, resource_size(res)); + if (!ret) { + dev_err(&pdev->dev, "Unable to map the %s registers\n", name); + return ERR_PTR(-EINVAL); + } + + return ret; +} + +static int a6xx_gmu_get_irq(struct a6xx_gmu *gmu, struct platform_device *pdev, + const char *name, irq_handler_t handler) +{ + int irq, ret; + + irq = platform_get_irq_byname(pdev, name); + + ret = devm_request_irq(&pdev->dev, irq, handler, IRQF_TRIGGER_HIGH, + name, gmu); + if (ret) { + dev_err(&pdev->dev, "Unable to get interrupt %s\n", name); + return ret; + } + + disable_irq(irq); + + return irq; +} + +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + + if (IS_ERR_OR_NULL(gmu->mmio)) + return; + + pm_runtime_disable(gmu->dev); + a6xx_gmu_stop(a6xx_gpu); + + a6xx_gmu_irq_disable(gmu); + a6xx_gmu_memory_free(gmu, gmu->hfi); + + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); +} + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node) +{ + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + struct platform_device *pdev = of_find_device_by_node(node); + int ret; + + if (!pdev) + return -ENODEV; + + gmu->dev = &pdev->dev; + + of_dma_configure(gmu->dev, node, false); + + /* Fow now, don't do anything fancy until we get our feet under us */ + gmu->idle_level = GMU_IDLE_STATE_ACTIVE; + + pm_runtime_enable(gmu->dev); + gmu->gx = devm_regulator_get(gmu->dev, "vdd"); + + /* Get the list of clocks */ + ret = a6xx_gmu_clocks_probe(gmu); + if (ret) + return ret; + + /* Set up the IOMMU context bank */ + ret = a6xx_gmu_memory_probe(gmu); + if (ret) + return ret; + + /* Allocate memory for for the HFI queues */ + gmu->hfi = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->hfi)) + goto err; + + /* Allocate memory for the GMU debug region */ + gmu->debug = a6xx_gmu_memory_alloc(gmu, SZ_16K); + if (IS_ERR(gmu->debug)) + goto err; + + /* Map the GMU registers */ + gmu->mmio = a6xx_gmu_get_mmio(pdev, "gmu"); + + /* Map the GPU power domain controller registers */ + gmu->pdc_mmio = a6xx_gmu_get_mmio(pdev, "gmu_pdc"); + + if (IS_ERR(gmu->mmio) || IS_ERR(gmu->pdc_mmio)) + goto err; + + /* Get the HFI and GMU interrupts */ + gmu->hfi_irq = a6xx_gmu_get_irq(gmu, pdev, "hfi", a6xx_hfi_irq); + gmu->gmu_irq = a6xx_gmu_get_irq(gmu, pdev, "gmu", a6xx_gmu_irq); + + if (gmu->hfi_irq < 0 || gmu->gmu_irq < 0) + goto err; + + /* Set up a tasklet to handle GMU HFI responses */ + tasklet_init(&gmu->hfi_tasklet, a6xx_hfi_task, (unsigned long) gmu); + + /* Get the power levels for the GMU and GPU */ + a6xx_gmu_pwrlevels_probe(gmu); + + /* Set up the HFI queues */ + a6xx_hfi_init(gmu); + + return 0; +err: + a6xx_gmu_memory_free(gmu, gmu->hfi); + + if (gmu->domain) { + iommu_detach_device(gmu->domain, gmu->dev); + + iommu_domain_free(gmu->domain); + } + + return -ENODEV; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.h b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h new file mode 100644 index 000000000000..d9a386c18799 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.h @@ -0,0 +1,162 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_GMU_H_ +#define _A6XX_GMU_H_ + +#include <linux/interrupt.h> +#include "msm_drv.h" +#include "a6xx_hfi.h" + +struct a6xx_gmu_bo { + void *virt; + size_t size; + u64 iova; + struct page **pages; +}; + +/* + * These define the different GMU wake up options - these define how both the + * CPU and the GMU bring up the hardware + */ + +/* THe GMU has already been booted and the rentention registers are active */ +#define GMU_WARM_BOOT 0 + +/* the GMU is coming up for the first time or back from a power collapse */ +#define GMU_COLD_BOOT 1 + +/* The GMU is being soft reset after a fault */ +#define GMU_RESET 2 + +/* + * These define the level of control that the GMU has - the higher the number + * the more things that the GMU hardware controls on its own. + */ + +/* The GMU does not do any idle state management */ +#define GMU_IDLE_STATE_ACTIVE 0 + +/* The GMU manages SPTP power collapse */ +#define GMU_IDLE_STATE_SPTP 2 + +/* The GMU does automatic IFPC (intra-frame power collapse) */ +#define GMU_IDLE_STATE_IFPC 3 + +struct a6xx_gmu { + struct device *dev; + + void * __iomem mmio; + void * __iomem pdc_mmio; + + int hfi_irq; + int gmu_irq; + + struct regulator *gx; + + struct iommu_domain *domain; + u64 uncached_iova_base; + + int idle_level; + + struct a6xx_gmu_bo *hfi; + struct a6xx_gmu_bo *debug; + + int nr_clocks; + struct clk_bulk_data *clocks; + struct clk *core_clk; + + int nr_gpu_freqs; + unsigned long gpu_freqs[16]; + u32 gx_arc_votes[16]; + + int nr_gmu_freqs; + unsigned long gmu_freqs[4]; + u32 cx_arc_votes[4]; + + struct a6xx_hfi_queue queues[2]; + + struct tasklet_struct hfi_tasklet; +}; + +static inline u32 gmu_read(struct a6xx_gmu *gmu, u32 offset) +{ + return msm_readl(gmu->mmio + (offset << 2)); +} + +static inline void gmu_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->mmio + (offset << 2)); +} + +static inline void pdc_write(struct a6xx_gmu *gmu, u32 offset, u32 value) +{ + return msm_writel(value, gmu->pdc_mmio + (offset << 2)); +} + +static inline void gmu_rmw(struct a6xx_gmu *gmu, u32 reg, u32 mask, u32 or) +{ + u32 val = gmu_read(gmu, reg); + + val &= ~mask; + + gmu_write(gmu, reg, val | or); +} + +#define gmu_poll_timeout(gmu, addr, val, cond, interval, timeout) \ + readl_poll_timeout((gmu)->mmio + ((addr) << 2), val, cond, \ + interval, timeout) + +/* + * These are the available OOB (out of band requests) to the GMU where "out of + * band" means that the CPU talks to the GMU directly and not through HFI. + * Normally this works by writing a ITCM/DTCM register and then triggering a + * interrupt (the "request" bit) and waiting for an acknowledgment (the "ack" + * bit). The state is cleared by writing the "clear' bit to the GMU interrupt. + * + * These are used to force the GMU/GPU to stay on during a critical sequence or + * for hardware workarounds. + */ + +enum a6xx_gmu_oob_state { + GMU_OOB_BOOT_SLUMBER = 0, + GMU_OOB_GPU_SET, + GMU_OOB_DCVS_SET, +}; + +/* These are the interrupt / ack bits for each OOB request that are set + * in a6xx_gmu_set_oob and a6xx_clear_oob + */ + +/* + * Let the GMU know that a boot or slumber operation has started. The value in + * REG_A6XX_GMU_BOOT_SLUMBER_OPTION lets the GMU know which operation we are + * doing + */ +#define GMU_OOB_BOOT_SLUMBER_REQUEST 22 +#define GMU_OOB_BOOT_SLUMBER_ACK 30 +#define GMU_OOB_BOOT_SLUMBER_CLEAR 30 + +/* + * Set a new power level for the GPU when the CPU is doing frequency scaling + */ +#define GMU_OOB_DCVS_REQUEST 23 +#define GMU_OOB_DCVS_ACK 31 +#define GMU_OOB_DCVS_CLEAR 31 + +/* + * Let the GMU know to not turn off any GPU registers while the CPU is in a + * critical section + */ +#define GMU_OOB_GPU_SET_REQUEST 16 +#define GMU_OOB_GPU_SET_ACK 24 +#define GMU_OOB_GPU_SET_CLEAR 24 + + +void a6xx_hfi_init(struct a6xx_gmu *gmu); +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state); +void a6xx_hfi_stop(struct a6xx_gmu *gmu); + +void a6xx_hfi_task(unsigned long data); + +#endif diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c new file mode 100644 index 000000000000..c629f742a1d1 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c @@ -0,0 +1,818 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + + +#include "msm_gem.h" +#include "msm_mmu.h" +#include "a6xx_gpu.h" +#include "a6xx_gmu.xml.h" + +static inline bool _a6xx_check_idle(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Check that the GMU is idle */ + if (!a6xx_gmu_isidle(&a6xx_gpu->gmu)) + return false; + + /* Check tha the CX master is idle */ + if (gpu_read(gpu, REG_A6XX_RBBM_STATUS) & + ~A6XX_RBBM_STATUS_CP_AHB_BUSY_CX_MASTER) + return false; + + return !(gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS) & + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT); +} + +bool a6xx_idle(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + /* wait for CP to drain ringbuffer: */ + if (!adreno_idle(gpu, ring)) + return false; + + if (spin_until(_a6xx_check_idle(gpu))) { + DRM_ERROR("%s: %ps: timeout waiting for GPU to idle: status %8.8X irq %8.8X rptr/wptr %d/%d\n", + gpu->name, __builtin_return_address(0), + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR)); + return false; + } + + return true; +} + +static void a6xx_flush(struct msm_gpu *gpu, struct msm_ringbuffer *ring) +{ + uint32_t wptr; + unsigned long flags; + + spin_lock_irqsave(&ring->lock, flags); + + /* Copy the shadow to the actual register */ + ring->cur = ring->next; + + /* Make sure to wrap wptr if we need to */ + wptr = get_wptr(ring); + + spin_unlock_irqrestore(&ring->lock, flags); + + /* Make sure everything is posted before making a decision */ + mb(); + + gpu_write(gpu, REG_A6XX_CP_RB_WPTR, wptr); +} + +static void a6xx_submit(struct msm_gpu *gpu, struct msm_gem_submit *submit, + struct msm_file_private *ctx) +{ + struct msm_drm_private *priv = gpu->dev->dev_private; + struct msm_ringbuffer *ring = submit->ring; + unsigned int i; + + /* Invalidate CCU depth and color */ + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_DEPTH); + + OUT_PKT7(ring, CP_EVENT_WRITE, 1); + OUT_RING(ring, PC_CCU_INVALIDATE_COLOR); + + /* Submit the commands */ + for (i = 0; i < submit->nr_cmds; i++) { + switch (submit->cmd[i].type) { + case MSM_SUBMIT_CMD_IB_TARGET_BUF: + break; + case MSM_SUBMIT_CMD_CTX_RESTORE_BUF: + if (priv->lastctx == ctx) + break; + case MSM_SUBMIT_CMD_BUF: + OUT_PKT7(ring, CP_INDIRECT_BUFFER_PFE, 3); + OUT_RING(ring, lower_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, upper_32_bits(submit->cmd[i].iova)); + OUT_RING(ring, submit->cmd[i].size); + break; + } + } + + /* Write the fence to the scratch register */ + OUT_PKT4(ring, REG_A6XX_CP_SCRATCH_REG(2), 1); + OUT_RING(ring, submit->seqno); + + /* + * Execute a CACHE_FLUSH_TS event. This will ensure that the + * timestamp is written to the memory and then triggers the interrupt + */ + OUT_PKT7(ring, CP_EVENT_WRITE, 4); + OUT_RING(ring, CACHE_FLUSH_TS | (1 << 31)); + OUT_RING(ring, lower_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, upper_32_bits(rbmemptr(ring, fence))); + OUT_RING(ring, submit->seqno); + + a6xx_flush(gpu, ring); +} + +static const struct { + u32 offset; + u32 value; +} a6xx_hwcg[] = { + {REG_A6XX_RBBM_CLOCK_CNTL_SP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_SP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP0, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP1, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP2, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_CNTL2_SP3, 0x02022220}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP0, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP1, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP2, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_DELAY_SP3, 0x00000080}, + {REG_A6XX_RBBM_CLOCK_HYST_SP0, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP1, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP2, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_HYST_SP3, 0x0000f3cf}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP0, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP1, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP2, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_TP3, 0x02222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_TP3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP0, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP1, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP2, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_TP3, 0x00022222}, + {REG_A6XX_RBBM_CLOCK_HYST_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST2_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP0, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP1, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP2, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST3_TP3, 0x77777777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP0, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP1, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP2, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_HYST4_TP3, 0x00077777}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY2_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP0, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP1, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP2, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY3_TP3, 0x11111111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP0, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP1, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP2, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_DELAY4_TP3, 0x00011111}, + {REG_A6XX_RBBM_CLOCK_CNTL_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL3_UCHE, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL4_UCHE, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_HYST_UCHE, 0x00000004}, + {REG_A6XX_RBBM_CLOCK_DELAY_UCHE, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB0, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB1, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB2, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL_RB3, 0x22222222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB0, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB1, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB2, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RB3, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU0, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU1, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU2, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_CNTL_CCU3, 0x00002220}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU0, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU1, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU2, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_HYST_RB_CCU3, 0x00040f00}, + {REG_A6XX_RBBM_CLOCK_CNTL_RAC, 0x05022022}, + {REG_A6XX_RBBM_CLOCK_CNTL2_RAC, 0x00005555}, + {REG_A6XX_RBBM_CLOCK_DELAY_RAC, 0x00000011}, + {REG_A6XX_RBBM_CLOCK_HYST_RAC, 0x00445044}, + {REG_A6XX_RBBM_CLOCK_CNTL_TSE_RAS_RBBM, 0x04222222}, + {REG_A6XX_RBBM_CLOCK_MODE_GPC, 0x00222222}, + {REG_A6XX_RBBM_CLOCK_MODE_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_HYST_TSE_RAS_RBBM, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_HYST_GPC, 0x04104004}, + {REG_A6XX_RBBM_CLOCK_HYST_VFD, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ, 0x00000000}, + {REG_A6XX_RBBM_CLOCK_DELAY_TSE_RAS_RBBM, 0x00004000}, + {REG_A6XX_RBBM_CLOCK_DELAY_GPC, 0x00000200}, + {REG_A6XX_RBBM_CLOCK_DELAY_VFD, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_DELAY_HLSQ_2, 0x00000002}, + {REG_A6XX_RBBM_CLOCK_MODE_HLSQ, 0x00002222}, + {REG_A6XX_RBBM_CLOCK_CNTL_GMU_GX, 0x00000222}, + {REG_A6XX_RBBM_CLOCK_DELAY_GMU_GX, 0x00000111}, + {REG_A6XX_RBBM_CLOCK_HYST_GMU_GX, 0x00000555} +}; + +static void a6xx_set_hwcg(struct msm_gpu *gpu, bool state) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct a6xx_gmu *gmu = &a6xx_gpu->gmu; + unsigned int i; + u32 val; + + val = gpu_read(gpu, REG_A6XX_RBBM_CLOCK_CNTL); + + /* Don't re-program the registers if they are already correct */ + if ((!state && !val) || (state && (val == 0x8aa8aa02))) + return; + + /* Disable SP clock before programming HWCG registers */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 1, 0); + + for (i = 0; i < ARRAY_SIZE(a6xx_hwcg); i++) + gpu_write(gpu, a6xx_hwcg[i].offset, + state ? a6xx_hwcg[i].value : 0); + + /* Enable SP clock */ + gmu_rmw(gmu, REG_A6XX_GPU_GMU_GX_SPTPRAC_CLOCK_CONTROL, 0, 1); + + gpu_write(gpu, REG_A6XX_RBBM_CLOCK_CNTL, state ? 0x8aa8aa02 : 0); +} + +static int a6xx_cp_init(struct msm_gpu *gpu) +{ + struct msm_ringbuffer *ring = gpu->rb[0]; + + OUT_PKT7(ring, CP_ME_INIT, 8); + + OUT_RING(ring, 0x0000002f); + + /* Enable multiple hardware contexts */ + OUT_RING(ring, 0x00000003); + + /* Enable error detection */ + OUT_RING(ring, 0x20000000); + + /* Don't enable header dump */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + /* No workarounds enabled */ + OUT_RING(ring, 0x00000000); + + /* Pad rest of the cmds with 0's */ + OUT_RING(ring, 0x00000000); + OUT_RING(ring, 0x00000000); + + a6xx_flush(gpu, ring); + return a6xx_idle(gpu, ring) ? 0 : -EINVAL; +} + +static int a6xx_ucode_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (!a6xx_gpu->sqe_bo) { + a6xx_gpu->sqe_bo = adreno_fw_create_bo(gpu, + adreno_gpu->fw[ADRENO_FW_SQE], &a6xx_gpu->sqe_iova); + + if (IS_ERR(a6xx_gpu->sqe_bo)) { + int ret = PTR_ERR(a6xx_gpu->sqe_bo); + + a6xx_gpu->sqe_bo = NULL; + DRM_DEV_ERROR(&gpu->pdev->dev, + "Could not allocate SQE ucode: %d\n", ret); + + return ret; + } + } + + gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE_LO, + REG_A6XX_CP_SQE_INSTR_BASE_HI, a6xx_gpu->sqe_iova); + + return 0; +} + +#define A6XX_INT_MASK (A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_CP_HW_ERROR | \ + A6XX_RBBM_INT_0_MASK_CP_IB2 | \ + A6XX_RBBM_INT_0_MASK_CP_IB1 | \ + A6XX_RBBM_INT_0_MASK_CP_RB | \ + A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS | \ + A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW | \ + A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT | \ + A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS | \ + A6XX_RBBM_INT_0_MASK_UCHE_TRAP_INTR) + +static int a6xx_hw_init(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + /* Make sure the GMU keeps the GPU on while we set it up */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_CNTL, 0); + + /* + * Disable the trusted memory range - we don't actually supported secure + * memory rendering at this point in time and we don't want to block off + * part of the virtual memory space. + */ + gpu_write64(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_LO, + REG_A6XX_RBBM_SECVID_TSB_TRUSTED_BASE_HI, 0x00000000); + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TSB_TRUSTED_SIZE, 0x00000000); + + /* enable hardware clockgating */ + a6xx_set_hwcg(gpu, true); + + /* VBIF start */ + gpu_write(gpu, REG_A6XX_VBIF_GATE_OFF_WRREQ_EN, 0x00000009); + gpu_write(gpu, REG_A6XX_RBBM_VBIF_CLIENT_QOS_CNTL, 0x3); + + /* Make all blocks contribute to the GPU BUSY perf counter */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_GPU_BUSY_MASKED, 0xffffffff); + + /* Disable L2 bypass in the UCHE */ + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_LO, 0xffffffc0); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_RANGE_MAX_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_TRAP_BASE_HI, 0x0001ffff); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_LO, 0xfffff000); + gpu_write(gpu, REG_A6XX_UCHE_WRITE_THRU_BASE_HI, 0x0001ffff); + + /* Set the GMEM VA range [0x100000:0x100000 + gpu->gmem - 1] */ + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MIN_LO, + REG_A6XX_UCHE_GMEM_RANGE_MIN_HI, 0x00100000); + + gpu_write64(gpu, REG_A6XX_UCHE_GMEM_RANGE_MAX_LO, + REG_A6XX_UCHE_GMEM_RANGE_MAX_HI, + 0x00100000 + adreno_gpu->gmem - 1); + + gpu_write(gpu, REG_A6XX_UCHE_FILTER_CNTL, 0x804); + gpu_write(gpu, REG_A6XX_UCHE_CACHE_WAYS, 0x4); + + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_2, 0x010000c0); + gpu_write(gpu, REG_A6XX_CP_ROQ_THRESHOLDS_1, 0x8040362c); + + /* Setting the mem pool size */ + gpu_write(gpu, REG_A6XX_CP_MEM_POOL_SIZE, 128); + + /* Setting the primFifo thresholds default values */ + gpu_write(gpu, REG_A6XX_PC_DBG_ECO_CNTL, (0x300 << 11)); + + /* Set the AHB default slave response to "ERROR" */ + gpu_write(gpu, REG_A6XX_CP_AHB_CNTL, 0x1); + + /* Turn on performance counters */ + gpu_write(gpu, REG_A6XX_RBBM_PERFCTR_CNTL, 0x1); + + /* Select CP0 to always count cycles */ + gpu_write(gpu, REG_A6XX_CP_PERFCTR_CP_SEL_0, PERF_CP_ALWAYS_COUNT); + + /* FIXME: not sure if this should live here or in a6xx_gmu.c */ + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GPU_GMU_AO_GPU_CX_BUSY_MASK, + 0xff000000); + gmu_rmw(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_SELECT_0, + 0xff, 0x20); + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_CX_GMU_POWER_COUNTER_ENABLE, + 0x01); + + gpu_write(gpu, REG_A6XX_RB_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_TPL1_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_SP_NC_MODE_CNTL, 2 << 1); + gpu_write(gpu, REG_A6XX_UCHE_MODE_CNTL, 2 << 21); + + /* Enable fault detection */ + gpu_write(gpu, REG_A6XX_RBBM_INTERFACE_HANG_INT_CNTL, + (1 << 30) | 0x1fffff); + + gpu_write(gpu, REG_A6XX_UCHE_CLIENT_PF, 1); + + /* Protect registers from the CP */ + gpu_write(gpu, REG_A6XX_CP_PROTECT_CNTL, 0x00000003); + + gpu_write(gpu, REG_A6XX_CP_PROTECT(0), + A6XX_PROTECT_RDONLY(0x600, 0x51)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(1), A6XX_PROTECT_RW(0xae50, 0x2)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(2), A6XX_PROTECT_RW(0x9624, 0x13)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(3), A6XX_PROTECT_RW(0x8630, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(4), A6XX_PROTECT_RW(0x9e70, 0x1)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(5), A6XX_PROTECT_RW(0x9e78, 0x187)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(6), A6XX_PROTECT_RW(0xf000, 0x810)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(7), + A6XX_PROTECT_RDONLY(0xfc00, 0x3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(8), A6XX_PROTECT_RW(0x50e, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(9), A6XX_PROTECT_RDONLY(0x50f, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(10), A6XX_PROTECT_RW(0x510, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(11), + A6XX_PROTECT_RDONLY(0x0, 0x4f9)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(12), + A6XX_PROTECT_RDONLY(0x501, 0xa)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(13), + A6XX_PROTECT_RDONLY(0x511, 0x44)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(14), A6XX_PROTECT_RW(0xe00, 0xe)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(15), A6XX_PROTECT_RW(0x8e00, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(16), A6XX_PROTECT_RW(0x8e50, 0xf)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(17), A6XX_PROTECT_RW(0xbe02, 0x0)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(18), + A6XX_PROTECT_RW(0xbe20, 0x11f3)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(19), A6XX_PROTECT_RW(0x800, 0x82)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(20), A6XX_PROTECT_RW(0x8a0, 0x8)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(21), A6XX_PROTECT_RW(0x8ab, 0x19)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(22), A6XX_PROTECT_RW(0x900, 0x4d)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(23), A6XX_PROTECT_RW(0x98d, 0x76)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(24), + A6XX_PROTECT_RDONLY(0x8d0, 0x23)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(25), + A6XX_PROTECT_RDONLY(0x980, 0x4)); + gpu_write(gpu, REG_A6XX_CP_PROTECT(26), A6XX_PROTECT_RW(0xa630, 0x0)); + + /* Enable interrupts */ + gpu_write(gpu, REG_A6XX_RBBM_INT_0_MASK, A6XX_INT_MASK); + + ret = adreno_hw_init(gpu); + if (ret) + goto out; + + ret = a6xx_ucode_init(gpu); + if (ret) + goto out; + + /* Always come up on rb 0 */ + a6xx_gpu->cur_ring = gpu->rb[0]; + + /* Enable the SQE_to start the CP engine */ + gpu_write(gpu, REG_A6XX_CP_SQE_CNTL, 1); + + ret = a6xx_cp_init(gpu); + if (ret) + goto out; + + gpu_write(gpu, REG_A6XX_RBBM_SECVID_TRUST_CNTL, 0x0); + +out: + /* + * Tell the GMU that we are done touching the GPU and it can start power + * management + */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + /* Take the GMU out of its special boot mode */ + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_BOOT_SLUMBER); + + return ret; +} + +static void a6xx_dump(struct msm_gpu *gpu) +{ + dev_info(&gpu->pdev->dev, "status: %08x\n", + gpu_read(gpu, REG_A6XX_RBBM_STATUS)); + adreno_dump(gpu); +} + +#define VBIF_RESET_ACK_TIMEOUT 100 +#define VBIF_RESET_ACK_MASK 0x00f0 + +static void a6xx_recover(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int i; + + adreno_dump_info(gpu); + + for (i = 0; i < 8; i++) + dev_info(&gpu->pdev->dev, "CP_SCRATCH_REG%d: %u\n", i, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(i))); + + if (hang_debug) + a6xx_dump(gpu); + + /* + * Turn off keep alive that might have been enabled by the hang + * interrupt + */ + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_GMU_PWR_COL_KEEPALIVE, 0); + + gpu->funcs->pm_suspend(gpu); + gpu->funcs->pm_resume(gpu); + + msm_gpu_hw_init(gpu); +} + +static int a6xx_fault_handler(void *arg, unsigned long iova, int flags) +{ + struct msm_gpu *gpu = arg; + + pr_warn_ratelimited("*** gpu fault: iova=%08lx, flags=%d (%u,%u,%u,%u)\n", + iova, flags, + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(4)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(5)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(6)), + gpu_read(gpu, REG_A6XX_CP_SCRATCH_REG(7))); + + return -EFAULT; +} + +static void a6xx_cp_hw_err_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_CP_INTERRUPT_STATUS); + + if (status & A6XX_CP_INT_CP_OPCODE_ERROR) { + u32 val; + + gpu_write(gpu, REG_A6XX_CP_SQE_STAT_ADDR, 1); + val = gpu_read(gpu, REG_A6XX_CP_SQE_STAT_DATA); + dev_err_ratelimited(&gpu->pdev->dev, + "CP | opcode error | possible opcode=0x%8.8X\n", + val); + } + + if (status & A6XX_CP_INT_CP_UCODE_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, + "CP ucode error interrupt\n"); + + if (status & A6XX_CP_INT_CP_HW_FAULT_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | HW fault | status=0x%8.8X\n", + gpu_read(gpu, REG_A6XX_CP_HW_FAULT)); + + if (status & A6XX_CP_INT_CP_REGISTER_PROTECTION_ERROR) { + u32 val = gpu_read(gpu, REG_A6XX_CP_PROTECT_STATUS); + + dev_err_ratelimited(&gpu->pdev->dev, + "CP | protected mode error | %s | addr=0x%8.8X | status=0x%8.8X\n", + val & (1 << 20) ? "READ" : "WRITE", + (val & 0x3ffff), val); + } + + if (status & A6XX_CP_INT_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP AHB error interrupt\n"); + + if (status & A6XX_CP_INT_CP_VSD_PARITY_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP VSD decoder parity error\n"); + + if (status & A6XX_CP_INT_CP_ILLEGAL_INSTR_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP illegal instruction error\n"); + +} + +static void a6xx_fault_detect_irq(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + struct drm_device *dev = gpu->dev; + struct msm_drm_private *priv = dev->dev_private; + struct msm_ringbuffer *ring = gpu->funcs->active_ring(gpu); + + /* + * Force the GPU to stay on until after we finish + * collecting information + */ + gmu_write(&a6xx_gpu->gmu, REG_A6XX_GMU_GMU_PWR_COL_KEEPALIVE, 1); + + DRM_DEV_ERROR(&gpu->pdev->dev, + "gpu fault ring %d fence %x status %8.8X rb %4.4x/%4.4x ib1 %16.16llX/%4.4x ib2 %16.16llX/%4.4x\n", + ring ? ring->id : -1, ring ? ring->seqno : 0, + gpu_read(gpu, REG_A6XX_RBBM_STATUS), + gpu_read(gpu, REG_A6XX_CP_RB_RPTR), + gpu_read(gpu, REG_A6XX_CP_RB_WPTR), + gpu_read64(gpu, REG_A6XX_CP_IB1_BASE, REG_A6XX_CP_IB1_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB1_REM_SIZE), + gpu_read64(gpu, REG_A6XX_CP_IB2_BASE, REG_A6XX_CP_IB2_BASE_HI), + gpu_read(gpu, REG_A6XX_CP_IB2_REM_SIZE)); + + /* Turn off the hangcheck timer to keep it from bothering us */ + del_timer(&gpu->hangcheck_timer); + + queue_work(priv->wq, &gpu->recover_work); +} + +static irqreturn_t a6xx_irq(struct msm_gpu *gpu) +{ + u32 status = gpu_read(gpu, REG_A6XX_RBBM_INT_0_STATUS); + + gpu_write(gpu, REG_A6XX_RBBM_INT_CLEAR_CMD, status); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_HANG_DETECT) + a6xx_fault_detect_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_CP_AHB_ERROR) + dev_err_ratelimited(&gpu->pdev->dev, "CP | AHB bus error\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_HW_ERROR) + a6xx_cp_hw_err_irq(gpu); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_ASYNCFIFO_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB ASYNC overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_RBBM_ATB_BUS_OVERFLOW) + dev_err_ratelimited(&gpu->pdev->dev, "RBBM | ATB bus overflow\n"); + + if (status & A6XX_RBBM_INT_0_MASK_UCHE_OOB_ACCESS) + dev_err_ratelimited(&gpu->pdev->dev, "UCHE | Out of bounds access\n"); + + if (status & A6XX_RBBM_INT_0_MASK_CP_CACHE_FLUSH_TS) + msm_gpu_retire(gpu); + + return IRQ_HANDLED; +} + +static const u32 a6xx_register_offsets[REG_ADRENO_REGISTER_MAX] = { + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE, REG_A6XX_CP_RB_BASE), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_BASE_HI, REG_A6XX_CP_RB_BASE_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR, + REG_A6XX_CP_RB_RPTR_ADDR_LO), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR_ADDR_HI, + REG_A6XX_CP_RB_RPTR_ADDR_HI), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_RPTR, REG_A6XX_CP_RB_RPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_WPTR, REG_A6XX_CP_RB_WPTR), + REG_ADRENO_DEFINE(REG_ADRENO_CP_RB_CNTL, REG_A6XX_CP_RB_CNTL), +}; + +static const u32 a6xx_registers[] = { + 0x0000, 0x0002, 0x0010, 0x0010, 0x0012, 0x0012, 0x0018, 0x001b, + 0x001e, 0x0032, 0x0038, 0x003c, 0x0042, 0x0042, 0x0044, 0x0044, + 0x0047, 0x0047, 0x0056, 0x0056, 0x00ad, 0x00ae, 0x00b0, 0x00fb, + 0x0100, 0x011d, 0x0200, 0x020d, 0x0210, 0x0213, 0x0218, 0x023d, + 0x0400, 0x04f9, 0x0500, 0x0500, 0x0505, 0x050b, 0x050e, 0x0511, + 0x0533, 0x0533, 0x0540, 0x0555, 0x0800, 0x0808, 0x0810, 0x0813, + 0x0820, 0x0821, 0x0823, 0x0827, 0x0830, 0x0833, 0x0840, 0x0843, + 0x084f, 0x086f, 0x0880, 0x088a, 0x08a0, 0x08ab, 0x08c0, 0x08c4, + 0x08d0, 0x08dd, 0x08f0, 0x08f3, 0x0900, 0x0903, 0x0908, 0x0911, + 0x0928, 0x093e, 0x0942, 0x094d, 0x0980, 0x0984, 0x098d, 0x0996, + 0x0998, 0x099e, 0x09a0, 0x09a6, 0x09a8, 0x09ae, 0x09b0, 0x09b1, + 0x09c2, 0x09c8, 0x0a00, 0x0a03, 0x0c00, 0x0c04, 0x0c06, 0x0c06, + 0x0c10, 0x0cd9, 0x0e00, 0x0e0e, 0x0e10, 0x0e13, 0x0e17, 0x0e19, + 0x0e1c, 0x0e2b, 0x0e30, 0x0e32, 0x0e38, 0x0e39, 0x8600, 0x8601, + 0x8610, 0x861b, 0x8620, 0x8620, 0x8628, 0x862b, 0x8630, 0x8637, + 0x8e01, 0x8e01, 0x8e04, 0x8e05, 0x8e07, 0x8e08, 0x8e0c, 0x8e0c, + 0x8e10, 0x8e1c, 0x8e20, 0x8e25, 0x8e28, 0x8e28, 0x8e2c, 0x8e2f, + 0x8e3b, 0x8e3e, 0x8e40, 0x8e43, 0x8e50, 0x8e5e, 0x8e70, 0x8e77, + 0x9600, 0x9604, 0x9624, 0x9637, 0x9e00, 0x9e01, 0x9e03, 0x9e0e, + 0x9e11, 0x9e16, 0x9e19, 0x9e19, 0x9e1c, 0x9e1c, 0x9e20, 0x9e23, + 0x9e30, 0x9e31, 0x9e34, 0x9e34, 0x9e70, 0x9e72, 0x9e78, 0x9e79, + 0x9e80, 0x9fff, 0xa600, 0xa601, 0xa603, 0xa603, 0xa60a, 0xa60a, + 0xa610, 0xa617, 0xa630, 0xa630, + ~0 +}; + +static int a6xx_pm_resume(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + int ret; + + ret = a6xx_gmu_resume(a6xx_gpu); + + gpu->needs_hw_init = true; + + return ret; +} + +static int a6xx_pm_suspend(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* + * Make sure the GMU is idle before continuing (because some transitions + * may use VBIF + */ + a6xx_gmu_wait_for_idle(a6xx_gpu); + + /* Clear the VBIF pipe before shutting down */ + /* FIXME: This accesses the GPU - do we need to make sure it is on? */ + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0xf); + spin_until((gpu_read(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL1) & 0xf) == 0xf); + gpu_write(gpu, REG_A6XX_VBIF_XIN_HALT_CTRL0, 0); + + return a6xx_gmu_stop(a6xx_gpu); +} + +static int a6xx_get_timestamp(struct msm_gpu *gpu, uint64_t *value) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + /* Force the GPU power on so we can read this register */ + a6xx_gmu_set_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + + *value = gpu_read64(gpu, REG_A6XX_RBBM_PERFCTR_CP_0_LO, + REG_A6XX_RBBM_PERFCTR_CP_0_HI); + + a6xx_gmu_clear_oob(&a6xx_gpu->gmu, GMU_OOB_GPU_SET); + return 0; +} + +#if defined(CONFIG_DEBUG_FS) || defined(CONFIG_DEV_COREDUMP) +static void a6xx_show(struct msm_gpu *gpu, struct msm_gpu_state *state, + struct drm_printer *p) +{ + adreno_show(gpu, state, p); +} +#endif + +static struct msm_ringbuffer *a6xx_active_ring(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + return a6xx_gpu->cur_ring; +} + +static void a6xx_destroy(struct msm_gpu *gpu) +{ + struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu); + struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu); + + if (a6xx_gpu->sqe_bo) { + if (a6xx_gpu->sqe_iova) + msm_gem_put_iova(a6xx_gpu->sqe_bo, gpu->aspace); + drm_gem_object_unreference_unlocked(a6xx_gpu->sqe_bo); + } + + a6xx_gmu_remove(a6xx_gpu); + + adreno_gpu_cleanup(adreno_gpu); + kfree(a6xx_gpu); +} + +static const struct adreno_gpu_funcs funcs = { + .base = { + .get_param = adreno_get_param, + .hw_init = a6xx_hw_init, + .pm_suspend = a6xx_pm_suspend, + .pm_resume = a6xx_pm_resume, + .recover = a6xx_recover, + .submit = a6xx_submit, + .flush = a6xx_flush, + .active_ring = a6xx_active_ring, + .irq = a6xx_irq, + .destroy = a6xx_destroy, +#if defined(CONFIG_DEBUG_FS) || defined(CONFIG_DEV_COREDUMP) + .show = a6xx_show, +#endif + }, + .get_timestamp = a6xx_get_timestamp, +}; + +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev) +{ + struct msm_drm_private *priv = dev->dev_private; + struct platform_device *pdev = priv->gpu_pdev; + struct device_node *node; + struct a6xx_gpu *a6xx_gpu; + struct adreno_gpu *adreno_gpu; + struct msm_gpu *gpu; + int ret; + + a6xx_gpu = kzalloc(sizeof(*a6xx_gpu), GFP_KERNEL); + if (!a6xx_gpu) + return ERR_PTR(-ENOMEM); + + adreno_gpu = &a6xx_gpu->base; + gpu = &adreno_gpu->base; + + adreno_gpu->registers = a6xx_registers; + adreno_gpu->reg_offsets = a6xx_register_offsets; + + ret = adreno_gpu_init(dev, pdev, adreno_gpu, &funcs, 1); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + /* Check if there is a GMU phandle and set it up */ + node = of_parse_phandle(pdev->dev.of_node, "gmu", 0); + + /* FIXME: How do we gracefully handle this? */ + BUG_ON(!node); + + ret = a6xx_gmu_probe(a6xx_gpu, node); + if (ret) { + a6xx_destroy(&(a6xx_gpu->base.base)); + return ERR_PTR(ret); + } + + if (gpu->aspace) + msm_mmu_set_fault_handler(gpu->aspace->mmu, gpu, + a6xx_fault_handler); + + return gpu; +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.h b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h new file mode 100644 index 000000000000..dd69e5b0e692 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.h @@ -0,0 +1,60 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef __A6XX_GPU_H__ +#define __A6XX_GPU_H__ + + +#include "adreno_gpu.h" +#include "a6xx.xml.h" + +#include "a6xx_gmu.h" + +extern bool hang_debug; + +struct a6xx_gpu { + struct adreno_gpu base; + + struct drm_gem_object *sqe_bo; + uint64_t sqe_iova; + + struct msm_ringbuffer *cur_ring; + + struct a6xx_gmu gmu; +}; + +#define to_a6xx_gpu(x) container_of(x, struct a6xx_gpu, base) + +/* + * Given a register and a count, return a value to program into + * REG_CP_PROTECT_REG(n) - this will block both reads and writes for _len + * registers starting at _reg. + */ +#define A6XX_PROTECT_RW(_reg, _len) \ + ((1 << 31) | \ + (((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + +/* + * Same as above, but allow reads over the range. For areas of mixed use (such + * as performance counters) this allows us to protect a much larger range with a + * single register + */ +#define A6XX_PROTECT_RDONLY(_reg, _len) \ + ((((_len) & 0x3FFF) << 18) | ((_reg) & 0x3FFFF)) + + +int a6xx_gmu_resume(struct a6xx_gpu *gpu); +int a6xx_gmu_stop(struct a6xx_gpu *gpu); + +int a6xx_gmu_wait_for_idle(struct a6xx_gpu *gpu); + +int a6xx_gmu_reset(struct a6xx_gpu *a6xx_gpu); +bool a6xx_gmu_isidle(struct a6xx_gmu *gmu); + +int a6xx_gmu_set_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); +void a6xx_gmu_clear_oob(struct a6xx_gmu *gmu, enum a6xx_gmu_oob_state state); + +int a6xx_gmu_probe(struct a6xx_gpu *a6xx_gpu, struct device_node *node); +void a6xx_gmu_remove(struct a6xx_gpu *a6xx_gpu); + +#endif /* __A6XX_GPU_H__ */ diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.c b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c new file mode 100644 index 000000000000..f19ef4cb6ea4 --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.c @@ -0,0 +1,435 @@ +// SPDX-License-Identifier: GPL-2.0 +/* Copyright (c) 2017-2018 The Linux Foundation. All rights reserved. */ + +#include <linux/completion.h> +#include <linux/circ_buf.h> +#include <linux/list.h> + +#include "a6xx_gmu.h" +#include "a6xx_gmu.xml.h" + +#define HFI_MSG_ID(val) [val] = #val + +static const char * const a6xx_hfi_msg_id[] = { + HFI_MSG_ID(HFI_H2F_MSG_INIT), + HFI_MSG_ID(HFI_H2F_MSG_FW_VERSION), + HFI_MSG_ID(HFI_H2F_MSG_BW_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_PERF_TABLE), + HFI_MSG_ID(HFI_H2F_MSG_TEST), +}; + +static int a6xx_hfi_queue_read(struct a6xx_hfi_queue *queue, u32 *data, + u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, hdr, index = header->read_index; + + if (header->read_index == header->write_index) { + header->rx_request = 1; + return 0; + } + + hdr = queue->data[index]; + + /* + * If we are to assume that the GMU firmware is in fact a rational actor + * and is programmed to not send us a larger response than we expect + * then we can also assume that if the header size is unexpectedly large + * that it is due to memory corruption and/or hardware failure. In this + * case the only reasonable course of action is to BUG() to help harden + * the failure. + */ + + BUG_ON(HFI_HEADER_SIZE(hdr) > dwords); + + for (i = 0; i < HFI_HEADER_SIZE(hdr); i++) { + data[i] = queue->data[index]; + index = (index + 1) % header->size; + } + + header->read_index = index; + return HFI_HEADER_SIZE(hdr); +} + +static int a6xx_hfi_queue_write(struct a6xx_gmu *gmu, + struct a6xx_hfi_queue *queue, u32 *data, u32 dwords) +{ + struct a6xx_hfi_queue_header *header = queue->header; + u32 i, space, index = header->write_index; + + spin_lock(&queue->lock); + + space = CIRC_SPACE(header->write_index, header->read_index, + header->size); + if (space < dwords) { + header->dropped++; + spin_unlock(&queue->lock); + return -ENOSPC; + } + + for (i = 0; i < dwords; i++) { + queue->data[index] = data[i]; + index = (index + 1) % header->size; + } + + header->write_index = index; + spin_unlock(&queue->lock); + + gmu_write(gmu, REG_A6XX_GMU_HOST2GMU_INTR_SET, 0x01); + return 0; +} + +struct a6xx_hfi_response { + u32 id; + u32 seqnum; + struct list_head node; + struct completion complete; + + u32 error; + u32 payload[16]; +}; + +/* + * Incoming HFI ack messages can come in out of order so we need to store all + * the pending messages on a list until they are handled. + */ +static spinlock_t hfi_ack_lock = __SPIN_LOCK_UNLOCKED(message_lock); +static LIST_HEAD(hfi_ack_list); + +static void a6xx_hfi_handle_ack(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_response *resp; + u32 id, seqnum; + + /* msg->ret_header contains the header of the message being acked */ + id = HFI_HEADER_ID(msg->ret_header); + seqnum = HFI_HEADER_SEQNUM(msg->ret_header); + + spin_lock(&hfi_ack_lock); + list_for_each_entry(resp, &hfi_ack_list, node) { + if (resp->id == id && resp->seqnum == seqnum) { + resp->error = msg->error; + memcpy(resp->payload, msg->payload, + sizeof(resp->payload)); + + complete(&resp->complete); + spin_unlock(&hfi_ack_lock); + return; + } + } + spin_unlock(&hfi_ack_lock); + + dev_err(gmu->dev, "Nobody was waiting for HFI message %d\n", seqnum); +} + +static void a6xx_hfi_handle_error(struct a6xx_gmu *gmu, + struct a6xx_hfi_msg_response *msg) +{ + struct a6xx_hfi_msg_error *error = (struct a6xx_hfi_msg_error *) msg; + + dev_err(gmu->dev, "GMU firmware error %d\n", error->code); +} + +void a6xx_hfi_task(unsigned long data) +{ + struct a6xx_gmu *gmu = (struct a6xx_gmu *) data; + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_RESPONSE_QUEUE]; + struct a6xx_hfi_msg_response resp; + + for (;;) { + u32 id; + int ret = a6xx_hfi_queue_read(queue, (u32 *) &resp, + sizeof(resp) >> 2); + + /* Returns the number of bytes copied or negative on error */ + if (ret <= 0) { + if (ret < 0) + dev_err(gmu->dev, + "Unable to read the HFI message queue\n"); + break; + } + + id = HFI_HEADER_ID(resp.header); + + if (id == HFI_F2H_MSG_ACK) + a6xx_hfi_handle_ack(gmu, &resp); + else if (id == HFI_F2H_MSG_ERROR) + a6xx_hfi_handle_error(gmu, &resp); + } +} + +static int a6xx_hfi_send_msg(struct a6xx_gmu *gmu, int id, + void *data, u32 size, u32 *payload, u32 payload_size) +{ + struct a6xx_hfi_queue *queue = &gmu->queues[HFI_COMMAND_QUEUE]; + struct a6xx_hfi_response resp = { 0 }; + int ret, dwords = size >> 2; + u32 seqnum; + + seqnum = atomic_inc_return(&queue->seqnum) % 0xfff; + + /* First dword of the message is the message header - fill it in */ + *((u32 *) data) = (seqnum << 20) | (HFI_MSG_CMD << 16) | + (dwords << 8) | id; + + init_completion(&resp.complete); + resp.id = id; + resp.seqnum = seqnum; + + spin_lock_bh(&hfi_ack_lock); + list_add_tail(&resp.node, &hfi_ack_list); + spin_unlock_bh(&hfi_ack_lock); + + ret = a6xx_hfi_queue_write(gmu, queue, data, dwords); + if (ret) { + dev_err(gmu->dev, "Unable to send message %s id %d\n", + a6xx_hfi_msg_id[id], seqnum); + goto out; + } + + /* Wait up to 5 seconds for the response */ + ret = wait_for_completion_timeout(&resp.complete, + msecs_to_jiffies(5000)); + if (!ret) { + dev_err(gmu->dev, + "Message %s id %d timed out waiting for response\n", + a6xx_hfi_msg_id[id], seqnum); + ret = -ETIMEDOUT; + } else + ret = 0; + +out: + spin_lock_bh(&hfi_ack_lock); + list_del(&resp.node); + spin_unlock_bh(&hfi_ack_lock); + + if (ret) + return ret; + + if (resp.error) { + dev_err(gmu->dev, "Message %s id %d returned error %d\n", + a6xx_hfi_msg_id[id], seqnum, resp.error); + return -EINVAL; + } + + if (payload && payload_size) { + int copy = min_t(u32, payload_size, sizeof(resp.payload)); + + memcpy(payload, resp.payload, copy); + } + + return 0; +} + +static int a6xx_hfi_send_gmu_init(struct a6xx_gmu *gmu, int boot_state) +{ + struct a6xx_hfi_msg_gmu_init_cmd msg = { 0 }; + + msg.dbg_buffer_addr = (u32) gmu->debug->iova; + msg.dbg_buffer_size = (u32) gmu->debug->size; + msg.boot_state = boot_state; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_INIT, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_get_fw_version(struct a6xx_gmu *gmu, u32 *version) +{ + struct a6xx_hfi_msg_fw_version msg = { 0 }; + + /* Currently supporting version 1.1 */ + msg.supported_version = (1 << 28) | (1 << 16); + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_FW_VERSION, &msg, sizeof(msg), + version, sizeof(*version)); +} + +static int a6xx_hfi_send_perf_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_perf_table msg = { 0 }; + int i; + + msg.num_gpu_levels = gmu->nr_gpu_freqs; + msg.num_gmu_levels = gmu->nr_gmu_freqs; + + for (i = 0; i < gmu->nr_gpu_freqs; i++) { + msg.gx_votes[i].vote = gmu->gx_arc_votes[i]; + msg.gx_votes[i].freq = gmu->gpu_freqs[i] / 1000; + } + + for (i = 0; i < gmu->nr_gmu_freqs; i++) { + msg.cx_votes[i].vote = gmu->cx_arc_votes[i]; + msg.cx_votes[i].freq = gmu->gmu_freqs[i] / 1000; + } + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_PERF_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_bw_table(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_bw_table msg = { 0 }; + + /* + * The sdm845 GMU doesn't do bus frequency scaling on its own but it + * does need at least one entry in the list because it might be accessed + * when the GMU is shutting down. Send a single "off" entry. + */ + + msg.bw_level_num = 1; + + msg.ddr_cmds_num = 3; + msg.ddr_wait_bitmask = 0x07; + + msg.ddr_cmds_addrs[0] = 0x50000; + msg.ddr_cmds_addrs[1] = 0x5005c; + msg.ddr_cmds_addrs[2] = 0x5000c; + + msg.ddr_cmds_data[0][0] = 0x40000000; + msg.ddr_cmds_data[0][1] = 0x40000000; + msg.ddr_cmds_data[0][2] = 0x40000000; + + /* + * These are the CX (CNOC) votes. This is used but the values for the + * sdm845 GMU are known and fixed so we can hard code them. + */ + + msg.cnoc_cmds_num = 3; + msg.cnoc_wait_bitmask = 0x05; + + msg.cnoc_cmds_addrs[0] = 0x50034; + msg.cnoc_cmds_addrs[1] = 0x5007c; + msg.cnoc_cmds_addrs[2] = 0x5004c; + + msg.cnoc_cmds_data[0][0] = 0x40000000; + msg.cnoc_cmds_data[0][1] = 0x00000000; + msg.cnoc_cmds_data[0][2] = 0x40000000; + + msg.cnoc_cmds_data[1][0] = 0x60000001; + msg.cnoc_cmds_data[1][1] = 0x20000001; + msg.cnoc_cmds_data[1][2] = 0x60000001; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_BW_TABLE, &msg, sizeof(msg), + NULL, 0); +} + +static int a6xx_hfi_send_test(struct a6xx_gmu *gmu) +{ + struct a6xx_hfi_msg_test msg = { 0 }; + + return a6xx_hfi_send_msg(gmu, HFI_H2F_MSG_TEST, &msg, sizeof(msg), + NULL, 0); +} + +int a6xx_hfi_start(struct a6xx_gmu *gmu, int boot_state) +{ + int ret; + + ret = a6xx_hfi_send_gmu_init(gmu, boot_state); + if (ret) + return ret; + + ret = a6xx_hfi_get_fw_version(gmu, NULL); + if (ret) + return ret; + + /* + * We have to get exchange version numbers per the sequence but at this + * point th kernel driver doesn't need to know the exact version of + * the GMU firmware + */ + + ret = a6xx_hfi_send_perf_table(gmu); + if (ret) + return ret; + + ret = a6xx_hfi_send_bw_table(gmu); + if (ret) + return ret; + + /* + * Let the GMU know that there won't be any more HFI messages until next + * boot + */ + a6xx_hfi_send_test(gmu); + + return 0; +} + +void a6xx_hfi_stop(struct a6xx_gmu *gmu) +{ + int i; + + for (i = 0; i < ARRAY_SIZE(gmu->queues); i++) { + struct a6xx_hfi_queue *queue = &gmu->queues[i]; + + if (!queue->header) + continue; + + if (queue->header->read_index != queue->header->write_index) + dev_err(gmu->dev, "HFI queue %d is not empty\n", i); + + queue->header->read_index = 0; + queue->header->write_index = 0; + } +} + +static void a6xx_hfi_queue_init(struct a6xx_hfi_queue *queue, + struct a6xx_hfi_queue_header *header, void *virt, u64 iova, + u32 id) +{ + spin_lock_init(&queue->lock); + queue->header = header; + queue->data = virt; + atomic_set(&queue->seqnum, 0); + + /* Set up the shared memory header */ + header->iova = iova; + header->type = 10 << 8 | id; + header->status = 1; + header->size = SZ_4K >> 2; + header->msg_size = 0; + header->dropped = 0; + header->rx_watermark = 1; + header->tx_watermark = 1; + header->rx_request = 1; + header->tx_request = 0; + header->read_index = 0; + header->write_index = 0; +} + +void a6xx_hfi_init(struct a6xx_gmu *gmu) +{ + struct a6xx_gmu_bo *hfi = gmu->hfi; + struct a6xx_hfi_queue_table_header *table = hfi->virt; + struct a6xx_hfi_queue_header *headers = hfi->virt + sizeof(*table); + u64 offset; + int table_size; + + /* + * The table size is the size of the table header plus all of the queue + * headers + */ + table_size = sizeof(*table); + table_size += (ARRAY_SIZE(gmu->queues) * + sizeof(struct a6xx_hfi_queue_header)); + + table->version = 0; + table->size = table_size; + /* First queue header is located immediately after the table header */ + table->qhdr0_offset = sizeof(*table) >> 2; + table->qhdr_size = sizeof(struct a6xx_hfi_queue_header) >> 2; + table->num_queues = ARRAY_SIZE(gmu->queues); + table->active_queues = ARRAY_SIZE(gmu->queues); + + /* Command queue */ + offset = SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[0], &headers[0], hfi->virt + offset, + hfi->iova + offset, 0); + + /* GMU response queue */ + offset += SZ_4K; + a6xx_hfi_queue_init(&gmu->queues[1], &headers[1], hfi->virt + offset, + hfi->iova + offset, 4); +} diff --git a/drivers/gpu/drm/msm/adreno/a6xx_hfi.h b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h new file mode 100644 index 000000000000..60d1319fa44f --- /dev/null +++ b/drivers/gpu/drm/msm/adreno/a6xx_hfi.h @@ -0,0 +1,127 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* Copyright (c) 2017 The Linux Foundation. All rights reserved. */ + +#ifndef _A6XX_HFI_H_ +#define _A6XX_HFI_H_ + +struct a6xx_hfi_queue_table_header { + u32 version; + u32 size; /* Size of the queue table in dwords */ + u32 qhdr0_offset; /* Offset of the first queue header */ + u32 qhdr_size; /* Size of the queue headers */ + u32 num_queues; /* Number of total queues */ + u32 active_queues; /* Number of active queues */ +}; + +struct a6xx_hfi_queue_header { + u32 status; + u32 iova; + u32 type; + u32 size; + u32 msg_size; + u32 dropped; + u32 rx_watermark; + u32 tx_watermark; + u32 rx_request; + u32 tx_request; + u32 read_index; + u32 write_index; +}; + +struct a6xx_hfi_queue { + struct a6xx_hfi_queue_header *header; + spinlock_t lock; + u32 *data; + atomic_t seqnum; +}; + +/* This is the outgoing queue to the GMU */ +#define HFI_COMMAND_QUEUE 0 + +/* THis is the incoming response queue from the GMU */ +#define HFI_RESPONSE_QUEUE 1 + +#define HFI_HEADER_ID(msg) ((msg) & 0xff) +#define HFI_HEADER_SIZE(msg) (((msg) >> 8) & 0xff) +#define HFI_HEADER_SEQNUM(msg) (((msg) >> 20) & 0xfff) + +/* FIXME: Do we need this or can we use ARRAY_SIZE? */ +#define HFI_RESPONSE_PAYLOAD_SIZE 16 + +/* HFI message types */ + +#define HFI_MSG_CMD 0 +#define HFI_MSG_ACK 2 + +#define HFI_F2H_MSG_ACK 126 + +struct a6xx_hfi_msg_response { + u32 header; + u32 ret_header; + u32 error; + u32 payload[HFI_RESPONSE_PAYLOAD_SIZE]; +}; + +#define HFI_F2H_MSG_ERROR 100 + +struct a6xx_hfi_msg_error { + u32 header; + u32 code; + u32 payload[2]; +}; + +#define HFI_H2F_MSG_INIT 0 + +struct a6xx_hfi_msg_gmu_init_cmd { + u32 header; + u32 seg_id; + u32 dbg_buffer_addr; + u32 dbg_buffer_size; + u32 boot_state; +}; + +#define HFI_H2F_MSG_FW_VERSION 1 + +struct a6xx_hfi_msg_fw_version { + u32 header; + u32 supported_version; +}; + +#define HFI_H2F_MSG_PERF_TABLE 4 + +struct perf_level { + u32 vote; + u32 freq; +}; + +struct a6xx_hfi_msg_perf_table { + u32 header; + u32 num_gpu_levels; + u32 num_gmu_levels; + + struct perf_level gx_votes[16]; + struct perf_level cx_votes[4]; +}; + +#define HFI_H2F_MSG_BW_TABLE 3 + +struct a6xx_hfi_msg_bw_table { + u32 header; + u32 bw_level_num; + u32 cnoc_cmds_num; + u32 ddr_cmds_num; + u32 cnoc_wait_bitmask; + u32 ddr_wait_bitmask; + u32 cnoc_cmds_addrs[6]; + u32 cnoc_cmds_data[2][6]; + u32 ddr_cmds_addrs[8]; + u32 ddr_cmds_data[16][8]; +}; + +#define HFI_H2F_MSG_TEST 5 + +struct a6xx_hfi_msg_test { + u32 header; +}; + +#endif diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c index 37746f1d54cf..7d3e9a129ac7 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_device.c +++ b/drivers/gpu/drm/msm/adreno/adreno_device.c @@ -111,6 +111,16 @@ static const struct adreno_info gpulist[] = { ADRENO_QUIRK_FAULT_DETECT_MASK, .init = a5xx_gpu_init, .zapfw = "a530_zap.mdt", + }, { + .rev = ADRENO_REV(6, 3, 0, ANY_ID), + .revn = 630, + .name = "A630", + .fw = { + [ADRENO_FW_SQE] = "a630_sqe.fw", + [ADRENO_FW_GMU] = "a630_gmu.bin", + }, + .gmem = SZ_1M, + .init = a6xx_gpu_init, }, }; @@ -127,6 +137,8 @@ MODULE_FIRMWARE("qcom/a530_zap.mdt"); MODULE_FIRMWARE("qcom/a530_zap.b00"); MODULE_FIRMWARE("qcom/a530_zap.b01"); MODULE_FIRMWARE("qcom/a530_zap.b02"); +MODULE_FIRMWARE("qcom/a630_sqe.fw"); +MODULE_FIRMWARE("qcom/a630_gmu.bin"); static inline bool _rev_match(uint8_t entry, uint8_t id) { diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.h b/drivers/gpu/drm/msm/adreno/adreno_gpu.h index d391ff377612..de6e6ee42fba 100644 --- a/drivers/gpu/drm/msm/adreno/adreno_gpu.h +++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.h @@ -50,7 +50,9 @@ enum adreno_regs { enum { ADRENO_FW_PM4 = 0, + ADRENO_FW_SQE = 0, /* a6xx */ ADRENO_FW_PFP = 1, + ADRENO_FW_GMU = 1, /* a6xx */ ADRENO_FW_GPMU = 2, ADRENO_FW_MAX, }; @@ -335,6 +337,7 @@ static inline void adreno_gpu_write(struct adreno_gpu *gpu, struct msm_gpu *a3xx_gpu_init(struct drm_device *dev); struct msm_gpu *a4xx_gpu_init(struct drm_device *dev); struct msm_gpu *a5xx_gpu_init(struct drm_device *dev); +struct msm_gpu *a6xx_gpu_init(struct drm_device *dev); static inline void adreno_gpu_write64(struct adreno_gpu *gpu, enum adreno_regs lo, enum adreno_regs hi, u64 data) diff --git a/drivers/gpu/drm/msm/msm_gpu.c b/drivers/gpu/drm/msm/msm_gpu.c index ca368490b3ee..5e808cfec345 100644 --- a/drivers/gpu/drm/msm/msm_gpu.c +++ b/drivers/gpu/drm/msm/msm_gpu.c @@ -88,7 +88,7 @@ static struct devfreq_dev_profile msm_devfreq_profile = { static void msm_devfreq_init(struct msm_gpu *gpu) { /* We need target support to do devfreq */ - if (!gpu->funcs->gpu_busy) + if (!gpu->funcs->gpu_busy || !gpu->core_clk) return; msm_devfreq_profile.initial_freq = gpu->fast_rate; -- 2.18.0 _______________________________________________ Freedreno mailing list Freedreno@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/freedreno ^ permalink raw reply related [flat|nested] 12+ messages in thread
end of thread, other threads:[~2018-08-06 17:33 UTC | newest]
Thread overview: 12+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2018-06-11 18:25 [PATCH 0/5] drm/msm: Add support for Adreno a6xx Jordan Crouse
[not found] ` <20180611182604.30467-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
2018-06-11 18:26 ` [PATCH 1/5] drm/msm: Remove pm_runtime operations from msm_iommu Jordan Crouse
[not found] ` <20180611182604.30467-2-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
2018-06-12 16:55 ` Kristian Høgsberg
2018-06-14 7:00 ` Vivek Gautam
2018-06-14 16:09 ` Jordan Crouse
2018-06-11 18:26 ` [PATCH 2/5] drm/msm: Add a helper function to parse clock names Jordan Crouse
2018-06-11 18:26 ` [PATCH 3/5] drm/msm/adreno: Load the firmware before bringing up the hardware Jordan Crouse
[not found] ` <20180611182604.30467-4-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
2018-06-11 21:50 ` kbuild test robot
2018-06-11 18:26 ` [PATCH 4/5] drm/msm: Add generated headers for A6XX Jordan Crouse
2018-06-11 18:26 ` [PATCH 5/5] drm/msm: Add A6XX device support Jordan Crouse
-- strict thread matches above, loose matches on Subject: below --
2018-06-19 22:30 [v6 PATCH 0/5] drm/msm: Add support for Adreno a6xx Jordan Crouse
[not found] ` <20180619223006.31519-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
2018-06-19 22:30 ` [PATCH 5/5] drm/msm: Add A6XX device support Jordan Crouse
2018-08-06 17:33 [v7 PATCH 0/5] Add support for Adreno a6xx Jordan Crouse
[not found] ` <20180806173324.16074-1-jcrouse-sgV2jX0FEOL9JmXXK+q4OQ@public.gmane.org>
2018-08-06 17:33 ` [PATCH 5/5] drm/msm: Add A6XX device support Jordan Crouse
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).