Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Kever Yang]

Hi Christoph,

     The ARM PL330 DMA driver in kernel only relate to:

- DTS kernel used, can be check in /proc/device-tree/

- kernel driver which should mach the compatible name.

This driver should has nothing to do with U-Boot SPL or TF-A, because we 
don't have any special setting for PL330 in loader stage.


Thanks,

- Kever

On 2023/4/3 03:01, Christoph Fritz wrote:
> Hello Kever,
>
>   on a rk3399, booting current U-Boot SPL with mainline TF-A leads to
> missing DMA (and no sound) on Linux.
>
> However, when using rockchip its so called mini-loader
> (rk3399_miniloader_v1.26.bin) and their BL31 (rk3399_bl31_v1.35.elf) to
> boot, DMA works perfectly fine.
>
> Tested on a custom rk3399 board and on ROCK Pi4.
>
> Attached to this mail are two boot logs with some debug prints:
>
>   - good_amba_log.txt (DMA works)
>   - no_amba_log.txt (no DMA device)
>
> The main difference I can spot between the two logs is that on Linux
> (drivers/amba/bus.c) AMBA_CID (0xb105f00d) cannot be found. Instead,
> only some CORESIGHT_CIDs (presumably for debugging) and four 0x00000000
> CIDs are detected.
>
> As a result, the "PL330 DMAC-241330" driver does not load. My theory is
> that DMA needs to be allowed somewhere in the undocumented syscon-
> registers, similar to what U-Boot is already doing for eMMC in
> arch_cpu_init() (arch/arm/mach-rockchip/rk3399/rk3399.c).
>
> Any ideas?
>
> Or maybe I'm just missing some configuration?
>
> Since multiple software projects are involved (TF-A, OP-TEE, U-Boot,
> Linux), I Cc'ed a bit.
>
>   Thanks
>    -- Christoph

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Christoph Fritz]

> 
>      The ARM PL330 DMA driver in kernel only relate to:
> 
> - DTS kernel used, can be check in /proc/device-tree/
> 
> - kernel driver which should mach the compatible name.

drivers/dma/pl330.c needs also a successfully matched amba, but this
fails when using mainline TF-A and U-Boot SPL.

I'm using the same kernel and devicetree on both tests, the only thing
changed is TF-A and U-Boot SPL   vs   mini-loader and rk3399_bl31.

> This driver should has nothing to do with U-Boot SPL or TF-A, because we 
> don't have any special setting for PL330 in loader stage.

It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the
ARM bus. The DMA driver not loading is just a symptom of this issue.

Any ideas?

Thanks
  -- Christoph


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Russell King (Oracle)]

On Mon, Apr 03, 2023 at 12:11:40PM +0200, Christoph Fritz wrote:
> > 
> >      The ARM PL330 DMA driver in kernel only relate to:
> > 
> > - DTS kernel used, can be check in /proc/device-tree/
> > 
> > - kernel driver which should mach the compatible name.
> 
> drivers/dma/pl330.c needs also a successfully matched amba, but this
> fails when using mainline TF-A and U-Boot SPL.
> 
> I'm using the same kernel and devicetree on both tests, the only thing
> changed is TF-A and U-Boot SPL   vs   mini-loader and rk3399_bl31.
> 
> > This driver should has nothing to do with U-Boot SPL or TF-A, because we 
> > don't have any special setting for PL330 in loader stage.
> 
> It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the
> ARM bus. The DMA driver not loading is just a symptom of this issue.

That brings up the obvious question: why is it unable to find the AMBA
CID? Is that because some resource needed to read it hasn't been
enabled yet? If the AMBA CID is not accessible, presumably the rest of
the PL330 also isn't accessible, so even if we could bind the driver,
it still wouldn't work?

-- 
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTP is here! 80Mbps down 10Mbps up. Decent connectivity at last!

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Christoph Fritz]

On Mon, 2023-04-03 at 14:17 +0100, Russell King (Oracle) wrote:
> On Mon, Apr 03, 2023 at 12:11:40PM +0200, Christoph Fritz wrote:
> > > 
> > >      The ARM PL330 DMA driver in kernel only relate to:
> > > 
> > > - DTS kernel used, can be check in /proc/device-tree/
> > > 
> > > - kernel driver which should mach the compatible name.
> > 
> > drivers/dma/pl330.c needs also a successfully matched amba, but this
> > fails when using mainline TF-A and U-Boot SPL.
> > 
> > I'm using the same kernel and devicetree on both tests, the only thing
> > changed is TF-A and U-Boot SPL   vs   mini-loader and rk3399_bl31.
> > 
> > > This driver should has nothing to do with U-Boot SPL or TF-A, because we 
> > > don't have any special setting for PL330 in loader stage.
> > 
> > It is drivers/amba/bus.c, which is unable to find an AMBA_CID on the
> > ARM bus. The DMA driver not loading is just a symptom of this issue.
> 
> That brings up the obvious question: why is it unable to find the AMBA
> CID? Is that because some resource needed to read it hasn't been
> enabled yet?

That's my guess too, and that's why I'm asking Kever from Rockchip. To
me, it seems that their closed-source BL31 and/or mini-loader is
enabling something that is missing in upstream TF-A and/or U-Boot SPL.
On top of that, the registers responsible for that are undocumented.

But I could be wrong, and it's possible that I'm missing something
else. Maybe dumping the syscon-registers and comparing them might help,
but as far as I can see, it's a bit of an effort because they require
special treatment to be accessed.

>  If the AMBA CID is not accessible, presumably the rest of
> the PL330 also isn't accessible, so even if we could bind the driver,
> it still wouldn't work?

The PL330 driver attempted to probe, but was deferred due to a missing
AMBA_CID. It's in the bootlog that I attached in my first email in this
thread.

Thanks
  -- Christoph



_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Jonathan Liu]

Hi Kever,

I also see this issue when switching between Rockchip ATF and Upstream ATF.

Versions:
Rockchip DDR Blob - rk3399_ddr_800MHz_v1.30.bin
Rockchip Miniloader - rk3399_miniloader_v1.30.bin
Rockchip ATF - rk3399_bl31_v1.36.elf
Upstream ATF - git://git.trustedfirmware.org/TF-A/trusted-firmware-a.git,
git tag v2.8.0, with RK3399_BAUDRATE changed from 115200 to 1500000 in
plat/rockchip/rk3399/rk3399_def.h
U-Boot - git://git.denx.de/u-boot.git, git tag v2022.01

Results:
Rockchip DDR Blob + Rockchip Miniloader + Rockchip ATF + U-Boot = DMA working
  dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330
  dma-pl330 ff6d0000.dma-controller:       DBUFF-32x8bytes Num_Chans-6
Num_Peri-12 Num_Events-12
  dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330
  dma-pl330 ff6e0000.dma-controller:       DBUFF-128x8bytes
Num_Chans-8 Num_Peri-20 Num_Events-16
Rockchip DDR Blob + Rockchip Miniloader + Upstream ATF + U-Boot = DMA
not working
  OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6d0000
  OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6e0000

I can't check the Rockchip ATF source code as it isn't available.
Any idea what is different between Rockchip ATF and Upstream ATF for
DMA to work properly?

Regards,
Jonathan

On Mon, 3 Apr 2023 at 18:38, Kever Yang <kever.yang@rock-chips.com> wrote:
>
> Hi Christoph,
>
>      The ARM PL330 DMA driver in kernel only relate to:
>
> - DTS kernel used, can be check in /proc/device-tree/
>
> - kernel driver which should mach the compatible name.
>
> This driver should has nothing to do with U-Boot SPL or TF-A, because we
> don't have any special setting for PL330 in loader stage.
>
>
> Thanks,
>
> - Kever
>
> On 2023/4/3 03:01, Christoph Fritz wrote:
> > Hello Kever,
> >
> >   on a rk3399, booting current U-Boot SPL with mainline TF-A leads to
> > missing DMA (and no sound) on Linux.
> >
> > However, when using rockchip its so called mini-loader
> > (rk3399_miniloader_v1.26.bin) and their BL31 (rk3399_bl31_v1.35.elf) to
> > boot, DMA works perfectly fine.
> >
> > Tested on a custom rk3399 board and on ROCK Pi4.
> >
> > Attached to this mail are two boot logs with some debug prints:
> >
> >   - good_amba_log.txt (DMA works)
> >   - no_amba_log.txt (no DMA device)
> >
> > The main difference I can spot between the two logs is that on Linux
> > (drivers/amba/bus.c) AMBA_CID (0xb105f00d) cannot be found. Instead,
> > only some CORESIGHT_CIDs (presumably for debugging) and four 0x00000000
> > CIDs are detected.
> >
> > As a result, the "PL330 DMAC-241330" driver does not load. My theory is
> > that DMA needs to be allowed somewhere in the undocumented syscon-
> > registers, similar to what U-Boot is already doing for eMMC in
> > arch_cpu_init() (arch/arm/mach-rockchip/rk3399/rk3399.c).
> >
> > Any ideas?
> >
> > Or maybe I'm just missing some configuration?
> >
> > Since multiple software projects are involved (TF-A, OP-TEE, U-Boot,
> > Linux), I Cc'ed a bit.
> >
> >   Thanks
> >    -- Christoph

_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel

Re: rk3399 issue: no DMA in Linux with mainline TF-A and U-Boot SPL

[Christoph Fritz]

Hello Jonathan and Kevar,

> I also see this issue when switching between Rockchip ATF and Upstream ATF.
> 
> Versions:
> Rockchip DDR Blob - rk3399_ddr_800MHz_v1.30.bin
> Rockchip Miniloader - rk3399_miniloader_v1.30.bin
> Rockchip ATF - rk3399_bl31_v1.36.elf
> Upstream ATF - git://git.trustedfirmware.org/TF-A/trusted-firmware-a.git,
> git tag v2.8.0, with RK3399_BAUDRATE changed from 115200 to 1500000 in
> plat/rockchip/rk3399/rk3399_def.h
> U-Boot - git://git.denx.de/u-boot.git, git tag v2022.01
> 
> Results:
> Rockchip DDR Blob + Rockchip Miniloader + Rockchip ATF + U-Boot = DMA working
>   dma-pl330 ff6d0000.dma-controller: Loaded driver for PL330 DMAC-241330
>   dma-pl330 ff6d0000.dma-controller:       DBUFF-32x8bytes Num_Chans-6
> Num_Peri-12 Num_Events-12
>   dma-pl330 ff6e0000.dma-controller: Loaded driver for PL330 DMAC-241330
>   dma-pl330 ff6e0000.dma-controller:       DBUFF-128x8bytes
> Num_Chans-8 Num_Peri-20 Num_Events-16
> Rockchip DDR Blob + Rockchip Miniloader + Upstream ATF + U-Boot = DMA
> not working
>   OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6d0000
>   OF: amba_device_add() failed (-19) for /bus/dma-controller@ff6e0000
> 
> I can't check the Rockchip ATF source code as it isn't available.
> Any idea what is different between Rockchip ATF and Upstream ATF for
> DMA to work properly?

@Kevar: It would be really great if you could have a look into it.

I am still having this issue.

Thanks
  -- Christoph


_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel