bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

I've been testing the boostd for a few Marvell boards and seeing this
error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
"bootflow scan scsi" command triggered the "CACHE: Misaligned
operation at range" error. However, this error did not affect the
result of the scan, i.e. the bootflow for scsi partition was created
correctly, and u-boot is running normally.

Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
Perhaps this is a case where the DCACHE is not required and should be
turned off?

Please see the log after the break below.

All the best,
Tony

========

U-Boot 2023.10-rc4-tld-1-00036-gbb16283b81-dirty (Sep 11 2023 - 11:56:18 -0700)
Thecus N2350

SoC:   MV88F6820-A0 at 1066 MHz
DRAM:  1 GiB (533 MHz, 32-bit, ECC not enabled)
Core:  65 devices, 23 uclasses, devicetree: separate
NAND:  512 MiB
MMC:
Loading Environment from SPIFlash... SF: Detected mx25l3205d with page
size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

Model: Thecus N2350
Net:
Warning: ethernet@70000 (eth0) using random MAC address - be:13:ae:99:49:ab
eth0: ethernet@70000
Hit any key to stop autoboot:  0

N2350 > env def -a
## Resetting to default environment

N2350 > bootdev l
Seq  Probed  Status  Uclass    Name
---  ------  ------  --------  ------------------
  0   [   ]      OK  ethernet  ethernet@70000.bootdev
---  ------  ------  --------  ------------------
(1 bootdev)

N2350 > bootdev hunt scsi
Hunting with: scsi
pcie0.0: Link down
pcie1.0: Link down
scanning bus for devices...
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq led only pmp fbss pio slum part sxs
  Device 0: (1:0) Vendor: ATA Prod.: ST750LX003-1AC15 Rev: SM12
            Type: Hard Disk
            Capacity: 715404.8 MB = 698.6 GB (1465149168 x 512)

N2350 > bootflow scan usb
Bus usb@58000: USB EHCI 1.00
Bus usb3@f0000: MVEBU XHCI INIT controller @ 0xf10f4000
Register 2000120 NbrPorts 2
Starting the controller
USB XHCI 1.00
Bus usb3@f8000: MVEBU XHCI INIT controller @ 0xf10fc000
Register 2000120 NbrPorts 2
Starting the controller
USB XHCI 1.00
scanning bus usb@58000 for devices... 1 USB Device(s) found
scanning bus usb3@f0000 for devices... 1 USB Device(s) found
scanning bus usb3@f8000 for devices... 2 USB Device(s) found
** File not found /boot/boot.bmp **

N2350 > bootflow scan scsi
CACHE: Misaligned operation at range [3fb7fe48, 3fb80248]
CACHE: Misaligned operation at range [3fb7fe48, 3fb80248]
CACHE: Misaligned operation at range [3fb7fe48, 3fb80248]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb7fe48
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80248
** File not found /boot/boot.bmp **
** File not found /boot/boot.bmp **

N2350 > bootflow scan scsi
CACHE: Misaligned operation at range [3fb88a88, 3fb88e88]
CACHE: Misaligned operation at range [3fb88a88, 3fb88e88]
CACHE: Misaligned operation at range [3fb88a88, 3fb88e88]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb88a88
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb88e88
** File not found /boot/boot.bmp **
** File not found /boot/boot.bmp **

N2350 > bootflow l
Showing all bootflows
Seq  Method       State   Uclass    Part  Name                      Filename
---  -----------  ------  --------  ----  ------------------------
----------------
  0  script       ready   scsi         1  ahci_scsi.id1lun0.bootdev
/boot/boot.scr
  1  script       ready   usb_mass_    1  usb_mass_storage.lun0.boo
/boot/boot.scr
---  -----------  ------  --------  ----  ------------------------
----------------
(2 bootflows, 2 valid)

N2350 > boot
Scanning for bootflows in all bootdevs
Seq  Method       State   Uclass    Part  Name                      Filename
---  -----------  ------  --------  ----  ------------------------
----------------
Scanning global bootmeth 'efi_mgr':
Hunting with: mmc
Scanning bootdev 'ahci_scsi.id1lun0.bootdev':
CACHE: Misaligned operation at range [3fb91d08, 3fb92108]
CACHE: Misaligned operation at range [3fb91d08, 3fb92108]
CACHE: Misaligned operation at range [3fb91d08, 3fb92108]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb91d08
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb92108
** File not found /boot/boot.bmp **
  0  script       ready   scsi         1  ahci_scsi.id1lun0.bootdev
/boot/boot.scr
** Booting bootflow 'ahci_scsi.id1lun0.bootdev.part_1' with script
Booting with distro boot script
loading uImage from scsi 0:1 ...
5118888 bytes read in 120 ms (40.7 MiB/s)
loading uInitrd from scsi 0:1 ...
7355687 bytes read in 197 ms (35.6 MiB/s)
loading DTB file from scsi 0:1 ...
20906 bytes read in 16 ms (1.2 MiB/s)
## Booting kernel from Legacy Image at 01000000 ...
   Image Name:   Linux-6.4.11-mvebu-tld-1
   Created:      2023-08-20  17:34:55 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    5118824 Bytes = 4.9 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
## Loading init Ramdisk from Legacy Image at 02200000 ...
   Image Name:   initramfs-6.4.11-mvebu-tld-1
   Created:      2023-08-20  17:34:57 UTC
   Image Type:   ARM Linux RAMDisk Image (gzip compressed)
   Data Size:    7355623 Bytes = 7 MiB
   Load Address: 00000000
   Entry Point:  00000000
   Verifying Checksum ... OK
## Flattened Device Tree blob at 02000000
   Booting using the fdt blob at 0x2000000
Working FDT set to 2000000


Starting kernel ...

   Loading Kernel Image
   Loading Ramdisk to 0f8fc000, end 0ffffce7 ... OK
   Loading Device Tree to 0f8f3000, end 0f8fb1a9 ... OK
Working FDT set to f8f3000

Starting kernel ...

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 801 bytes --]

On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:

> I've been testing the boostd for a few Marvell boards and seeing this
> error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> "bootflow scan scsi" command triggered the "CACHE: Misaligned
> operation at range" error. However, this error did not affect the
> result of the scan, i.e. the bootflow for scsi partition was created
> correctly, and u-boot is running normally.
> 
> Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> Perhaps this is a case where the DCACHE is not required and should be
> turned off?
> 
> Please see the log after the break below.
 
Can you please try -next ?  There's at least one SCSI related cache
alignment fix there that's not in master, thanks.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Tom,

On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
>
> On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
>
> > I've been testing the boostd for a few Marvell boards and seeing this
> > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > operation at range" error. However, this error did not affect the
> > result of the scan, i.e. the bootflow for scsi partition was created
> > correctly, and u-boot is running normally.
> >
> > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > Perhaps this is a case where the DCACHE is not required and should be
> > turned off?
> >
> > Please see the log after the break below.
>
> Can you please try -next ?  There's at least one SCSI related cache
> alignment fix there that's not in master, thanks.

Unfortunately I got the same errors. This time the ranges are
different, of course.

master:

N2350 > bootflow scan scsi
CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388

next:

N2350 > bootflow scan scsi
CACHE: Misaligned operation at range [3fb80388, 3fb80788]
CACHE: Misaligned operation at range [3fb80388, 3fb80788]
CACHE: Misaligned operation at range [3fb80388, 3fb80788]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788

Thanks,
Tony

>
> --
> Tom

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tom Rini]

[-- Attachment #1: Type: text/plain, Size: 2042 bytes --]

On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> Hi Tom,
> 
> On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> >
> > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> >
> > > I've been testing the boostd for a few Marvell boards and seeing this
> > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > operation at range" error. However, this error did not affect the
> > > result of the scan, i.e. the bootflow for scsi partition was created
> > > correctly, and u-boot is running normally.
> > >
> > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > Perhaps this is a case where the DCACHE is not required and should be
> > > turned off?
> > >
> > > Please see the log after the break below.
> >
> > Can you please try -next ?  There's at least one SCSI related cache
> > alignment fix there that's not in master, thanks.
> 
> Unfortunately I got the same errors. This time the ranges are
> different, of course.
> 
> master:
> 
> N2350 > bootflow scan scsi
> CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> 
> next:
> 
> N2350 > bootflow scan scsi
> CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788

Can you debug to where these calls are so we can align these buffers?
See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Tom,

On Wed, Sep 13, 2023 at 1:13 PM Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > Hi Tom,
> >
> > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > >
> > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > operation at range" error. However, this error did not affect the
> > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > correctly, and u-boot is running normally.
> > > >
> > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > turned off?
> > > >
> > > > Please see the log after the break below.
> > >
> > > Can you please try -next ?  There's at least one SCSI related cache
> > > alignment fix there that's not in master, thanks.
> >
> > Unfortunately I got the same errors. This time the ranges are
> > different, of course.
> >
> > master:
> >
> > N2350 > bootflow scan scsi
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> >
> > next:
> >
> > N2350 > bootflow scan scsi
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
>
> Can you debug to where these calls are so we can align these buffers?
> See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.

I've added some printfs to the code. Looks like the last calls were
SCSI_READ16 (0x48) in this sequence:
scsi_exec() --> ahci_scsi_exec() --> ata_scsiop_read_write()

Note that the misaligned errors do not always occur.

I've also looked at the other recent commit by Marek and tried to
enable CONFIG_BOUNCE_BUFFER:
https://github.com/u-boot/u-boot/commit/4f543e82b9831333bc0effe9540d8e6a9dde3cb5
But I think it is a noop without a callback from the driver.

Please see the boot log below after the break.

Thanks,
Tony

============

U-Boot 2023.10-rc4-tld-1-00284-gce67ba1e30-dirty (Sep 13 2023 - 16:44:56 -0700)
Thecus N2350

SoC:   MV88F6820-A0 at 1066 MHz
DRAM:  1 GiB (533 MHz, 32-bit, ECC not enabled)
Core:  65 devices, 23 uclasses, devicetree: separate
NAND:  512 MiB
MMC:
Loading Environment from SPIFlash... SF: Detected mx25l3205d with page
size 256 Bytes, erase size 4 KiB, total 4 MiB
*** Warning - bad CRC, using default environment

Model: Thecus N2350
Net:
Warning: ethernet@70000 (eth0) using random MAC address - a2:6f:f1:4a:2e:51
eth0: ethernet@70000
Hit any key to stop autoboot:  0

N2350 > env def -a
## Resetting to default environment

N2350 > bootdev l
Seq  Probed  Status  Uclass    Name
---  ------  ------  --------  ------------------
  0   [   ]      OK  ethernet  ethernet@70000.bootdev
---  ------  ------  --------  ------------------
(1 bootdev)

N2350 > bootdev hunt scsi
Hunting with: scsi
pcie0.0: Link down
pcie1.0: Link down
scsi_scan
scanning bus for devices...
scsi_scan_dev
SATA link 0 timeout.
Target spinup took 0 ms.
AHCI 0001.0000 32 slots 2 ports 6 Gbps 0x3 impl SATA mode
flags: 64bit ncq led only pmp fbss pio slum part sxs
do_scsi_scan_one
scsi_detect_dev
scsi_setup_inquiry
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x12
do_scsi_scan_one
scsi_detect_dev
scsi_setup_inquiry
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x12
scsi_ident_cpy
scsi_ident_cpy
scsi_ident_cpy
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x0
scsi_read_capacity
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x25
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
  Device 0: (1:0) Vendor: ATA Prod.: ST750LX003-1AC15 Rev: SM12
            Type: Hard Disk
            Capacity: 715404.8 MB = 698.6 GB (1465149168 x 512)

N2350 > bootflow scan scsi
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x28
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb6f8d8
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb6fcd8
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
CACHE: Misaligned operation at range [3fb6f8d8, 3fb6fcd8]
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb6f8d8
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb6fcd8
scsi_exec
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ata_scsiop_read_write
** File not found /boot/boot.bmp **




>
> --
> Tom

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Simon Glass]

Hi Tom,

On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini@konsulko.com> wrote:
>
> On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > Hi Tom,
> >
> > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > >
> > > > I've been testing the boostd for a few Marvell boards and seeing
this
> > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > operation at range" error. However, this error did not affect the
> > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > correctly, and u-boot is running normally.
> > > >
> > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > Perhaps this is a case where the DCACHE is not required and should
be
> > > > turned off?
> > > >
> > > > Please see the log after the break below.
> > >
> > > Can you please try -next ?  There's at least one SCSI related cache
> > > alignment fix there that's not in master, thanks.
> >
> > Unfortunately I got the same errors. This time the ranges are
> > different, of course.
> >
> > master:
> >
> > N2350 > bootflow scan scsi
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > ERROR: v7_outer_cache_inval_range - start address is not aligned -
0x3fb99f88
> > ERROR: v7_outer_cache_inval_range - stop address is not aligned -
0x3fb9a388
> >
> > next:
> >
> > N2350 > bootflow scan scsi
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > ERROR: v7_outer_cache_inval_range - start address is not aligned -
0x3fb80388
> > ERROR: v7_outer_cache_inval_range - stop address is not aligned -
0x3fb80788
>
> Can you debug to where these calls are so we can align these buffers?
> See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.

I wonder if we need to use memalign() when allocating memory to read things
from the media? But I am not sure which file time is being read, or which
bootmeth it is.

Regards,
Simon

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Simon,

On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg@chromium.org> wrote:
>
> Hi Tom,
>
> On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini@konsulko.com> wrote:
> >
> > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > Hi Tom,
> > >
> > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > >
> > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > operation at range" error. However, this error did not affect the
> > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > correctly, and u-boot is running normally.
> > > > >
> > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > turned off?
> > > > >
> > > > > Please see the log after the break below.
> > > >
> > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > alignment fix there that's not in master, thanks.
> > >
> > > Unfortunately I got the same errors. This time the ranges are
> > > different, of course.
> > >
> > > master:
> > >
> > > N2350 > bootflow scan scsi
> > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > >
> > > next:
> > >
> > > N2350 > bootflow scan scsi
> > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> >
> > Can you debug to where these calls are so we can align these buffers?
> > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
>
> I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.

Looks like we probably need to align the buffer tempbuff.

/drivers/scsi/scsi.c
static int scsi_detect_dev(struct udevice *dev, int target, int lun,
  struct blk_desc *dev_desc)
{
unsigned char perq, modi;
lbaint_t capacity;
unsigned long blksz;
struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
int count, err;

pccb->target = target;
pccb->lun = lun;
pccb->pdata = (unsigned char *)&tempbuff;
pccb->datalen = 512;

If you look at the log I posted previously, this error shows up during
"bootflow scan scsi".

All the best,
Tony



>
> Regards,
> Simon

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Tom, Hi Simon,

On Wed, Sep 13, 2023 at 9:53 PM Tony Dinh <mibodhi@gmail.com> wrote:
>
> Hi Simon,
>
> On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg@chromium.org> wrote:
> >
> > Hi Tom,
> >
> > On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini@konsulko.com> wrote:
> > >
> > > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > > Hi Tom,
> > > >
> > > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > > >
> > > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > > operation at range" error. However, this error did not affect the
> > > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > > correctly, and u-boot is running normally.
> > > > > >
> > > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > > turned off?
> > > > > >
> > > > > > Please see the log after the break below.
> > > > >
> > > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > > alignment fix there that's not in master, thanks.
> > > >
> > > > Unfortunately I got the same errors. This time the ranges are
> > > > different, of course.
> > > >
> > > > master:
> > > >
> > > > N2350 > bootflow scan scsi
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > > >
> > > > next:
> > > >
> > > > N2350 > bootflow scan scsi
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> > >
> > > Can you debug to where these calls are so we can align these buffers?
> > > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
> >
> > I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.
>
> Looks like we probably need to align the buffer tempbuff.
>
> /drivers/scsi/scsi.c
> static int scsi_detect_dev(struct udevice *dev, int target, int lun,
>   struct blk_desc *dev_desc)
> {
> unsigned char perq, modi;
> lbaint_t capacity;
> unsigned long blksz;
> struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
> int count, err;
>
> pccb->target = target;
> pccb->lun = lun;
> pccb->pdata = (unsigned char *)&tempbuff;
> pccb->datalen = 512;
>
> If you look at the log I posted previously, this error shows up during
> "bootflow scan scsi".
>

Taking the hint from Simon. I turned on log_debug and can see where
the alignment is not correct. It is fs.c fs_read_alloc(). The
memalign() call here probably needs to be revised to take into
consideration ARCH_DMA_MINALIGN somehow? It is 64 for armv7.

diff --git a/fs/fs.c b/fs/fs.c
index 2b815b1db0..b70281532e 100644
--- a/fs/fs.c
+++ b/fs/fs.c
@@ -1019,9 +1019,12 @@ int fs_read_alloc(const char *fname, ulong
size, uint align, void **bufp)
        int ret;

        buf = memalign(align, size + 1);
+       log_debug("aligned buf addr 0x%x\n", (unsigned int)buf);
+
        if (!buf)
                return log_msg_ret("buf", -ENOMEM);
        addr = map_to_sysmem(buf);
+       log_debug("aligned buf sysmem addr 0x%x\n", (unsigned int)addr);

        ret = fs_read(fname, addr, 0, size, &bytes_read);
        if (ret) {

Please see the log below after the break.

All the best,
Tony

========

Log:

   bootmeth_try_file()    /boot/boot.scr - err=0
          fs_devread()  <2, 0, 1024>
          fs_devread()  <8, 0, 32>
          fs_devread()  <8216, 256, 128>

bootmeth_alloc_file()    - script file size 4b7

       fs_read_alloc() aligned buf addr 0x3eb4b608
       fs_read_alloc() aligned buf sysmem addr 0x3eb4b608
          fs_devread()  <12312, 0, 8>

<snip>

          fs_devread()  <551551112, 256, 128>
          fs_devread()  <551592072, 0, 1207>

scsi_read buffer addr=0x3eb4b608
scsi_read_ext: startblk 20e0a88a, blccnt 0 buffer 3EB4B608
scsi_exec pdata=0x3eb4b608
ahci_scsi_exec
ahci_scsi_exec cmd = 0x48
ahci_scsi_exec pdata=0x3eb4b608
ata_scsiop_read_write
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
ata_scsiop_read_write exiting...
ahci_scsi_exec exiting...
scsi_read exiting...
ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08

All the best,
Tony

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

On Fri, Sep 15, 2023 at 6:32 PM Tony Dinh <mibodhi@gmail.com> wrote:
>
> Hi Tom, Hi Simon,
>
> On Wed, Sep 13, 2023 at 9:53 PM Tony Dinh <mibodhi@gmail.com> wrote:
> >
> > Hi Simon,
> >
> > On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg@chromium.org> wrote:
> > >
> > > Hi Tom,
> > >
> > > On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini@konsulko.com> wrote:
> > > >
> > > > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > > > Hi Tom,
> > > > >
> > > > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > > > > >
> > > > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > > > >
> > > > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > > > operation at range" error. However, this error did not affect the
> > > > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > > > correctly, and u-boot is running normally.
> > > > > > >
> > > > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > > > turned off?
> > > > > > >
> > > > > > > Please see the log after the break below.
> > > > > >
> > > > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > > > alignment fix there that's not in master, thanks.
> > > > >
> > > > > Unfortunately I got the same errors. This time the ranges are
> > > > > different, of course.
> > > > >
> > > > > master:
> > > > >
> > > > > N2350 > bootflow scan scsi
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > > > >
> > > > > next:
> > > > >
> > > > > N2350 > bootflow scan scsi
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> > > >
> > > > Can you debug to where these calls are so we can align these buffers?
> > > > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
> > >
> > > I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.
> >
> > Looks like we probably need to align the buffer tempbuff.
> >
> > /drivers/scsi/scsi.c
> > static int scsi_detect_dev(struct udevice *dev, int target, int lun,
> >   struct blk_desc *dev_desc)
> > {
> > unsigned char perq, modi;
> > lbaint_t capacity;
> > unsigned long blksz;
> > struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
> > int count, err;
> >
> > pccb->target = target;
> > pccb->lun = lun;
> > pccb->pdata = (unsigned char *)&tempbuff;
> > pccb->datalen = 512;
> >
> > If you look at the log I posted previously, this error shows up during
> > "bootflow scan scsi".
> >
>
> Taking the hint from Simon. I turned on log_debug and can see where
> the alignment is not correct. It is fs.c fs_read_alloc(). The
> memalign() call here probably needs to be revised to take into
> consideration ARCH_DMA_MINALIGN somehow? It is 64 for armv7.
>
> diff --git a/fs/fs.c b/fs/fs.c
> index 2b815b1db0..b70281532e 100644
> --- a/fs/fs.c
> +++ b/fs/fs.c
> @@ -1019,9 +1019,12 @@ int fs_read_alloc(const char *fname, ulong
> size, uint align, void **bufp)
>         int ret;
>
>         buf = memalign(align, size + 1);
> +       log_debug("aligned buf addr 0x%x\n", (unsigned int)buf);
> +
>         if (!buf)
>                 return log_msg_ret("buf", -ENOMEM);
>         addr = map_to_sysmem(buf);
> +       log_debug("aligned buf sysmem addr 0x%x\n", (unsigned int)addr);
>
>         ret = fs_read(fname, addr, 0, size, &bytes_read);
>         if (ret) {
>
> Please see the log below after the break.

Actually, it looks like the fix should be in bootmeth_script.c.

diff --git a/boot/bootmeth_script.c b/boot/bootmeth_script.c
index 0269e0f9b0..68e77aa50a 100644
--- a/boot/bootmeth_script.c
+++ b/boot/bootmeth_script.c
@@ -99,7 +99,7 @@ static int script_read_bootflow_file(struct udevice *bootstd,
        if (!bflow->subdir)
                return log_msg_ret("prefix", -ENOMEM);

-       ret = bootmeth_alloc_file(bflow, 0x10000, 1);
+       ret = bootmeth_alloc_file(bflow, 0x10000, ARCH_DMA_MINALIGN);

All the best,
Tony

> ========
>
> Log:
>
>    bootmeth_try_file()    /boot/boot.scr - err=0
>           fs_devread()  <2, 0, 1024>
>           fs_devread()  <8, 0, 32>
>           fs_devread()  <8216, 256, 128>
>
> bootmeth_alloc_file()    - script file size 4b7
>
>        fs_read_alloc() aligned buf addr 0x3eb4b608
>        fs_read_alloc() aligned buf sysmem addr 0x3eb4b608
>           fs_devread()  <12312, 0, 8>
>
> <snip>
>
>           fs_devread()  <551551112, 256, 128>
>           fs_devread()  <551592072, 0, 1207>
>
> scsi_read buffer addr=0x3eb4b608
> scsi_read_ext: startblk 20e0a88a, blccnt 0 buffer 3EB4B608
> scsi_exec pdata=0x3eb4b608
> ahci_scsi_exec
> ahci_scsi_exec cmd = 0x48
> ahci_scsi_exec pdata=0x3eb4b608
> ata_scsiop_read_write
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
> ata_scsiop_read_write exiting...
> ahci_scsi_exec exiting...
> scsi_read exiting...
> ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3eb4b608
> ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3eb4ba08
>
> All the best,
> Tony

Re: bootstd: CACHE Misaligned operation errors (Marvell Armada 385)

[Tony Dinh]

Hi Simon,

On Fri, Sep 15, 2023 at 8:40 PM Tony Dinh <mibodhi@gmail.com> wrote:
>
> On Fri, Sep 15, 2023 at 6:32 PM Tony Dinh <mibodhi@gmail.com> wrote:
> >
> > Hi Tom, Hi Simon,
> >
> > On Wed, Sep 13, 2023 at 9:53 PM Tony Dinh <mibodhi@gmail.com> wrote:
> > >
> > > Hi Simon,
> > >
> > > On Wed, Sep 13, 2023 at 8:38 PM Simon Glass <sjg@chromium.org> wrote:
> > > >
> > > > Hi Tom,
> > > >
> > > > On Wed, 13 Sept 2023 at 14:14, Tom Rini <trini@konsulko.com> wrote:
> > > > >
> > > > > On Wed, Sep 13, 2023 at 12:56:53PM -0700, Tony Dinh wrote:
> > > > > > Hi Tom,
> > > > > >
> > > > > > On Wed, Sep 13, 2023 at 9:22 AM Tom Rini <trini@konsulko.com> wrote:
> > > > > > >
> > > > > > > On Tue, Sep 12, 2023 at 12:38:00PM -0700, Tony Dinh wrote:
> > > > > > >
> > > > > > > > I've been testing the boostd for a few Marvell boards and seeing this
> > > > > > > > error on the Thecus N2350 (Marvell Armada 385, dual-core CPU). The
> > > > > > > > "bootflow scan scsi" command triggered the "CACHE: Misaligned
> > > > > > > > operation at range" error. However, this error did not affect the
> > > > > > > > result of the scan, i.e. the bootflow for scsi partition was created
> > > > > > > > correctly, and u-boot is running normally.
> > > > > > > >
> > > > > > > > Enabling CONFIG_SYS_DCACHE_OFF got rid of the errors altogether.
> > > > > > > > Perhaps this is a case where the DCACHE is not required and should be
> > > > > > > > turned off?
> > > > > > > >
> > > > > > > > Please see the log after the break below.
> > > > > > >
> > > > > > > Can you please try -next ?  There's at least one SCSI related cache
> > > > > > > alignment fix there that's not in master, thanks.
> > > > > >
> > > > > > Unfortunately I got the same errors. This time the ranges are
> > > > > > different, of course.
> > > > > >
> > > > > > master:
> > > > > >
> > > > > > N2350 > bootflow scan scsi
> > > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > > CACHE: Misaligned operation at range [3fb99f88, 3fb9a388]
> > > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb99f88
> > > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb9a388
> > > > > >
> > > > > > next:
> > > > > >
> > > > > > N2350 > bootflow scan scsi
> > > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > > CACHE: Misaligned operation at range [3fb80388, 3fb80788]
> > > > > > ERROR: v7_outer_cache_inval_range - start address is not aligned - 0x3fb80388
> > > > > > ERROR: v7_outer_cache_inval_range - stop address is not aligned - 0x3fb80788
> > > > >
> > > > > Can you debug to where these calls are so we can align these buffers?
> > > > > See 02660defdc8a ("scsi: Cache align temporary buffer") for an example.
> > > >
> > > > I wonder if we need to use memalign() when allocating memory to read things from the media? But I am not sure which file time is being read, or which bootmeth it is.
> > >
> > > Looks like we probably need to align the buffer tempbuff.
> > >
> > > /drivers/scsi/scsi.c
> > > static int scsi_detect_dev(struct udevice *dev, int target, int lun,
> > >   struct blk_desc *dev_desc)
> > > {
> > > unsigned char perq, modi;
> > > lbaint_t capacity;
> > > unsigned long blksz;
> > > struct scsi_cmd *pccb = (struct scsi_cmd *)&tempccb;
> > > int count, err;
> > >
> > > pccb->target = target;
> > > pccb->lun = lun;
> > > pccb->pdata = (unsigned char *)&tempbuff;
> > > pccb->datalen = 512;
> > >
> > > If you look at the log I posted previously, this error shows up during
> > > "bootflow scan scsi".
> > >
> >
> > Taking the hint from Simon. I turned on log_debug and can see where
> > the alignment is not correct. It is fs.c fs_read_alloc(). The
> > memalign() call here probably needs to be revised to take into
> > consideration ARCH_DMA_MINALIGN somehow? It is 64 for armv7.
> >
> > diff --git a/fs/fs.c b/fs/fs.c
> > index 2b815b1db0..b70281532e 100644
> > --- a/fs/fs.c
> > +++ b/fs/fs.c
> > @@ -1019,9 +1019,12 @@ int fs_read_alloc(const char *fname, ulong
> > size, uint align, void **bufp)
> >         int ret;
> >
> >         buf = memalign(align, size + 1);
> > +       log_debug("aligned buf addr 0x%x\n", (unsigned int)buf);
> > +
> >         if (!buf)
> >                 return log_msg_ret("buf", -ENOMEM);
> >         addr = map_to_sysmem(buf);
> > +       log_debug("aligned buf sysmem addr 0x%x\n", (unsigned int)addr);
> >
> >         ret = fs_read(fname, addr, 0, size, &bytes_read);
> >         if (ret) {
> >
> > Please see the log below after the break.
>
> Actually, it looks like the fix should be in bootmeth_script.c.
>
> diff --git a/boot/bootmeth_script.c b/boot/bootmeth_script.c
> index 0269e0f9b0..68e77aa50a 100644
> --- a/boot/bootmeth_script.c
> +++ b/boot/bootmeth_script.c
> @@ -99,7 +99,7 @@ static int script_read_bootflow_file(struct udevice *bootstd,
>         if (!bflow->subdir)
>                 return log_msg_ret("prefix", -ENOMEM);
>
> -       ret = bootmeth_alloc_file(bflow, 0x10000, 1);
> +       ret = bootmeth_alloc_file(bflow, 0x10000, ARCH_DMA_MINALIGN);

Yes, it is working with the patch above. I don't see the misaligned
errors anymore.

All the best ,
Tony