[U-Boot] [PATCH v7 3/7] ARM: socfpga: Add FPGA drivers for Arria 10 FPGA bitstream loading

Marek Vasut
2019-02-05T08:41:51Z

On 2/2/19 4:27 AM, Chee, Tien Fong wrote:
> On Fri, 2019-02-01 at 12:12 -0800, Dalon L Westergreen wrote:
>> On Thu, 2019-01-31 at 22:51 +0800, tien.fong.chee at intel.com wrote:
>>>
>>> From: Tien Fong Chee <tien.fong.chee@intel.com>
>>>
>>> Add FPGA driver to support program FPGA with FPGA bitstream loading
>>> from
>>> filesystem. The driver are designed based on generic firmware
>>> loader
>>> framework. The driver can handle FPGA program operation from
>>> loading FPGA
>>> bitstream in flash to memory and then to program FPGA.
>>>
>>> Signed-off-by: Tien Fong Chee <tien.fong.chee@intel.com>
>>>
>>> ---
>>>
>>> changes for v7
>>> - Restructure the FPGA driver to support both peripheral bitstream
>>> and core
>>>   bitstream bundled into FIT image.
>>> - Support loadable property for core bitstream. User can set
>>> loadable
>>>   in DDR for better performance. This loading would be done in one
>>> large
>>>   chunk instead of chunk by chunk loading with small memory buffer.
>>> ---
>>>  arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts       |  18 +
>>>  .../include/mach/fpga_manager_arria10.h            |  39 +-
>>>  drivers/fpga/socfpga_arria10.c                     | 417
>>> ++++++++++++++++++++-
>>>  3 files changed, 457 insertions(+), 17 deletions(-)
>>>
>>> diff --git a/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> b/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> index 998d811..dc55618 100644
>>> --- a/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> +++ b/arch/arm/dts/socfpga_arria10_socdk_sdmmc.dts
>>> @@ -18,6 +18,24 @@
>>>  /dts-v1/;
>>>  #include "socfpga_arria10_socdk.dtsi"
>>>
>>> +/ {
>>> + chosen {
>>> + firmware-loader = &fs_loader0;
>>> + };
>>> +
>>> + fs_loader0: fs-loader at 0 {
>>> + u-boot,dm-pre-reloc;
>>> + compatible = "u-boot,fs-loader";
>>> + phandlepart = <&mmc 1>;
>>> + };
>>> +};
>>> +
>>> +&fpga_mgr {
>>> + u-boot,dm-pre-reloc;
>>> + altr,bitstream = "fit_spl_fpga.itb";
>>> + altr,bitstream-core = "fit_spl_fpga.itb";
>>> +};
>>> +
>>>  &mmc {
>>>   u-boot,dm-pre-reloc;
>>>   status = "okay";
>>> diff --git a/arch/arm/mach-
>>> socfpga/include/mach/fpga_manager_arria10.h
>>> b/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> index 09d13f6..683c84c 100644
>>> --- a/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> +++ b/arch/arm/mach-socfpga/include/mach/fpga_manager_arria10.h
>>> @@ -1,9 +1,13 @@
>>>  /* SPDX-License-Identifier: GPL-2.0 */
>>>  /*
>>> - * Copyright (C) 2017 Intel Corporation <www.intel.com>
>>> + * Copyright (C) 2017-2019 Intel Corporation <www.intel.com>
>>>   * All rights reserved.
>>>   */
>>>
>>>
>> [...] }
>>>
>>> +
>>> + if (!is_fpgamgr_early_user_mode() ||
>>> is_fpgamgr_user_mode()) {
>>> + fpga_node_name = "fpga-1";
>>> + printf("FPGA: Start to program peripheral
>>> bitstream ...\n");
>>> + } else if (!is_fpgamgr_user_mode()) {
>>> + fpga_node_name = "fpga-2";
>>> + printf("FPGA: Start to program core bitstream
>>> ...\n");
>>> + }
>> Why are you relying on the node name to define the core / periph RBF?
>> Would
>> it not be better to define this in a property?
> I need to ensure correct sequence of programming the rbf images at
> different FPGA mode. fpga with index 1 always means the 1st to program.
>
>>  how would one have multiple
>> core and periph rbfs in a single fit image?
> I want strict control to avoid the performance penalty, when this
> happen, performance is almost same as programming full image.
>
> So, in a single FIT image, only have one core rbf and periph rbf, and
> core rbf must always be the 1st and with the right absolute data
> position to avoid the buffer unaligned. Strict sequence control is also
> one of the reason having one core rbf and one periph rbf in a single
> FIT image.

I thought we established that you cannot depend on the placement and
alignment of files within fitImage ?

Also, you should be able to use fitImage configurations to select from
multiple core and periph RBFs, there's no point in limiting ourselves to
1 core and 1 periph RBF with flexible format such as the fitImage.

> You can create multiple FIT image for multiple core and periph rbfs.
> There are properties such as "altr,bitstream" and "altr,bitstream-core"
> for supporting different FIT image.
>
> You can change those properties in U-Boot env(for runtime), or using
> different DTS for differnt configuration.
>
>>
>> --dalon
>>
>>>
>>> +
>>> + node_offset = fit_image_get_node(buffer_p,
>>> fpga_node_name);
>>> + if (node_offset < 0) {
>>> + debug("FPGA: Could not find node '%s' in FIT\n",
>>> +      fpga_node_name);
>>> + return -ENOENT;
>>> + }
>>> +
>>> + const char *uname = fit_get_name(buffer_p, node_offset,
>>> NULL);
>>> +
>>> + if (uname)
>>> + debug("FPGA: %s\n", uname);
>>> +
>>> + ret = fit_image_get_data_position(buffer_p, node_offset,
>>> &rbf_offset);
>>> + if (ret < 0) {
>>> + debug("FPGA: Could not find data position
>>> (err=%d)\n", ret);
>>> + return -ENOENT;
>>> + }
>>> +
>>> + ret = fit_image_get_data_size(buffer_p, node_offset,
>>> &rbf_size);
>>> + if (ret < 0) {
>>> + debug("FPGA: Could not find data size (err=%d)\n",
>>> ret);
>>> + return -ENOENT;
>>> + }
>>> +
>>> + ret = fit_image_get_load(buffer_p, node_offset, (ulong
>>> *)loadable);
>>> + if (ret < 0) {
>>> + debug("FPGA: Could not find loadable (err=%d)\n",
>>> ret);
>>> + debug("FPGA: Using default buffer and size\n");
>>> + } else {
>>> + buffer_p = (u32 *)*loadable;
>>> + buffer_size = rbf_size;
>>> + debug("FPGA: Found loadable address = 0x%x\n",
>>> *loadable);
>>> + }
>>> +
>>> + debug("FPGA: External data: offset = 0x%x, size = 0x%x\n",
>>> +       rbf_offset, rbf_size);
>>> +
>>> + fpga_loadfs->remaining = rbf_size;
>>> +
>>> + /*
>>> +  * Determine buffer size vs bitstream size, and
>>> calculating number of
>>> +  * chunk by chunk transfer is required due to smaller
>>> buffer size
>>> +  * compare to bitstream
>>> +  */
>>> + if (rbf_size <= buffer_size) {
>>> + /* Loading whole bitstream into buffer */
>>> + buffer_size = rbf_size;
>>> + fpga_loadfs->remaining = 0;
>>> + } else {
>>> + fpga_loadfs->remaining -= buffer_size;
>>> + }
>>> +
>>> + fpga_loadfs->offset = rbf_offset;
>>> + /* Loading bitstream into buffer */
>>> + ret = request_firmware_into_buf(dev,
>>> + fpga_loadfs->fpga_fsinfo-
>>>> filename,
>>> + buffer_p,
>>> + buffer_size,
>>> + fpga_loadfs->offset);
>>> + if (ret < 0) {
>>> + debug("FPGA: Failed to read bitstream from
>>> flash.\n");
>>> + return -ENOENT;
>>> + }
>>> +
>>> + /* Update next reading bitstream offset */
>>> + fpga_loadfs->offset += buffer_size;
>>> +
>>> + /* Getting info about bitstream types */
>>> + get_rbf_image_info(&fpga_loadfs->rbfinfo, (u16
>>> *)buffer_p);
>>> +
>>> + /* Update the final addr for bitstream */
>>> + *buffer = (u32)buffer_p;
>>> +
>>> + /* Update the size of bitstream to be programmed into FPGA
>>> */
>>> + *buffer_bsize = buffer_size;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +static int subsequent_loading_rbf_to_buffer(struct udevice *dev,
>>> + struct fpga_loadfs_info
>>> *fpga_loadfs,
>>> + u32 *buffer, size_t
>>> *buffer_bsize)
>>> +{
>>> + int ret = 0;
>>> + u32 *buffer_p = (u32 *)*buffer;
>>> +
>>> + /* Read the bitstream chunk by chunk. */
>>> + if (fpga_loadfs->remaining > *buffer_bsize) {
>>> + fpga_loadfs->remaining -= *buffer_bsize;
>>> + } else {
>>> + *buffer_bsize = fpga_loadfs->remaining;
>>> + fpga_loadfs->remaining = 0;
>>> + }
>>> +
>>> + ret = request_firmware_into_buf(dev,
>>> + fpga_loadfs->fpga_fsinfo-
>>>> filename,
>>> + buffer_p,
>>> + *buffer_bsize,
>>> + fpga_loadfs->offset);
>>> + if (ret < 0) {
>>> + debug("FPGA: Failed to read bitstream from
>>> flash.\n");
>>> + return -ENOENT;
>>> + }
>>> +
>>> + /* Update next reading bitstream offset */
>>> + fpga_loadfs->offset += *buffer_bsize;
>>> +
>>> + return 0;
>>> +}
>>> +
>>> +int socfpga_loadfs(fpga_fs_info *fpga_fsinfo, const void *buf,
>>> size_t bsize,
>>> + u32 offset)
>>> +{
>>> + struct fpga_loadfs_info fpga_loadfs;
>>> + int status = 0;
>>> + int ret = 0;
>>> + u32 buffer = (u32)buf;
>>> + size_t buffer_sizebytes = bsize;
>>> + size_t buffer_sizebytes_ori = bsize;
>>> + size_t total_sizeof_image = 0;
>>> + struct udevice *dev;
>>> +
>>> + ret = uclass_get_device(UCLASS_FS_FIRMWARE_LOADER, 0,
>>> &dev);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + memset(&fpga_loadfs, 0, sizeof(fpga_loadfs));
>>> +
>>> + WATCHDOG_RESET();
>>> +
>>> + fpga_loadfs.fpga_fsinfo = fpga_fsinfo;
>>> + fpga_loadfs.offset = offset;
>>> +
>>> + printf("FPGA: Start to program ...\n");
>>> +
>>> + /*
>>> +  * Note: Both buffer and buffer_sizebytes values can be
>>> altered by
>>> +  * function below.
>>> +  */
>>> + ret = first_loading_rbf_to_buffer(dev, &fpga_loadfs,
>>> &buffer,
>>> +    &buffer_sizebytes);
>>> + if (ret)
>>> + return ret;
>>> +
>>> + if (fpga_loadfs.rbfinfo.section == core_section &&
>>> + !(is_fpgamgr_early_user_mode() &&
>>> !is_fpgamgr_user_mode())) {
>>> + debug("FPGA : Must be in Early Release mode to
>>> program ");
>>> + debug("core bitstream.\n");
>>> + return 0;
>>> + }
>>> +
>>> + /* Disable all signals from HPS peripheral controller to
>>> FPGA */
>>> + writel(0, &system_manager_base->fpgaintf_en_global);
>>> +
>>> + /* Disable all axi bridges (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>> + socfpga_bridges_reset();
>>> +
>>> + if (fpga_loadfs.rbfinfo.section == periph_section) {
>>> + /* Initialize the FPGA Manager */
>>> + status = fpgamgr_program_init((u32 *)buffer,
>>> buffer_sizebytes);
>>> + if (status) {
>>> + debug("FPGA: Init with peripheral
>>> bitstream failed.\n");
>>> + return -EPERM;
>>> + }
>>> + }
>>> +
>>> + WATCHDOG_RESET();
>>> +
>>> + /* Transfer bitstream to FPGA Manager */
>>> + fpgamgr_program_write((void *)buffer, buffer_sizebytes);
>>> +
>>> + total_sizeof_image += buffer_sizebytes;
>>> +
>>> + WATCHDOG_RESET();
>>> +
>>> + while (fpga_loadfs.remaining) {
>>> + ret = subsequent_loading_rbf_to_buffer(dev,
>>> + &fpga_load
>>> fs,
>>> + &buffer,
>>> + &buffer_si
>>> zebytes_ori);
>>> +
>>> + if (ret)
>>> + return ret;
>>> +
>>> + /* Transfer data to FPGA Manager */
>>> + fpgamgr_program_write((void *)buffer,
>>> + buffer_sizebytes_ori);
>>> +
>>> + total_sizeof_image += buffer_sizebytes_ori;
>>> +
>>> + WATCHDOG_RESET();
>>> + }
>>> +
>>> + if (fpga_loadfs.rbfinfo.section == periph_section) {
>>> + if (fpgamgr_wait_early_user_mode() != -ETIMEDOUT)
>>> {
>>> + config_pins(gd->fdt_blob, "shared");
>>> + puts("FPGA: Early Release Succeeded.\n");
>>> + } else {
>>> + debug("FPGA: Failed to see Early
>>> Release.\n");
>>> + return -EIO;
>>> + }
>>> +
>>> + /* For monolithic bitstream */
>>> + if (is_fpgamgr_user_mode()) {
>>> + /* Ensure the FPGA entering config done */
>>> + status = fpgamgr_program_finish();
>>> + if (status)
>>> + return status;
>>> +
>>> + config_pins(gd->fdt_blob, "fpga");
>>> + puts("FPGA: Enter user mode.\n");
>>> + }
>>> + } else if (fpga_loadfs.rbfinfo.section == core_section) {
>>> + /* Ensure the FPGA entering config done */
>>> + status = fpgamgr_program_finish();
>>> + if (status)
>>> + return status;
>>> +
>>> + config_pins(gd->fdt_blob, "fpga");
>>> + puts("FPGA: Enter user mode.\n");
>>> + } else {
>>> + debug("FPGA: Config Error: Unsupported bitstream
>>> type.\n");
>>> + return -ENOEXEC;
>>> + }
>>> +
>>> + return (int)total_sizeof_image;
>>> +}
>>> +#endif
>>> +
>>> +/* This function is used to load the core bitstream from the
>>> OCRAM. */
>>>  int socfpga_load(Altera_desc *desc, const void *rbf_data, size_t
>>> rbf_size)
>>>  {
>>> - int status;
>>> + unsigned long status;
>>> + struct rbf_info rbfinfo;
>>>
>>> - /* disable all signals from hps peripheral controller to
>>> fpga */
>>> + memset(&rbfinfo, 0, sizeof(rbfinfo));
>>> +
>>> + /* Disable all signals from hps peripheral controller to
>>> fpga */
>>>   writel(0, &system_manager_base->fpgaintf_en_global);
>>>
>>> - /* disable all axi bridge (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>> + /* Disable all axi bridge (hps2fpga, lwhps2fpga &
>>> fpga2hps) */
>>>   socfpga_bridges_reset();
>>>
>>> - /* Initialize the FPGA Manager */
>>> - status = fpgamgr_program_init((u32 *)rbf_data, rbf_size);
>>> - if (status)
>>> - return status;
>>> + /* Getting info about bitstream types */
>>> + get_rbf_image_info(&rbfinfo, (u16 *)rbf_data);
>>> +
>>> + if (rbfinfo.section == periph_section) {
>>> + /* Initialize the FPGA Manager */
>>> + status = fpgamgr_program_init((u32 *)rbf_data,
>>> rbf_size);
>>> + if (status)
>>> + return status;
>>> + }
>>> +
>>> + if (rbfinfo.section == core_section &&
>>> + !(is_fpgamgr_early_user_mode() &&
>>> !is_fpgamgr_user_mode())) {
>>> + debug("FPGA : Must be in early release mode to
>>> program ");
>>> + debug("core bitstream.\n");
>>> + return 0;
>>> + }
>>>
>>> - /* Write the RBF data to FPGA Manager */
>>> + /* Write the bitstream to FPGA Manager */
>>>   fpgamgr_program_write(rbf_data, rbf_size);
>>>
>>> - return fpgamgr_program_finish();
>>> + status = fpgamgr_program_finish();
>>> + if (status) {
>>> + config_pins(gd->fdt_blob, "fpga");
>>> + puts("FPGA: Enter user mode.\n");
>>> + }
>>> +
>>> + return status;
>>>  }

--
Best regards,
Marek Vasut