* OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions @ 2024-05-20 17:32 Johannes Truschnigg 2024-05-21 1:05 ` Andrew Jeffery 2024-07-07 9:19 ` Johannes Truschnigg 0 siblings, 2 replies; 11+ messages in thread From: Johannes Truschnigg @ 2024-05-20 17:32 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 3709 bytes --] Dear list, I am new to this whole BMC-innards thing, and the very opposite of familiar with embedded devices, so please bear with me on this :) I have a Gigabyte MC12-LE0[0] mainboard in hand, which has an AST2500-based integrated BMC running AMI-supplied (mildly customized, I gather) BMC firmware. I would very much like to try to get OpenBMC to boot on this BMC instead of the stock fw, but have hit a number of walls during my first baby steps with this over the last few days, which is why I turn to you to seek help and guidance. The BMC implementation seems lean very much onto the AST2500 Evaluation Board (the unchanged name pops up in a number of places in the stock firmware), so I would guess that the OpenBMC evb-ast2500 machine should be able to get somewhere (and even if it's just to see OpenBMC's instead of AMI's Linux start, and then crash hard due to some subtle incompatibility :)) at least. Afaict, the AMI BMC firmware lineage uses another (custom, undisclosed?) image format that seems to be called "FMH" internally. While there are some third party tools[1] and patches to support it[2] floating around on the net, I failed to get anywhere with either until now. My understanding is that there are two paths forward: 1.) To make OpenBMC's build artifacts into FMH-style BLOBs, and find a way to feed them to the stock fw's u-boot, which would ideally result in OpenBMC being able to boot this way. 2.) To replace the stock fw's u-boot release with an OpenBMC u-boot that can load the evb-ast2500 artifacts natively, and have the BMC boot OpenBMC that way. Can you please confirm that this assessment is sane, or correct me if it isn't? If so, I would presume option 2.) to be the easier road to travel, but I am somewhat stumped as to how I can get OpenBMC's u-boot onto the BMC, *ideally* (but not necessarily) in a non-destructive way (as I do not know how to or have a way to revert to the original state without access to the bootloader or even stock firmware running). So far, I tried to put the u-boot-evb-ast2500-v2019.04+git-r0.bin artifact from my OpenBMC build results into a memory location on the BMC using TFTP, and `go` to that address afterwards - but that effectively reloaded the *original* u-boot from the vendor fw. I don't know enough about the inner workings of an AST2500 or u-boot (or any other embedded bootloader, really), but if you could enlighten me as to what has happened there, I'd be very grateful (... maybe u-boot does a check if it's already reloacted itself towards end of phys. memory, and directly jump there if that is deemed the case?) Do you have any hints how I could achieve what I am trying to do? Is this even feasible? I did find a Github issue[3] from 2017 which hints at others having tried something like this, but no documented successes. I'd be very grateful for your input on this matter. Please keep my address CC'd as I am currently not subscribed to this list. Thank you all very much for your time reading this! :) PS: I've included some potentially useful u-boot monitor output from the stock firmware at [4], as I am not sure if attachments of this size (or inline text) would be OK on-list. [0]: https://www.gigabyte.com/Enterprise/Server-Motherboard/MC12-LE0-rev-1x [1]: https://github.com/ya-mouse/bmc-ami [2]: https://github.com/ami-megarac/OSSW/blob/master/SPX-12/Kernel_amiext_ex-src/12-drivers_mtd_maps_fmh.h [3]: https://github.com/openbmc/openbmc/issues/1649 [4]: https://paste.debian.net/hidden/43509e70/ -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-20 17:32 OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions Johannes Truschnigg @ 2024-05-21 1:05 ` Andrew Jeffery 2024-05-21 17:05 ` Johannes Truschnigg 2024-07-07 9:19 ` Johannes Truschnigg 1 sibling, 1 reply; 11+ messages in thread From: Andrew Jeffery @ 2024-05-21 1:05 UTC (permalink / raw) To: Johannes Truschnigg, openbmc Hello, On Mon, 2024-05-20 at 19:32 +0200, Johannes Truschnigg wrote: > Dear list, > > I am new to this whole BMC-innards thing, and the very opposite of familiar > with embedded devices, so please bear with me on this :) :) > > The BMC implementation seems lean very much onto the AST2500 Evaluation Board > (the unchanged name pops up in a number of places in the stock firmware), so I > would guess that the OpenBMC evb-ast2500 machine should be able to get > somewhere (and even if it's just to see OpenBMC's instead of AMI's Linux start, > and then crash hard due to some subtle incompatibility :)) at least. Perhaps. While the SoC itself is obviously supported, how it's integrated into the rest of the platform will likely be very different to how things are wired up on the EVB. > My understanding is that there are two paths forward: > > 1.) To make OpenBMC's build artifacts into FMH-style BLOBs, and find a way to > feed them to the stock fw's u-boot, which would ideally result in OpenBMC being > able to boot this way. > > 2.) To replace the stock fw's u-boot release with an OpenBMC u-boot that can > load the evb-ast2500 artifacts natively, and have the BMC boot OpenBMC that > way. > > Can you please confirm that this assessment is sane, or correct me if it isn't? Yeah, that's sane. I'm not familiar with FMH so can't really comment on how feasible that approach might be, but my intuition is it's probably a bunch of pain. > > If so, I would presume option 2.) to be the easier road to travel, but I am > somewhat stumped as to how I can get OpenBMC's u-boot onto the BMC, *ideally* > (but not necessarily) in a non-destructive way (as I do not know how to or have > a way to revert to the original state without access to the bootloader or even > stock firmware running). Something to consider is using qemu to boot the proprietary firmware, and do your experiments there. That way it's non-destructive in the sense that it's all just software and you won't need to break out a flash programmer. A quick look at the BMC firmware from your [1] shows Gigabyte do their usual thing and ship the entire flash image. I downloaded the zip file, unpacked it, and booted it like ``` 0 andrew@heihei:~$ mkdir /tmp/mc12-le0 0 andrew@heihei:~$ cd !$ cd /tmp/mc12-le0 0 andrew@heihei:/tmp/mc12-le0$ unzip ~/Downloads/server_firmware_ast2500_AMI_12.x.zip Archive: /home/andrew/Downloads/server_firmware_ast2500_AMI_12.x.zip creating: 126121/ inflating: 126121/bmc_fw_update_linux.sh inflating: 126121/bmc_fw_update_uefi.nsh inflating: 126121/bmc_fw_update_win.bat inflating: 126121/BMC_Release_Note_126121.doc creating: 126121/fw/ inflating: 126121/fw/126121.bin inflating: 126121/fw/rom.ima_enc inflating: 126121/GBT_BMC_Firmware_Upgrade_User_Guide_v006_pre.pdf inflating: 126121/projects.txt creating: 126121/Tools/ inflating: 126121/Tools/gigaflash inflating: 126121/Tools/gigaflash.efi inflating: 126121/Tools/gigaflash.exe inflating: 126121/Tools/gigaflash_arm inflating: 126121/Tools/gigaflash_arm.efi inflating: 126121/Tools/gigaflash_x64 inflating: 126121/Tools/gigaflash_x64.exe creating: 126121/Tools/x64/ inflating: 126121/Tools/x64/astio.cat inflating: 126121/Tools/x64/astio.sys creating: 126121/Tools/x64_Win8/ inflating: 126121/Tools/x64_Win8/astio.cat inflating: 126121/Tools/x64_Win8/astio.Inf inflating: 126121/Tools/x64_Win8/astio.sys creating: 126121/Tools/x86/ inflating: 126121/Tools/x86/astio.sys 0 andrew@heihei:/tmp/mc12-le0$ cd 126121/fw 0 andrew@heihei:/tmp/mc12-le0/126121/fw$ binwalk 126121.bin DECIMAL HEXADECIMAL DESCRIPTION -------------------------------------------------------------------------------- 163848 0x28008 CRC32 polynomial table, little endian 213604 0x34264 CRC32 polynomial table, little endian 393216 0x60000 JFFS2 filesystem, little endian 5636096 0x560000 CramFS filesystem, little endian, size: 40951808, version 2, sorted_dirs, CRC 0x417607BC, edition 0, 27801 blocks, 6575 files 46596160 0x2C70040 uImage header, header size: 64 bytes, header CRC: 0x596F847A, created: 2024-03-12 06:08:06, image size: 2792592 bytes, Data Address: 0x80008000, Entry Point: 0x80008000, data CRC: 0xC9C2F025, OS: Linux, CPU: ARM, image type: OS Kernel Image, compression type: none, image name: "Linux-3.14.17-ami" 46596224 0x2C70080 Linux kernel ARM boot executable zImage (little-endian) 46613079 0x2C74257 gzip compressed data, maximum compression, from Unix, last modified: 1970-01-01 00:00:00 (null date) 49479680 0x2F30000 JFFS2 filesystem, little endian 50003968 0x2FB0000 CramFS filesystem, little endian, size: 5963776, version 2, sorted_dirs, CRC 0xBCBB0E63, edition 0, 1566 blocks, 131 files 0 andrew@heihei:/tmp/mc12-le0/126121/fw$ truncate -s 64M 126121.bin 0 andrew@heihei:/tmp/mc12-le0/126121/fw$ qemu-system-arm -M ast2500-evb,fmc-model=n25q512a -drive file=126121.bin,if=mtd,format=raw -nographic qemu-system-arm: warning: Aspeed iBT has no chardev backend U-Boot 2013.07 (Mar 12 2024 - 14:08:49) I2C: ready DRAM: 424 MiB eSPI Handshake complete OEM_BOARD_INIT - Start (BMC) LPC mode OEM_BOARD_INIT - End Flash: Found SPI Chip Micron/Numonyx N25Q512A(0x20ba) 2x I/O READ, NORMAL WRITE 64 MiB MMC: *** Warning - bad CRC, using default environment Un-Protected 1 sectors Erasing Flash... Erasing sector 4 ... ok. Erased 1 sectors Writing to Flash... done Protected 1 sectors Net: RTL8211E, EEECR = 0x00 RTL8211E, EEEAR = 0x00 RTL8211E, EEELPAR = 0x00 RTL8211E, LACR = 0x00 RTL8211E, LCR = 0x00 ast_eth0, ast_eth1 DRAM ECC enabled Hit any key to stop autoboot: 0 Image to be booted is 1 EMMC and EXT4 is not enabled - Cannot locate kernel file in Root Initing KCS...done Uboot waiting for firmware update to start... Uboot waiting for fwupdate to start timed out Disabling Watchdog 2 Timer AST2500EVB> ``` Not sure how productive that is, but maybe you can experiment. If you enable networking and tftp on the qemu command-line you could potentially boot your own kernels. Or you could mess with booting the OpenBMC firmware as well. If you want to do something more destructive and overwrite the firmware on the flash, then you should probably have a flash programmer handy, and/or become familiar with something like culvert (which I maintain): https://github.com/amboar/culvert Note that gigabyte do ship "gigaflash" in that zip file above, which is a closed-source application similar to culvert, that derives from Aspeed's own "socflash" tool. > > So far, I tried to put the u-boot-evb-ast2500-v2019.04+git-r0.bin artifact from > my OpenBMC build results into a memory location on the BMC using TFTP, and `go` > to that address afterwards - but that effectively reloaded the *original* > u-boot from the vendor fw. I don't know enough about the inner workings of an > AST2500 or u-boot (or any other embedded bootloader, really), but if you could > enlighten me as to what has happened there, I'd be very grateful (... maybe > u-boot does a check if it's already reloacted itself towards end of phys. > memory, and directly jump there if that is deemed the case?) > u-boot is linked at 0 and uses XIP from the SPI-NOR (which is memory- mapped at 0), so I suspect it has read out the vendor copy again. > Do you have any > hints how I could achieve what I am trying to do? Is this even feasible? Given the above, you'd have to replace u-boot on the hardware. > > I did find a Github issue[3] from 2017 which hints at others having tried > something like this, but no documented successes. > > I'd be very grateful for your input on this matter. Please keep my address CC'd > as I am currently not subscribed to this list. A potential collaborator on Gigabyte stuff is bielids. I don't have their contact details, but you can find some of our past discussion here: https://github.com/amboar/culvert/issues/51 Andrew > > Thank you all very much for your time reading this! :) > > > PS: I've included some potentially useful u-boot monitor output from the stock > firmware at [4], as I am not sure if attachments of this size (or inline text) > would be OK on-list. > > > [0]: https://www.gigabyte.com/Enterprise/Server-Motherboard/MC12-LE0-rev-1x > [1]: https://github.com/ya-mouse/bmc-ami > [2]: https://github.com/ami-megarac/OSSW/blob/master/SPX-12/Kernel_amiext_ex-src/12-drivers_mtd_maps_fmh.h > [3]: https://github.com/openbmc/openbmc/issues/1649 > [4]: https://paste.debian.net/hidden/43509e70/ > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-21 1:05 ` Andrew Jeffery @ 2024-05-21 17:05 ` Johannes Truschnigg 2024-05-22 1:29 ` Andrew Jeffery 0 siblings, 1 reply; 11+ messages in thread From: Johannes Truschnigg @ 2024-05-21 17:05 UTC (permalink / raw) To: Andrew Jeffery; +Cc: openbmc [-- Attachment #1: Type: text/plain, Size: 7930 bytes --] Hi Andrew, thanks so much for your kind response! I've come across your project (culvert) on my little quest to liberate my board's BMC already, and I'm eager to try working with it as soon as I manage to build it (which will leventually happen)! But first, I need to share the good news: Yesterday, shortly after I had sent my initial post to this list, my friend Paul on IRC supplied me with ideas and instructions to get the OpenBMC stuff to fly on the Gigabyte-supplied AST2500, and we had smashing success after a rather short while! We patched OpenBMC's u-boot to assume a CONFIG_SYS_TEXT_BASE of 0x83000000, tftp'd the resulting binary to that address, issued `go 0x83000000` in the AMI u-boot monitor, and... had successfully chainloaded the OpenBMC u-boot on the Gigabyte BMC! From there, I could load the OpenBMC FIT and went to a initrd rescue shell rather quickly. The sequence of commands was, for posterity (assuming a TFTP server on 192.168.2.123 can supply the mentioned artifacts after successful DHCPv4 autoconfig): ``` ### original/stock AMI uboot monitor: AST2500EVB> dhcp AST2500EVB> tftp 0x83000000 192.168.2.123:hacked-uboot AST2500EVB> go 0x83000000 ### chainloaded OpenBMC u-boot: ast# setenv ethact ethernet@1e680000 ast# dhcp ast# tftpboot 0x84000000 192.168.2.123:openbmc-fit-image ast# setenv bootargs console=ttyS4,115200n8 root=/dev/ram rw enable-initrd-debug-sh debug-init-sh ast# bootm 0x84000000 ``` I have provided a copy of the `hacked-uboot` file contents at [0]. Its sha256sum hexdigest reads 2937963c78f577355dc3bd15b8b8e57dcacf72fd74cd82854d1136bb26356e68 From there, I could reconfigure the BMC's Ethernet interface for my LAN, shuttle in the OpenBMC squashfs-xz image via tftp onto tmpfs, loop-mount that, and chroot into the resulting directory tree - the userland seems to work, too! :) What I am struggling with (as other plans leave me little time today unfortunately) is how to boot into this environment after manually massaging the rescue environment into the half-working state. If you have any advice on how to streamline my boot process, I'd be eager to read about it! On Tue, May 21, 2024 at 10:35:34AM +0930, Andrew Jeffery wrote: > [...] > > The BMC implementation seems lean very much onto the AST2500 Evaluation Board > > (the unchanged name pops up in a number of places in the stock firmware), so I > > would guess that the OpenBMC evb-ast2500 machine should be able to get > > somewhere (and even if it's just to see OpenBMC's instead of AMI's Linux start, > > and then crash hard due to some subtle incompatibility :)) at least. > > Perhaps. While the SoC itself is obviously supported, how it's > integrated into the rest of the platform will likely be very different > to how things are wired up on the EVB. I think I've already discovered some of the rough edges that are still coming for me and this attempt at liberating the Gigabyte BMC, as the host does not seem to want to power on without the (stock) BMC running. That, and all the other obstacles that are sure to come, I will try to tackle in time :) > > My understanding is that there are two paths forward: > > > > 1.) To make OpenBMC's build artifacts into FMH-style BLOBs, and find a way to > > feed them to the stock fw's u-boot, which would ideally result in OpenBMC being > > able to boot this way. > > > > 2.) To replace the stock fw's u-boot release with an OpenBMC u-boot that can > > load the evb-ast2500 artifacts natively, and have the BMC boot OpenBMC that > > way. > > > > Can you please confirm that this assessment is sane, or correct me if it isn't? > > Yeah, that's sane. I'm not familiar with FMH so can't really comment on > how feasible that approach might be, but my intuition is it's probably > a bunch of pain. Because of the successfull attempt at chainloading a hacked u-boot, I think we can safely discard trying to travel down path #1 and skip learning more about FMH than absolutely necessary :) > > If so, I would presume option 2.) to be the easier road to travel, but I am > > somewhat stumped as to how I can get OpenBMC's u-boot onto the BMC, *ideally* > > (but not necessarily) in a non-destructive way (as I do not know how to or have > > a way to revert to the original state without access to the bootloader or even > > stock firmware running). > > Something to consider is using qemu to boot the proprietary firmware, [...] > ast_eth0, ast_eth1 > DRAM ECC enabled > Hit any key to stop autoboot: 0 > Image to be booted is 1 > EMMC and EXT4 is not enabled - Cannot locate kernel file in Root > Initing KCS...done > Uboot waiting for firmware update to start... > Uboot waiting for fwupdate to start timed out > Disabling Watchdog 2 Timer > AST2500EVB> > ``` Thanks for this; while I've had success in booting the evb-2500ast artifacts in QEMU (as documented by OpenBMC and QEMU), I had not successfully tried with the AMI-supplied artifacts yet. This saved me a ton of time trying to figure that out! However, what I do not really understand about the situation in the emulator is that `fmh` will not list any FMH-style blocks/partitions/area in that case, whilst it will on the physical BMC. As a consequence, I cannot actually boot the stock firmware in QEMU passt the u-boot stage, as that relies on `bootfmh` and the FMH areas to be properly recognized. Maybe the munging of the image size with `truncate` changes something that the FMH format can't cope with? I am unsure, and haven't yet looked into the public FMH bits to confirm this theory's merit. > If you want to do something more destructive and overwrite the firmware > on the flash, then you should probably have a flash programmer handy, > and/or become familiar with something like culvert (which I maintain): > > https://github.com/amboar/culvert I'll be sure to try my luck with culvert as soon as I am able, and I would have tried working with it earlier more eagerly if I had not made such unexpectedly quick progress with the other ways tried ;) > Note that gigabyte do ship "gigaflash" in that zip file above, which is > a closed-source application similar to culvert, that derives from > Aspeed's own "socflash" tool. I noticed that (but did not know about its lineage), but cannot get it to actually work for me on my Linux-based host system (grml 2024.02 for amd64 at the moment), neither for updating nor for dumping the BMC ROM. Since the binary is not stripped (and seems to have some understanding of FMH and whatever funny signature business that is going on in the `rom.ima_enc` parts of the BMC FW ZIP archives), it might still be useful in other ways after all... > > [...] > > u-boot is linked at 0 and uses XIP from the SPI-NOR (which is memory- > mapped at 0), so I suspect it has read out the vendor copy again. That's what Paul and me guessed on IRC yesterday, and the first attempt at working around that seems to have been successful, as stated above! :) > [...] > A potential collaborator on Gigabyte stuff is bielids. I don't have > their contact details, but you can find some of our past discussion > here: > > https://github.com/amboar/culvert/issues/51 Thanks, I've had this issue bookmarked since the start of last weekend already, and your patient and helpful approach there actually was one of the main motivating factors for me to actually embark on this journey :) It's kind people like you who make FOSS a great world to engage in - so let me say thank you for your service to the general public! :) Have a nice day! - Johannes [0]: https://johannes.truschnigg.info/assets/AMI_ast2500_hacked_u-boot_0x8300000 -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-21 17:05 ` Johannes Truschnigg @ 2024-05-22 1:29 ` Andrew Jeffery 2024-05-27 18:09 ` Johannes Truschnigg 0 siblings, 1 reply; 11+ messages in thread From: Andrew Jeffery @ 2024-05-22 1:29 UTC (permalink / raw) To: Johannes Truschnigg; +Cc: openbmc On Tue, 2024-05-21 at 19:05 +0200, Johannes Truschnigg wrote: > Hi Andrew, > > thanks so much for your kind response! I've come across your project (culvert) > on my little quest to liberate my board's BMC already, and I'm eager to try > working with it as soon as I manage to build it (which will leventually > happen)! > > But first, I need to share the good news: Yesterday, shortly after I had sent > my initial post to this list, my friend Paul on IRC supplied me with ideas and > instructions to get the OpenBMC stuff to fly on the Gigabyte-supplied AST2500, > and we had smashing success after a rather short while! We patched OpenBMC's > u-boot to assume a CONFIG_SYS_TEXT_BASE of 0x83000000, tftp'd the resulting > binary to that address, issued > > `go 0x83000000` > > in the AMI u-boot monitor, and... had successfully chainloaded the OpenBMC > u-boot on the Gigabyte BMC! Nice! > > From there, I could load the OpenBMC FIT and went to a initrd rescue shell > rather quickly. The sequence of commands was, for posterity (assuming a TFTP > server on 192.168.2.123 can supply the mentioned artifacts after successful > DHCPv4 autoconfig): > > ``` > ### original/stock AMI uboot monitor: > AST2500EVB> dhcp > AST2500EVB> tftp 0x83000000 192.168.2.123:hacked-uboot > AST2500EVB> go 0x83000000 > > ### chainloaded OpenBMC u-boot: > ast# setenv ethact ethernet@1e680000 > ast# dhcp > ast# tftpboot 0x84000000 192.168.2.123:openbmc-fit-image > ast# setenv bootargs console=ttyS4,115200n8 root=/dev/ram rw enable-initrd-debug-sh debug-init-sh > ast# bootm 0x84000000 > ``` > > I have provided a copy of the `hacked-uboot` file contents at [0]. Its > sha256sum hexdigest reads > 2937963c78f577355dc3bd15b8b8e57dcacf72fd74cd82854d1136bb26356e68 > > From there, I could reconfigure the BMC's Ethernet interface for my LAN, > shuttle in the OpenBMC squashfs-xz image via tftp onto tmpfs, loop-mount that, > and chroot into the resulting directory tree - the userland seems to work, > too! :) > > > What I am struggling with (as other plans leave me little time today > unfortunately) is how to boot into this environment after manually massaging > the rescue environment into the half-working state. If you have any advice on > how to streamline my boot process, I'd be eager to read about it! Perhaps using `setenv` to write a u-boot script with the commands above (executed via the `run` command`) along with `saveenv` might get you some way to automation? Anyway, nice work. > > > On Tue, May 21, 2024 at 10:35:34AM +0930, Andrew Jeffery wrote: > > [...] > > > The BMC implementation seems lean very much onto the AST2500 Evaluation Board > > > (the unchanged name pops up in a number of places in the stock firmware), so I > > > would guess that the OpenBMC evb-ast2500 machine should be able to get > > > somewhere (and even if it's just to see OpenBMC's instead of AMI's Linux start, > > > and then crash hard due to some subtle incompatibility :)) at least. > > > > Perhaps. While the SoC itself is obviously supported, how it's > > integrated into the rest of the platform will likely be very different > > to how things are wired up on the EVB. > > I think I've already discovered some of the rough edges that are still coming > for me and this attempt at liberating the Gigabyte BMC, as the host does not > seem to want to power on without the (stock) BMC running. That, and all the > other obstacles that are sure to come, I will try to tackle in time :) :) > > > > If so, I would presume option 2.) to be the easier road to travel, but I am > > > somewhat stumped as to how I can get OpenBMC's u-boot onto the BMC, *ideally* > > > (but not necessarily) in a non-destructive way (as I do not know how to or have > > > a way to revert to the original state without access to the bootloader or even > > > stock firmware running). > > > > Something to consider is using qemu to boot the proprietary firmware, > [...] > > ast_eth0, ast_eth1 > > DRAM ECC enabled > > Hit any key to stop autoboot: 0 > > Image to be booted is 1 > > EMMC and EXT4 is not enabled - Cannot locate kernel file in Root > > Initing KCS...done > > Uboot waiting for firmware update to start... > > Uboot waiting for fwupdate to start timed out > > Disabling Watchdog 2 Timer > > AST2500EVB> > > ``` > > Thanks for this; while I've had success in booting the evb-2500ast artifacts > in QEMU (as documented by OpenBMC and QEMU), I had not successfully tried with > the AMI-supplied artifacts yet. This saved me a ton of time trying to figure > that out! > > However, what I do not really understand about the situation in the emulator > is that `fmh` will not list any FMH-style blocks/partitions/area in that case, > whilst it will on the physical BMC. As a consequence, I cannot actually boot > the stock firmware in QEMU passt the u-boot stage, as that relies on `bootfmh` > and the FMH areas to be properly recognized. > > Maybe the munging of the image size with `truncate` changes something that the > FMH format can't cope with? > I doubt it; the `truncate` command I used actually extends the image out to the full 64M. The image supplied by Gigabyte is a few bytes short, and qemu barfs if the flash image size does not exactly match the size of the emulated SPI-NOR part. But yeah, I'm also unsure about the FMH stuff. Perhaps something interesting is that the u-boot output under qemu talks about eMMC not being enabled: > EMMC and EXT4 is not enabled - Cannot locate kernel file in Root Not sure of what influences that output, but maybe try hooking up the eMMC as well? > > > Note that gigabyte do ship "gigaflash" in that zip file above, which is > > a closed-source application similar to culvert, that derives from > > Aspeed's own "socflash" tool. > > I noticed that (but did not know about its lineage), but cannot get it to > actually work for me on my Linux-based host system (grml 2024.02 for amd64 at > the moment), neither for updating nor for dumping the BMC ROM. > > Since the binary is not stripped (and seems to have some understanding of FMH > and whatever funny signature business that is going on in the `rom.ima_enc` > parts of the BMC FW ZIP archives), it might still be useful in other ways > after all... > Nice. > > > > [...] > > > > u-boot is linked at 0 and uses XIP from the SPI-NOR (which is memory- > > mapped at 0), so I suspect it has read out the vendor copy again. > > That's what Paul and me guessed on IRC yesterday, and the first attempt at > working around that seems to have been successful, as stated above! :) Glad we were all on the right track! > > > > [...] > > A potential collaborator on Gigabyte stuff is bielids. I don't have > > their contact details, but you can find some of our past discussion > > here: > > > > https://github.com/amboar/culvert/issues/51 > > Thanks, I've had this issue bookmarked since the start of last weekend > already, and your patient and helpful approach there actually was one of the > main motivating factors for me to actually embark on this journey :) It's kind > people like you who make FOSS a great world to engage in - so let me say thank > you for your service to the general public! :) Ah, thanks for the kind words :) Andrew ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-22 1:29 ` Andrew Jeffery @ 2024-05-27 18:09 ` Johannes Truschnigg 2024-05-28 0:26 ` Andrew Jeffery 0 siblings, 1 reply; 11+ messages in thread From: Johannes Truschnigg @ 2024-05-27 18:09 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 20865 bytes --] Hello again, dear list members! I've spent some time with my/Gigabyte's/AMI's AST2500 today, and I might have messed it up for good in the process, unwillingly. However, the result might also present a fresh opportunity - I reall do not know! :) But first things first! Because I could not find any documentation about potantial OpenBMC nfsroot support, I decided to "just try it", with unforseen consequences (anyone remember Half-Life? :)) Since `picocom` was kind enough to preserve a session log of the whole ordeal, the excerpt below should contain the gist of how I messed up: ``` U-Boot 2013.07 (Mar 12 2024 - 14:08:49) I2C: ready DRAM: 424 MiB eSPI Handshake complete OEM_BOARD_INIT - Start (BMC) LPC mode OEM_BOARD_INIT - End Flash: Found SPI Chip Micron/Numonyx N25Q512A(0x20ba) 2x I/O READ, NORMAL WRITE 64 MiB MMC: Net: RTL8211E, EEECR = 0x06 RTL8211E, EEEAR = 0x00 RTL8211E, EEELPAR = 0x06 RTL8211E, LACR = 0xc1 RTL8211E, LCR = 0x9742 ast_eth0, ast_eth1 DRAM ECC enabled Hit any key to stop autoboot: 0 AST2500EVB>set autoload no AST2500EVB>dhcp BOOTP broadcast 1 [...] BOOTP broadcast 25 Retry count exceeded; starting again BOOTP broadcast 1 BOOTP broadcast 2 DHCP client bound to address 172.18.3.62 AST2500EVB>tftp 0x83000000 172.18.3.199:uboot Using ast_eth1 device TFTP from server 172.18.3.199; our IP address is 172.18.3.62 Filename 'uboot'. Load address: 0x83000000 Loading: ################################################################# ######### 366.7 KiB done Bytes transferred = 375501 (5bacd hex) AST2500EVB>go 0x83000000 ## Starting application at 0x83000000 ... U-Boot 2019.04 (Mar 25 2024 - 04:43:19 +0000) SOC : AST2500-A2 RST : WDT1 RST : WDT3 - Boot LPC Mode : SIO:Disable Eth : MAC0: RMII/NCSI, , MAC1: RGMII, Model: AST2500 EVB DRAM: 424 MiB (capacity:512 MiB, VGA:32 MiB, ECC:on, ECC size:424 MiB) MMC: sdhci_slot0@100: 0, sdhci_slot1@200: 1 Loading Environment from SPI Flash... SF: Detected n25q512ax3 with page size 256 Bytes, erase size 4 KiB, total 64 MiB *** Warning - bad CRC, using default environment In: serial@1e784000 Out: serial@1e784000 Err: serial@1e784000 Net: Warning: ethernet@1e660000 (eth0) using random MAC address - 56:4b:8e:16:48:eb eth0: ethernet@1e660000 Warning: ethernet@1e680000 (eth1) using random MAC address - 32:f3:07:0a:d2:ad , eth1: ethernet@1e680000 Hit any key to stop autoboot: 0 ast# setenv ethact ethernet@1e680000 ast# setenv autoload no ast# dhcp ethernet@1e680000: link up, 1000 Mbps full-duplex mac:32:f3:07:0a:d2:ad BOOTP broadcast 1 BOOTP broadcast 2 BOOTP broadcast 3 BOOTP broadcast 4 DHCP client bound to address 172.18.3.83 (3007 ms) ast# tftpboot 0x84000000 172.18.3.199:fit.bin ethernet@1e680000: link up, 1000 Mbps full-duplex mac:32:f3:07:0a:d2:ad Using ethernet@1e680000 device TFTP from server 172.18.3.199; our IP address is 172.18.3.83 Filename 'fit.bin'. Load address: 0x84000000 Loading: ################################################################# ####################################### 5.6 MiB/s done Bytes transferred = 4375688 (42c488 hex) ast# setenv bootargs console=ttyS4,115200n8 root=/dev/nfs rootfstype=nfs ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337 ast# bootm 0x84000000 ## Loading kernel from FIT Image at 84000000 ... Using 'conf-aspeed-ast2500-evb.dtb' configuration Trying 'kernel-1' kernel subimage Description: Linux kernel Type: Kernel Image Compression: uncompressed Data Start: 0x84000118 Data Size: 3257440 Bytes = 3.1 MiB Architecture: ARM OS: Linux Load Address: 0x80001000 Entry Point: 0x80001000 Hash algo: sha256 Hash value: 0c874e335b61b661a7f1d638258ee1652728f021921b74bdcbad2bbb4ceef3f9 Verifying Hash Integrity ... sha256+ OK ## Loading ramdisk from FIT Image at 84000000 ... Using 'conf-aspeed-ast2500-evb.dtb' configuration Trying 'ramdisk-1' ramdisk subimage Description: obmc-phosphor-initramfs Type: RAMDisk Image Compression: uncompressed Data Start: 0x84322138 Data Size: 1088964 Bytes = 1 MiB Architecture: ARM OS: Linux Load Address: unavailable Entry Point: unavailable Hash algo: sha256 Hash value: 94cc4badfdacec0b053b68954c69c55af5e50b3d8bc15b2852005fd6c84c238f Verifying Hash Integrity ... sha256+ OK ## Loading fdt from FIT Image at 84000000 ... Using 'conf-aspeed-ast2500-evb.dtb' configuration Trying 'fdt-aspeed-ast2500-evb.dtb' fdt subimage Description: Flattened Device Tree blob Type: Flat Device Tree Compression: uncompressed Data Start: 0x8431b68c Data Size: 27103 Bytes = 26.5 KiB Architecture: ARM Hash algo: sha256 Hash value: ca0066dfb087e1b1e022dec0c9aef3ac1c1ac350a49f5959e056acf64383b637 Verifying Hash Integrity ... sha256+ OK Booting using the fdt blob at 0x8431b68c Loading Kernel Image ... OK Loading Ramdisk to 8fef6000, end 8ffffdc4 ... OK Loading Device Tree to 8feec000, end 8fef59de ... OK Starting kernel ... [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 6.6.30-f013890-00167-gf013890407d8 (oe-user@oe-host) (arm-openbmc-linux-gnueabi-gcc (GCC) 13.2.0, GNU ld (GNU Binutils) 2.42.0.20240216) #1 Mon May 13 00:55:18 UTC 2024 [ 0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache [ 0.000000] OF: fdt: Machine model: AST2500 EVB [ 0.000000] Memory policy: Data cache writeback [ 0.000000] Reserved memory: created CMA memory pool at 0x99000000, size 16 MiB [ 0.000000] OF: reserved mem: initialized node framebuffer, compatible id shared-dma-pool [ 0.000000] OF: reserved mem: 0x99000000..0x99ffffff (16384 KiB) map reusable framebuffer [ 0.000000] cma: Reserved 16 MiB at 0x98000000 on node -1 [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000080000000-0x000000009a7fffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000080000000-0x000000009a7fffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000009a7fffff] [ 0.000000] Kernel command line: console=ttyS4,115200n8 root=/dev/nfs rootfstype=nfs ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337 [ 0.000000] Unknown kernel command line parameters "ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337", will be passed to user space. [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear) [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 107696 [ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off [ 0.000000] Memory: 385236K/434176K available (7168K kernel code, 686K rwdata, 1664K rodata, 1024K init, 141K bss, 16172K reserved, 32768K cma-reserved) [ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 [ 0.000000] ftrace: allocating 24994 entries in 49 pages [ 0.000000] ftrace: allocated 49 pages with 3 groups [ 0.000000] trace event string verifier disabled [ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16 [ 0.000000] i2c controller registered, irq 17 [ 0.000000] clocksource: FTTMR010-TIMER2: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 77222644334 ns [ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 86767015915ns [ 0.000079] Switching to timer-based delay loop, resolution 40ns [ 0.001275] Calibrating delay loop (skipped), value calculated using timer frequency.. 49.50 BogoMIPS (lpj=247500) [ 0.001340] CPU: Testing write buffer coherency: ok [ 0.001456] pid_max: default: 32768 minimum: 301 [ 0.011962] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.012029] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.027599] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. [ 0.028143] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. [ 0.028703] Setting up static identity map for 0x80100000 - 0x80100038 [ 0.030465] ASPEED AST2500 rev A2 (04030303) [ 0.032008] devtmpfs: initialized [ 0.053913] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.053996] futex hash table entries: 256 (order: -1, 3072 bytes, linear) [ 0.060628] pinctrl core: initialized pinctrl subsystem [ 0.065337] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 0.067525] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 0.070964] hw-breakpoint: found 6 breakpoint and 1 watchpoint registers. [ 0.071009] hw-breakpoint: maximum watchpoint size is 4 bytes. [ 0.132580] mc: Linux media interface: v0.10 [ 0.132897] videodev: Linux video capture interface: v2.00 [ 0.133060] pps_core: LinuxPPS API ver. 1 registered [ 0.133085] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> [ 0.133161] PTP clock support registered [ 0.142848] clocksource: Switched to clocksource FTTMR010-TIMER2 [ 0.174185] NET: Registered PF_INET protocol family [ 0.174819] IP idents hash table entries: 8192 (order: 4, 65536 bytes, linear) [ 0.177558] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear) [ 0.177651] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.177714] TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear) [ 0.177806] TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear) [ 0.178014] TCP: Hash tables configured (established 4096 bind 4096) [ 0.178317] UDP hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.178411] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear) [ 0.179584] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 0.186070] Unpacking initramfs... [ 0.213300] workingset: timestamp_bits=30 max_order=17 bucket_order=0 [ 0.214001] squashfs: version 4.0 (2009/01/31) Phillip Lougher [ 0.214060] jffs2: version 2.2. (SUMMARY) © 2001-2006 Red Hat, Inc. [ 0.263220] Serial: 8250/16550 driver, 6 ports, IRQ sharing enabled [ 0.273969] printk: console [ttyS4] disabled [ 0.275132] 1e784000.serial: ttyS4 at MMIO 0x1e784000 (irq = 20, base_baud = 1500000) is a 16550A [ 0.275268] printk: console [ttyS4] enabled [ 0.935649] timeriomem_rng 1e6e2078.hwrng: 32bits from 0x(ptrval) @ 1us [ 0.943808] aspeed_gfx 1e6e6000.display: assigned reserved memory node framebuffer [ 0.967762] random: crng init done [ 0.984563] [drm] Initialized aspeed-gfx-drm 1.0.0 20180319 for 1e6e6000.display on minor 0 [ 1.007161] aspeed_gfx 1e6e6000.display: [drm] fb0: aspeed-gfx-drmd frame buffer device [ 1.055838] loop: module loaded [ 1.141860] spi-nor spi0.0: mt25ql512a (65536 Kbytes) [ 1.274576] spi-aspeed-smc 1e620000.spi: CE0 read buswidth:2 [0x203c0641] [ 1.346136] 5 fixed-partitions partitions found on MTD device bmc [ 1.352312] Creating 5 MTD partitions on "bmc": [ 1.356984] 0x000000000000-0x000000060000 : "u-boot" [ 1.385686] 0x000000060000-0x000000080000 : "u-boot-env" [ 1.406591] 0x000000080000-0x0000004c0000 : "kernel" [ 1.414849] 0x0000004c0000-0x000001c00000 : "rofs" [ 1.436205] 0x000001c00000-0x000002000000 : "rwfs" [ 1.448668] spi-nor spi1.0: unrecognized JEDEC id bytes: 00 03 00 00 00 00 [ 1.463133] ftgmac100 1e660000.ethernet: Error applying setting, reverse things back [ 1.471391] ftgmac100 1e660000.ethernet: Read MAC address 56:4b:8e:16:48:eb from chip [ 1.510983] Generic PHY 1e660000.ethernet--1:00: attached PHY driver (mii_bus:phy_addr=1e660000.ethernet--1:00, irq=POLL) [ 1.523568] ftgmac100 1e660000.ethernet eth0: irq 22, mapped at 96ea64c4 [ 1.531266] ftgmac100 1e680000.ethernet: Read MAC address 32:f3:07:0a:d2:ad from chip [ 1.570263] RTL8211E Gigabit Ethernet 1e680000.ethernet--1:00: attached PHY driver (mii_bus:phy_addr=1e680000.ethernet--1:00, irq=POLL) [ 1.583921] ftgmac100 1e680000.ethernet eth1: irq 23, mapped at ec62a3ab [ 1.746308] aspeed_vhub 1e6a0000.usb-vhub: Initialized virtual hub in USB2 mode [ 1.754377] Mass Storage Function, version: 2009/09/11 [ 1.759586] LUN: removable file: (no medium) [ 1.764058] no file given for LUN0 [ 1.767569] udc 1e6a0000.usb-vhub:p1: failed to start g_mass_storage: -22 [ 1.774490] g_mass_storage: probe of gadget.0 failed with error -22 [ 1.780961] Mass Storage Function, version: 2009/09/11 [ 1.786234] LUN: removable file: (no medium) [ 1.790615] no file given for LUN0 [ 1.794196] udc 1e6a0000.usb-vhub:p2: failed to start g_mass_storage: -22 [ 1.801044] g_mass_storage: probe of gadget.1 failed with error -22 [ 1.807565] Mass Storage Function, version: 2009/09/11 [ 1.812765] LUN: removable file: (no medium) [ 1.817213] no file given for LUN0 [ 1.820709] udc 1e6a0000.usb-vhub:p3: failed to start g_mass_storage: -22 [ 1.827628] g_mass_storage: probe of gadget.2 failed with error -22 [ 1.834157] Mass Storage Function, version: 2009/09/11 [ 1.839354] LUN: removable file: (no medium) [ 1.843823] no file given for LUN0 [ 1.847327] udc 1e6a0000.usb-vhub:p4: failed to start g_mass_storage: -22 [ 1.854243] g_mass_storage: probe of gadget.3 failed with error -22 [ 1.860710] Mass Storage Function, version: 2009/09/11 [ 1.865982] LUN: removable file: (no medium) [ 1.870363] no file given for LUN0 [ 1.873934] udc 1e6a0000.usb-vhub:p5: failed to start g_mass_storage: -22 [ 1.880778] g_mass_storage: probe of gadget.4 failed with error -22 [ 1.887290] UDC core: g_mass_storage: couldn't find an available UDC [ 1.894510] i2c_dev: i2c /dev entries driver [ 2.009530] aspeed-i2c-bus 1e78a100.i2c-bus: i2c bus 3 registered, irq 25 [ 2.033760] aspeed-i2c-bus 1e78a300.i2c-bus: i2c bus 7 registered, irq 26 [ 2.041576] Driver for 1-wire Dallas network protocol. [ 2.080643] NET: Registered PF_INET6 protocol family [ 2.114646] Segment Routing with IPv6 [ 2.118479] In-situ OAM (IOAM) with IPv6 [ 2.122989] NET: Registered PF_PACKET protocol family [ 2.128116] 8021q: 802.1Q VLAN Support v1.8 [ 2.173478] printk: console [netcon0] enabled [ 2.177927] netconsole: network logging started [ 2.183614] clk: Disabling unused clocks [ 2.368191] Freeing initrd memory: 1064K [ 2.378419] Freeing unused kernel image (initmem) memory: 1024K [ 2.386528] Checked W+X mappings: passed, no W+X pages found [ 2.392272] Run /init as init process rofs = mtd4 squashfs rwfs = mtd5 jffs2 mount: mounting /dev/mtdblock4 on run/initramfs/ro failed: Invalid argument [ 3.427189] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x49f3 instead [ 3.436958] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0xe73d instead [... XXX *many* more of those XXX at various offsets...] [ 10.907145] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x003f0024: 0x68b6 instead [ 10.916740] jffs2: Further such events for this erase block will not be printed [ 10.934585] jffs2: Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes [ 10.943154] jffs2: empty_blocks 0, bad_blocks 0, c->nr_blocks 64 mount: mounting /dev/mtdblock5 on run/initramfs/rw failed: Input/output error Mounting read-write /dev/mtdblock5 filesystem failed. Please fix and run mount /dev/mtdblock5 run/initramfs/rw -t jffs2 -o rw or perform a factory reset with the clean-rwfs-filesystem option. Fatal error, tri[ 10.990385] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 [ 10.999145] CPU: 0 PID: 1 Comm: init Not tainted 6.6.30-f013890-00167-gf013890407d8 #1 [ 11.007102] Hardware name: Generic DT based system [ 11.011934] unwind_backtrace from show_stack+0x18/0x1c [ 11.017265] show_stack from dump_stack_lvl+0x24/0x2c [ 11.022404] dump_stack_lvl from panic+0xf4/0x30c [ 11.027180] panic from do_exit+0x8a8/0x8d0 [ 11.031441] do_exit from do_group_exit+0x40/0x84 [ 11.036209] do_group_exit from __wake_up_parent+0x0/0x1c U-Boot 2013.07 (Mar 12 2024 - 14:08:49) I2C: ready DRAM: 424 MiB eSPI Handshake complete OEM_BOARD_INIT - Start (BMC) LPC mode OEM_BOARD_INIT - End Flash: Found SPI Chip Micron/Numonyx N25Q512A(0x20ba) 2x I/O READ, NORMAL WRITE 64 MiB MMC: *** Warning - bad CRC, using default environment Un-Protected 1 sectors Erasing Flash... Erasing sector 4 ... ok. Erased 1 sectors Writing to Flash... done Protected 1 sectors Net: RTL8211E, EEECR = 0x06 RTL8211E, EEEAR = 0x00 RTL8211E, EEELPAR = 0x06 RTL8211E, LACR = 0xc1 RTL8211E, LCR = 0x9742 ast_eth0, ast_eth1 DRAM ECC enabled Hit any key to stop autoboot: 0 AST2500EVB> ``` So afaiui my attempt misfired, the kernel panic'd, reset the system, and then... u-boot kinda decided to erase flash sector 4 (why?), which seems to have contained something important for (the original u-boot from Gigabyte on SPI flash) to boot. I did not realize that and cut power, waiting for a few secs before powering up again... only to notice that my UART serial console was staying dark this time. Meanwhile, the BMC NIC LEDs blink in a strange loop that hints at some restart procedure that is repeated ad nauseam. To my luck, however, the host itself still willingly boots into the live system that I had set up before, but with a subtle difference once it's loaded: The AST2500 seems to be in a state much more amenable to Andrew's `culvert` utility now, which reports its neighborhood BMC like this now: ``` root@grml ~ # /tmp/culvert probe [*] failed to initialise devmem bridge: -1 [*] Accessing the BMC's AHB via the p2a bridge debug: Permissive Debug UART port: UART5 xdma: Restricted BMC: Disabled VGA: Enabled XDMA on VGA: Enabled XDMA is constrained: Yes p2a: Permissive BMC: Disabled VGA: Enabled MMIO on VGA: Enabled [0x00000000 - 0x0fffffff] Firmware: Writable [0x10000000 - 0x1fffffff] SoC IO: Writable [0x20000000 - 0x2fffffff] BMC Flash: Writable [0x30000000 - 0x3fffffff] Host Flash: Writable [0x40000000 - 0x5fffffff] Reserved: Writable [0x60000000 - 0x7fffffff] LPC Host: Writable [0x80000000 - 0xffffffff] DRAM: Writable ilpc: Permissive SuperIO address: 0x2e ``` (Before my mishap, whenever I tried to run that command, all but `ilpc` yielded nothing, and even `ilpc` reported "Restricted" - none of which I know how to interpret at all, by the way! :D) So I guess I have at least six questions now: 1.) What happened when the kernel called it quits, u-boot reloaded and decided to format some of its flash? 2.) Can I influence this behavior on my (spare) board before I try again? 3.) Can I restore the original firmware/SPI content on my board by any means from the now running host OS? If so, what way would you suggest I try first? 4.) Does having `culvert` have this new level of access have any new advantages or open possibilities that I should be aware of? 5.) Suppose I can restore the BMC's original SPI content and behavior - what's a recommended way to have the TFPT'd kernel boot into an OpenBMC rootfs *without* having it store on the BMC's main storage/overwriting SPI? 6.) Assuming this cannot be recovered in software - what are my chances of identifying the SPI flash on my board as such, and re-writing its contents using an affordable SPI programmer solution, given that I've never done anything like this with hardware before? :^) If any of you could answer me any of these, I would be very grateful indeed. Thanks a lot for your time reading this far! :) -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-27 18:09 ` Johannes Truschnigg @ 2024-05-28 0:26 ` Andrew Jeffery 2024-05-28 18:37 ` Johannes Truschnigg 0 siblings, 1 reply; 11+ messages in thread From: Andrew Jeffery @ 2024-05-28 0:26 UTC (permalink / raw) To: Johannes Truschnigg, openbmc On Mon, 2024-05-27 at 20:09 +0200, Johannes Truschnigg wrote: > Hello again, dear list members! > > I've spent some time with my/Gigabyte's/AMI's AST2500 today, and I might have > messed it up for good in the process, unwillingly. However, the result might > also present a fresh opportunity - I reall do not know! :) But first things > first! > > Because I could not find any documentation about potantial OpenBMC nfsroot > support, I decided to "just try it", with unforseen consequences (anyone > remember Half-Life? :)) Since `picocom` was kind enough to preserve a session > log of the whole ordeal, the excerpt below should contain the gist of how I > messed up: > > > ``` > > U-Boot 2013.07 (Mar 12 2024 - 14:08:49) > > I2C: ready > DRAM: 424 MiB > eSPI Handshake complete > OEM_BOARD_INIT - Start (BMC) > LPC mode > OEM_BOARD_INIT - End > Flash: Found SPI Chip Micron/Numonyx N25Q512A(0x20ba) 2x I/O READ, NORMAL WRITE > 64 MiB > MMC: > Net: RTL8211E, EEECR = 0x06 > RTL8211E, EEEAR = 0x00 > RTL8211E, EEELPAR = 0x06 > RTL8211E, LACR = 0xc1 > RTL8211E, LCR = 0x9742 > ast_eth0, ast_eth1 > DRAM ECC enabled > Hit any key to stop autoboot: 0 > AST2500EVB>set autoload no > AST2500EVB>dhcp > BOOTP broadcast 1 > [...] > BOOTP broadcast 25 > > Retry count exceeded; starting again > BOOTP broadcast 1 > BOOTP broadcast 2 > DHCP client bound to address 172.18.3.62 > AST2500EVB>tftp 0x83000000 172.18.3.199:uboot > Using ast_eth1 device > TFTP from server 172.18.3.199; our IP address is 172.18.3.62 > Filename 'uboot'. > Load address: 0x83000000 > Loading: ################################################################# > ######### > 366.7 KiB > done > Bytes transferred = 375501 (5bacd hex) > AST2500EVB>go 0x83000000 > ## Starting application at 0x83000000 ... > > > U-Boot 2019.04 (Mar 25 2024 - 04:43:19 +0000) > > SOC : AST2500-A2 > RST : WDT1 > RST : WDT3 - Boot > LPC Mode : SIO:Disable > Eth : MAC0: RMII/NCSI, , MAC1: RGMII, > Model: AST2500 EVB > DRAM: 424 MiB (capacity:512 MiB, VGA:32 MiB, ECC:on, ECC size:424 MiB) > MMC: sdhci_slot0@100: 0, sdhci_slot1@200: 1 > Loading Environment from SPI Flash... SF: Detected n25q512ax3 with page size 256 Bytes, erase size 4 KiB, total 64 MiB > *** Warning - bad CRC, using default environment > > In: serial@1e784000 > Out: serial@1e784000 > Err: serial@1e784000 > Net: > Warning: ethernet@1e660000 (eth0) using random MAC address - 56:4b:8e:16:48:eb > eth0: ethernet@1e660000 > Warning: ethernet@1e680000 (eth1) using random MAC address - 32:f3:07:0a:d2:ad > , eth1: ethernet@1e680000 > Hit any key to stop autoboot: 0 > ast# setenv ethact ethernet@1e680000 > ast# setenv autoload no > ast# dhcp > ethernet@1e680000: link up, 1000 Mbps full-duplex mac:32:f3:07:0a:d2:ad > BOOTP broadcast 1 > BOOTP broadcast 2 > BOOTP broadcast 3 > BOOTP broadcast 4 > DHCP client bound to address 172.18.3.83 (3007 ms) > ast# tftpboot 0x84000000 172.18.3.199:fit.bin > ethernet@1e680000: link up, 1000 Mbps full-duplex mac:32:f3:07:0a:d2:ad > Using ethernet@1e680000 device > TFTP from server 172.18.3.199; our IP address is 172.18.3.83 > Filename 'fit.bin'. > Load address: 0x84000000 > Loading: ################################################################# > ####################################### > 5.6 MiB/s > done > Bytes transferred = 4375688 (42c488 hex) > ast# setenv bootargs console=ttyS4,115200n8 root=/dev/nfs rootfstype=nfs ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337 > ast# bootm 0x84000000 > ## Loading kernel from FIT Image at 84000000 ... > Using 'conf-aspeed-ast2500-evb.dtb' configuration > Trying 'kernel-1' kernel subimage > Description: Linux kernel > Type: Kernel Image > Compression: uncompressed > Data Start: 0x84000118 > Data Size: 3257440 Bytes = 3.1 MiB > Architecture: ARM > OS: Linux > Load Address: 0x80001000 > Entry Point: 0x80001000 > Hash algo: sha256 > Hash value: 0c874e335b61b661a7f1d638258ee1652728f021921b74bdcbad2bbb4ceef3f9 > Verifying Hash Integrity ... sha256+ OK > ## Loading ramdisk from FIT Image at 84000000 ... > Using 'conf-aspeed-ast2500-evb.dtb' configuration > Trying 'ramdisk-1' ramdisk subimage > Description: obmc-phosphor-initramfs > Type: RAMDisk Image > Compression: uncompressed > Data Start: 0x84322138 > Data Size: 1088964 Bytes = 1 MiB > Architecture: ARM > OS: Linux > Load Address: unavailable > Entry Point: unavailable > Hash algo: sha256 > Hash value: 94cc4badfdacec0b053b68954c69c55af5e50b3d8bc15b2852005fd6c84c238f > Verifying Hash Integrity ... sha256+ OK > ## Loading fdt from FIT Image at 84000000 ... > Using 'conf-aspeed-ast2500-evb.dtb' configuration > Trying 'fdt-aspeed-ast2500-evb.dtb' fdt subimage > Description: Flattened Device Tree blob > Type: Flat Device Tree > Compression: uncompressed > Data Start: 0x8431b68c > Data Size: 27103 Bytes = 26.5 KiB > Architecture: ARM > Hash algo: sha256 > Hash value: ca0066dfb087e1b1e022dec0c9aef3ac1c1ac350a49f5959e056acf64383b637 > Verifying Hash Integrity ... sha256+ OK > Booting using the fdt blob at 0x8431b68c > Loading Kernel Image ... OK > Loading Ramdisk to 8fef6000, end 8ffffdc4 ... OK > Loading Device Tree to 8feec000, end 8fef59de ... OK > > Starting kernel ... > > [ 0.000000] Booting Linux on physical CPU 0x0 > [ 0.000000] Linux version 6.6.30-f013890-00167-gf013890407d8 (oe-user@oe-host) (arm-openbmc-linux-gnueabi-gcc (GCC) 13.2.0, GNU ld (GNU Binutils) 2.42.0.20240216) #1 Mon May 13 00:55:18 UTC 2024 > [ 0.000000] CPU: ARMv6-compatible processor [410fb767] revision 7 (ARMv7), cr=00c5387d > [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT nonaliasing instruction cache > [ 0.000000] OF: fdt: Machine model: AST2500 EVB > [ 0.000000] Memory policy: Data cache writeback > [ 0.000000] Reserved memory: created CMA memory pool at 0x99000000, size 16 MiB > [ 0.000000] OF: reserved mem: initialized node framebuffer, compatible id shared-dma-pool > [ 0.000000] OF: reserved mem: 0x99000000..0x99ffffff (16384 KiB) map reusable framebuffer > [ 0.000000] cma: Reserved 16 MiB at 0x98000000 on node -1 > [ 0.000000] Zone ranges: > [ 0.000000] Normal [mem 0x0000000080000000-0x000000009a7fffff] > [ 0.000000] Movable zone start for each node > [ 0.000000] Early memory node ranges > [ 0.000000] node 0: [mem 0x0000000080000000-0x000000009a7fffff] > [ 0.000000] Initmem setup node 0 [mem 0x0000000080000000-0x000000009a7fffff] > [ 0.000000] Kernel command line: console=ttyS4,115200n8 root=/dev/nfs rootfstype=nfs ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337 > [ 0.000000] Unknown kernel command line parameters "ip=172.18.3.173:172.18.3.138:172.18.3.254:255.255.255.0:bmctest:eth1:off:172.18.3.254::172.18.3.254 nfsroot=172.18.3.138:/,mountport=13337", will be passed to user space. > [ 0.000000] Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear) > [ 0.000000] Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear) > [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 107696 > [ 0.000000] mem auto-init: stack:all(zero), heap alloc:off, heap free:off > [ 0.000000] Memory: 385236K/434176K available (7168K kernel code, 686K rwdata, 1664K rodata, 1024K init, 141K bss, 16172K reserved, 32768K cma-reserved) > [ 0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1 > [ 0.000000] ftrace: allocating 24994 entries in 49 pages > [ 0.000000] ftrace: allocated 49 pages with 3 groups > [ 0.000000] trace event string verifier disabled > [ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16 > [ 0.000000] i2c controller registered, irq 17 > [ 0.000000] clocksource: FTTMR010-TIMER2: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 77222644334 ns > [ 0.000004] sched_clock: 32 bits at 25MHz, resolution 40ns, wraps every 86767015915ns > [ 0.000079] Switching to timer-based delay loop, resolution 40ns > [ 0.001275] Calibrating delay loop (skipped), value calculated using timer frequency.. 49.50 BogoMIPS (lpj=247500) > [ 0.001340] CPU: Testing write buffer coherency: ok > [ 0.001456] pid_max: default: 32768 minimum: 301 > [ 0.011962] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) > [ 0.012029] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear) > [ 0.027599] RCU Tasks Rude: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. > [ 0.028143] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1. > [ 0.028703] Setting up static identity map for 0x80100000 - 0x80100038 > [ 0.030465] ASPEED AST2500 rev A2 (04030303) > [ 0.032008] devtmpfs: initialized > [ 0.053913] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns > [ 0.053996] futex hash table entries: 256 (order: -1, 3072 bytes, linear) > [ 0.060628] pinctrl core: initialized pinctrl subsystem > [ 0.065337] NET: Registered PF_NETLINK/PF_ROUTE protocol family > [ 0.067525] DMA: preallocated 256 KiB pool for atomic coherent allocations > [ 0.070964] hw-breakpoint: found 6 breakpoint and 1 watchpoint registers. > [ 0.071009] hw-breakpoint: maximum watchpoint size is 4 bytes. > [ 0.132580] mc: Linux media interface: v0.10 > [ 0.132897] videodev: Linux video capture interface: v2.00 > [ 0.133060] pps_core: LinuxPPS API ver. 1 registered > [ 0.133085] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> > [ 0.133161] PTP clock support registered > [ 0.142848] clocksource: Switched to clocksource FTTMR010-TIMER2 > [ 0.174185] NET: Registered PF_INET protocol family > [ 0.174819] IP idents hash table entries: 8192 (order: 4, 65536 bytes, linear) > [ 0.177558] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear) > [ 0.177651] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) > [ 0.177714] TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear) > [ 0.177806] TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear) > [ 0.178014] TCP: Hash tables configured (established 4096 bind 4096) > [ 0.178317] UDP hash table entries: 256 (order: 0, 4096 bytes, linear) > [ 0.178411] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear) > [ 0.179584] NET: Registered PF_UNIX/PF_LOCAL protocol family > [ 0.186070] Unpacking initramfs... > [ 0.213300] workingset: timestamp_bits=30 max_order=17 bucket_order=0 > [ 0.214001] squashfs: version 4.0 (2009/01/31) Phillip Lougher > [ 0.214060] jffs2: version 2.2. (SUMMARY) © 2001-2006 Red Hat, Inc. > [ 0.263220] Serial: 8250/16550 driver, 6 ports, IRQ sharing enabled > [ 0.273969] printk: console [ttyS4] disabled > [ 0.275132] 1e784000.serial: ttyS4 at MMIO 0x1e784000 (irq = 20, base_baud = 1500000) is a 16550A > [ 0.275268] printk: console [ttyS4] enabled > [ 0.935649] timeriomem_rng 1e6e2078.hwrng: 32bits from 0x(ptrval) @ 1us > [ 0.943808] aspeed_gfx 1e6e6000.display: assigned reserved memory node framebuffer > [ 0.967762] random: crng init done > [ 0.984563] [drm] Initialized aspeed-gfx-drm 1.0.0 20180319 for 1e6e6000.display on minor 0 > [ 1.007161] aspeed_gfx 1e6e6000.display: [drm] fb0: aspeed-gfx-drmd frame buffer device > [ 1.055838] loop: module loaded > [ 1.141860] spi-nor spi0.0: mt25ql512a (65536 Kbytes) > [ 1.274576] spi-aspeed-smc 1e620000.spi: CE0 read buswidth:2 [0x203c0641] > [ 1.346136] 5 fixed-partitions partitions found on MTD device bmc > [ 1.352312] Creating 5 MTD partitions on "bmc": > [ 1.356984] 0x000000000000-0x000000060000 : "u-boot" > [ 1.385686] 0x000000060000-0x000000080000 : "u-boot-env" > [ 1.406591] 0x000000080000-0x0000004c0000 : "kernel" > [ 1.414849] 0x0000004c0000-0x000001c00000 : "rofs" > [ 1.436205] 0x000001c00000-0x000002000000 : "rwfs" > [ 1.448668] spi-nor spi1.0: unrecognized JEDEC id bytes: 00 03 00 00 00 00 > [ 1.463133] ftgmac100 1e660000.ethernet: Error applying setting, reverse things back > [ 1.471391] ftgmac100 1e660000.ethernet: Read MAC address 56:4b:8e:16:48:eb from chip > [ 1.510983] Generic PHY 1e660000.ethernet--1:00: attached PHY driver (mii_bus:phy_addr=1e660000.ethernet--1:00, irq=POLL) > [ 1.523568] ftgmac100 1e660000.ethernet eth0: irq 22, mapped at 96ea64c4 > [ 1.531266] ftgmac100 1e680000.ethernet: Read MAC address 32:f3:07:0a:d2:ad from chip > [ 1.570263] RTL8211E Gigabit Ethernet 1e680000.ethernet--1:00: attached PHY driver (mii_bus:phy_addr=1e680000.ethernet--1:00, irq=POLL) > [ 1.583921] ftgmac100 1e680000.ethernet eth1: irq 23, mapped at ec62a3ab > [ 1.746308] aspeed_vhub 1e6a0000.usb-vhub: Initialized virtual hub in USB2 mode > [ 1.754377] Mass Storage Function, version: 2009/09/11 > [ 1.759586] LUN: removable file: (no medium) > [ 1.764058] no file given for LUN0 > [ 1.767569] udc 1e6a0000.usb-vhub:p1: failed to start g_mass_storage: -22 > [ 1.774490] g_mass_storage: probe of gadget.0 failed with error -22 > [ 1.780961] Mass Storage Function, version: 2009/09/11 > [ 1.786234] LUN: removable file: (no medium) > [ 1.790615] no file given for LUN0 > [ 1.794196] udc 1e6a0000.usb-vhub:p2: failed to start g_mass_storage: -22 > [ 1.801044] g_mass_storage: probe of gadget.1 failed with error -22 > [ 1.807565] Mass Storage Function, version: 2009/09/11 > [ 1.812765] LUN: removable file: (no medium) > [ 1.817213] no file given for LUN0 > [ 1.820709] udc 1e6a0000.usb-vhub:p3: failed to start g_mass_storage: -22 > [ 1.827628] g_mass_storage: probe of gadget.2 failed with error -22 > [ 1.834157] Mass Storage Function, version: 2009/09/11 > [ 1.839354] LUN: removable file: (no medium) > [ 1.843823] no file given for LUN0 > [ 1.847327] udc 1e6a0000.usb-vhub:p4: failed to start g_mass_storage: -22 > [ 1.854243] g_mass_storage: probe of gadget.3 failed with error -22 > [ 1.860710] Mass Storage Function, version: 2009/09/11 > [ 1.865982] LUN: removable file: (no medium) > [ 1.870363] no file given for LUN0 > [ 1.873934] udc 1e6a0000.usb-vhub:p5: failed to start g_mass_storage: -22 > [ 1.880778] g_mass_storage: probe of gadget.4 failed with error -22 > [ 1.887290] UDC core: g_mass_storage: couldn't find an available UDC > [ 1.894510] i2c_dev: i2c /dev entries driver > [ 2.009530] aspeed-i2c-bus 1e78a100.i2c-bus: i2c bus 3 registered, irq 25 > [ 2.033760] aspeed-i2c-bus 1e78a300.i2c-bus: i2c bus 7 registered, irq 26 > [ 2.041576] Driver for 1-wire Dallas network protocol. > [ 2.080643] NET: Registered PF_INET6 protocol family > [ 2.114646] Segment Routing with IPv6 > [ 2.118479] In-situ OAM (IOAM) with IPv6 > [ 2.122989] NET: Registered PF_PACKET protocol family > [ 2.128116] 8021q: 802.1Q VLAN Support v1.8 > [ 2.173478] printk: console [netcon0] enabled > [ 2.177927] netconsole: network logging started > [ 2.183614] clk: Disabling unused clocks > [ 2.368191] Freeing initrd memory: 1064K > [ 2.378419] Freeing unused kernel image (initmem) memory: 1024K > [ 2.386528] Checked W+X mappings: passed, no W+X pages found > [ 2.392272] Run /init as init process > rofs = mtd4 squashfs rwfs = mtd5 jffs2 > mount: mounting /dev/mtdblock4 on run/initramfs/ro failed: Invalid argument > [ 3.427189] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000000: 0x49f3 instead > [ 3.436958] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x00000004: 0xe73d instead > > [... XXX *many* more of those XXX at various offsets...] > > [ 10.907145] jffs2: jffs2_scan_eraseblock(): Magic bitmask 0x1985 not found at 0x003f0024: 0x68b6 instead > [ 10.916740] jffs2: Further such events for this erase block will not be printed > [ 10.934585] jffs2: Cowardly refusing to erase blocks on filesystem with no valid JFFS2 nodes > [ 10.943154] jffs2: empty_blocks 0, bad_blocks 0, c->nr_blocks 64 > mount: mounting /dev/mtdblock5 on run/initramfs/rw failed: Input/output error > > Mounting read-write /dev/mtdblock5 filesystem failed. Please fix and run > mount /dev/mtdblock5 run/initramfs/rw -t jffs2 -o rw > or perform a factory reset with the clean-rwfs-filesystem option. > Fatal error, tri[ 10.990385] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000100 > [ 10.999145] CPU: 0 PID: 1 Comm: init Not tainted 6.6.30-f013890-00167-gf013890407d8 #1 > [ 11.007102] Hardware name: Generic DT based system > [ 11.011934] unwind_backtrace from show_stack+0x18/0x1c > [ 11.017265] show_stack from dump_stack_lvl+0x24/0x2c > [ 11.022404] dump_stack_lvl from panic+0xf4/0x30c > [ 11.027180] panic from do_exit+0x8a8/0x8d0 > [ 11.031441] do_exit from do_group_exit+0x40/0x84 > [ 11.036209] do_group_exit from __wake_up_parent+0x0/0x1c > > > U-Boot 2013.07 (Mar 12 2024 - 14:08:49) > > I2C: ready > DRAM: 424 MiB > eSPI Handshake complete > OEM_BOARD_INIT - Start (BMC) > LPC mode > OEM_BOARD_INIT - End > Flash: Found SPI Chip Micron/Numonyx N25Q512A(0x20ba) 2x I/O READ, NORMAL WRITE > 64 MiB > MMC: > *** Warning - bad CRC, using default environment > > Un-Protected 1 sectors > Erasing Flash... > Erasing sector 4 ... ok. > Erased 1 sectors > Writing to Flash... done > Protected 1 sectors > Net: RTL8211E, EEECR = 0x06 > RTL8211E, EEEAR = 0x00 > RTL8211E, EEELPAR = 0x06 > RTL8211E, LACR = 0xc1 > RTL8211E, LCR = 0x9742 > ast_eth0, ast_eth1 > DRAM ECC enabled > Hit any key to stop autoboot: 0 > AST2500EVB> > ``` > > So afaiui my attempt misfired, the kernel panic'd, reset the system, and > then... u-boot kinda decided to erase flash sector 4 (why?), > Not sure, especially because it's the vendor u-boot :) > which seems to have contained something important for (the original u-boot from Gigabyte on SPI flash) to boot. I did not realize that and cut power, waiting for a few secs before powering up again... only to notice that my UART serial console was staying dark this time. Meanwhile, the BMC NIC LEDs blink in a strange loop that hints at some restart procedure that is repeated ad nauseam. > > To my luck, however, the host itself still willingly boots into the live system that I had set up before, but with a subtle difference once it's loaded: The AST2500 seems to be in a state much more amenable to Andrew's `culvert` utility now, which reports its neighborhood BMC like this now: > > ``` > root@grml ~ # /tmp/culvert probe > [*] failed to initialise devmem bridge: -1 > [*] Accessing the BMC's AHB via the p2a bridge > debug: Permissive > Debug UART port: UART5 > xdma: Restricted > BMC: Disabled > VGA: Enabled > XDMA on VGA: Enabled > XDMA is constrained: Yes > p2a: Permissive > BMC: Disabled > VGA: Enabled > MMIO on VGA: Enabled > [0x00000000 - 0x0fffffff] Firmware: Writable > [0x10000000 - 0x1fffffff] SoC IO: Writable > [0x20000000 - 0x2fffffff] BMC Flash: Writable > [0x30000000 - 0x3fffffff] Host Flash: Writable > [0x40000000 - 0x5fffffff] Reserved: Writable > [0x60000000 - 0x7fffffff] LPC Host: Writable > [0x80000000 - 0xffffffff] DRAM: Writable > ilpc: Permissive > SuperIO address: 0x2e > ``` > > (Before my mishap, whenever I tried to run that command, all but `ilpc` yielded > nothing, and even `ilpc` reported "Restricted" - none of which I know how to > interpret at all, by the way! :D) So I have a bit of an educated guess here: 1. For AST SoCs prior to the AST2600, disabling the hardware backdoors must be done in firmware 2. Your process above has corrupted u-boot 3. The corruption is such that u-boot fails prior to applying the backdoor mitigations (if that's where the mitigation was done to begin with - however it's where we do the mitigation in OpenBMC) As for the culvert 'probe' output and the Permissive/Restricted/Disabled states: - Permissive means there are no constraints on the bridge - culvert can read and write any AHB address - Restricted means some constraints apply to the bridge - either the address space is restricted (e.g. XDMA is constrained to the VGA reserved memory), or write access is disabled for some portion or all of the AHB address space (e.g. the P2A write filters for the listed regions) - Disabled means what it says on the tin, the BMC's AHB address space cannot be accessed via the bridge at hand, we cannot read or write. > > > So I guess I have at least six questions now: > > 1.) What happened when the kernel called it quits, u-boot reloaded and decided > to format some of its flash? Not sure, but you'd do well to boot a kernel that doesn't try to mount partitions from the flash. > > 2.) Can I influence this behavior on my (spare) board before I try again? With the above suggestion it largely comes back to how you assemble the initrd used by the kernel > > 3.) Can I restore the original firmware/SPI content on my board by any means > from the now running host OS? If so, what way would you suggest I try first? > 4.) Does having `culvert` have this new level of access have any new advantages > or open possibilities that I should be aware of? So one of the motivations for culvert is to reflash the BMC over the AHB bridges reported in the probe output. This works regardless of the state of the BMC firmware, so long as it hasn't disabled the hardware backdoors. You can try it with e.g. ``` # culvert -vv write firmware < $firmware_image ``` (you may want to experiment with `culvert -vv read firmware` first). That said, experience in [1] suggests Gigabyte have introduced some gremlins that aren't accounted for by culvert, and that you might have more success with gigaflash. [1]: https://github.com/amboar/culvert/issues/51#issuecomment-2129043859 > > 5.) Suppose I can restore the BMC's original SPI content and behavior - what's > a recommended way to have the TFPT'd kernel boot into an OpenBMC rootfs > *without* having it store on the BMC's main storage/overwriting SPI? If you're looking to deal with OpenBMC directly then this collection of patches from Patrick will probably help: https://gerrit.openbmc.org/q/topic:%22no-rootfs%22 Otherwise, using buildroot to create a self-contained kernel and initramfs is a quick way to make progress. > > 6.) Assuming this cannot be recovered in software - what are my chances of > identifying the SPI flash on my board as such, and re-writing its contents > using an affordable SPI programmer solution, given that I've never done > anything like this with hardware before? :^) From the manual[2] I expect it's in the unmarked socket between the PCIe x4 (27), M.2 (28) and PCIe x16 (29) slots. [2]: https://download.gigabyte.com/FileList/Manual/server_manual_MC12-LE0_e_v1.0.pdf?v=24c023a8081dd64c15be33bf2abf5220 Andrew ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-28 0:26 ` Andrew Jeffery @ 2024-05-28 18:37 ` Johannes Truschnigg 2024-05-29 0:52 ` Andrew Jeffery 0 siblings, 1 reply; 11+ messages in thread From: Johannes Truschnigg @ 2024-05-28 18:37 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 21458 bytes --] Hi again! :) On Tue, May 28, 2024 at 09:56:38AM +0930, Andrew Jeffery wrote: > [...] > > ``` > > root@grml ~ # /tmp/culvert probe > > [*] failed to initialise devmem bridge: -1 > > [*] Accessing the BMC's AHB via the p2a bridge > > debug: Permissive > > Debug UART port: UART5 > > xdma: Restricted > > BMC: Disabled > > VGA: Enabled > > XDMA on VGA: Enabled > > XDMA is constrained: Yes > > p2a: Permissive > > BMC: Disabled > > VGA: Enabled > > MMIO on VGA: Enabled > > [0x00000000 - 0x0fffffff] Firmware: Writable > > [0x10000000 - 0x1fffffff] SoC IO: Writable > > [0x20000000 - 0x2fffffff] BMC Flash: Writable > > [0x30000000 - 0x3fffffff] Host Flash: Writable > > [0x40000000 - 0x5fffffff] Reserved: Writable > > [0x60000000 - 0x7fffffff] LPC Host: Writable > > [0x80000000 - 0xffffffff] DRAM: Writable > > ilpc: Permissive > > SuperIO address: 0x2e > > ``` > > > > (Before my mishap, whenever I tried to run that command, all but `ilpc` yielded > > nothing, and even `ilpc` reported "Restricted" - none of which I know how to > > interpret at all, by the way! :D) > > So I have a bit of an educated guess here: > > 1. For AST SoCs prior to the AST2600, disabling the hardware backdoors > must be done in firmware > 2. Your process above has corrupted u-boot > 3. The corruption is such that u-boot fails prior to applying the > backdoor mitigations (if that's where the mitigation was done to begin > with - however it's where we do the mitigation in OpenBMC) > > As for the culvert 'probe' output and the > Permissive/Restricted/Disabled states: > > - Permissive means there are no constraints on the bridge - culvert can > read and write any AHB address > - Restricted means some constraints apply to the bridge - either the > address space is restricted (e.g. XDMA is constrained to the VGA > reserved memory), or write access is disabled for some portion or all > of the AHB address space (e.g. the P2A write filters for the listed > regions) > - Disabled means what it says on the tin, the BMC's AHB address space > cannot be accessed via the bridge at hand, we cannot read or write. Thanks for this summary and explanation. The "I know some of these words" is strong in me still, esp. during the later paragraphs, but I think I have a chance to get to understand more about how it all comes together eventually ;) > > So I guess I have at least six questions now: > > > > 1.) What happened when the kernel called it quits, u-boot reloaded and decided > > to format some of its flash? > > Not sure, but you'd do well to boot a kernel that doesn't try to mount > partitions from the flash. Understood. Is that always the case for OpenBMC kernel images with default config? > > 3.) Can I restore the original firmware/SPI content on my board by any means > > from the now running host OS? If so, what way would you suggest I try first? > > > 4.) Does having `culvert` have this new level of access have any new advantages > > or open possibilities that I should be aware of? > > So one of the motivations for culvert is to reflash the BMC over the > AHB bridges reported in the probe output. This works regardless of the > state of the BMC firmware, so long as it hasn't disabled the hardware > backdoors. You can try it with e.g. > > ``` > # culvert -vv write firmware < $firmware_image > ``` > > (you may want to experiment with `culvert -vv read firmware` first). > > That said, experience in [1] suggests Gigabyte have introduced some > gremlins that aren't accounted for by culvert, and that you might have > more success with gigaflash. > > [1]: > https://github.com/amboar/culvert/issues/51#issuecomment-2129043859 So I did play around with this after I'd read the above github issue's latest additions (how nice it was to learn that someone actually could use what little info I had produced so far already! :)), and I concluded I would want to try to get `culvert` to not flash back stock fw, but the OpenBMC phosphor image I had prepared a while ago. And indeed, it can be done, and I did it. I am just nor exactly sure *how* or *why* I could do it, but maybe you can figure that out with the information I gathered. First, I used `gigaflash` to dump BMC ROM to a file, and that worked somewhat unreliably at first, to my surprise - the first `gigaflash_x64 -dump somefile` attempt caused my host to hang, but it recovered to its original state (BMC dead, live distro (grml amd64) booted OK) after a power cycle. The subsequent attempts using a more up-to-date release of gigaflash worked, and while fooling around with culvert and gigaflash (to check if gigaflash always produced the same result and to find out if the switches that Gigabyte's provided wrapper shellscript used to updaye fw actually made a difference), I noticed that at some point, culvert suddenly could (correctly, bit-by-bit identically!) dump the ROM, too! For posterity, this is what it looked like when gigaflash successfully dumped the image for me: ``` # Tools/gigaflash_x64 -dump test.img -cs 0 -2500 gigaflash v2.0.10 Failed to connect BMC, try to dump image! --- Dump image from BMC... Find ASPEED Device 1a03:2000 on 4:0.0 MMIO Virtual Address: a4e84000 Relocate IO Base: f000 Found ASPEED Device 1a03:2500 rev. 41 Static Memory Controller Information: CS0 Flash Type is SPI CS1 Flash Type is SPI CS2 Flash Type is SPI CS3 Flash Type is NOR CS4 Flash Type is NOR Boot CS is 0 Option Information: CS: 0 Flash Type: SPI [Warning] Don't AC OFF or Reboot System During BMC Firmware Update!! [SOCFLASH] Flash ID : 20ba20 Find Flash Chip #1: Numonyx N25Q512 Backup Flash Chip O.K. --- Dump image finished --- Wait 90 secs for BMC ready... ``` I tried to arrive at a minimal reproducer on what I had to do after a power cut to get `culvert read firmware` to work, and in the end, it seems to be that one needs to run `gigaflash -dump some_filename -2500` ... once, and *only then* culvert can read the data as well. A session log I kept when it successfully dumped looked like this: ``` [*] Found 5 registered bridge drivers [*] Trying bridge driver l2a [*] Failed to initialise L2A bridge: -95 [*] Trying bridge driver ilpc [*] Probing ilpc [*] Probing 0x2e for SuperIO [*] Unlocking SuperIO: 0 [*] Selecting SuperIO device 2 (SUART1): 0 [*] Found device 2 selected: 0 [*] Selecting SuperIO device 12 (SUART4): 0 [*] Found device 12 selected: 0 [*] Locking SuperIO [*] Found SuperIO device at 0x2e [*] Probing for SoC revision registers [*] ahb_readl: 0x1e6e2004: 0xf7cffedc [*] ahb_readl: 0x1e6e207c: 0x04030303 [*] Found revision 0x4030303 [*] Trying bridge driver devmem [*] failed to initialise devmem bridge: -1 [*] Trying bridge driver debug-uart [*] Unrecognised argument list for debug interface (0) [*] Trying bridge driver p2a [*] Probing p2a [*] Probing for SoC revision registers [*] ahb_readl: 0x1e6e2004: 0xf7cffedc [*] ahb_readl: 0x1e6e207c: 0x04030303 [*] Found revision 0x4030303 [*] Accessing the BMC's AHB via the p2a bridge [*] Probing for SoC revision registers [*] ahb_readl: 0x1e6e2004: 0xf7cffedc [*] ahb_readl: 0x1e6e207c: 0x04030303 [*] Found revision 0x4030303 [*] Selected devicetree for SoC 'aspeed,ast2500' [*] Found 15 registered drivers [*] Processing devicetree node at /aliases [*] Processing devicetree node at /memory@80000000 [*] Processing devicetree node at /ahb [*] Processing devicetree node at /ahb/sram@1e720000 [*] Processing devicetree node at /ahb/bus-controller@1e600000 [*] Bound trace driver to /ahb/bus-controller@1e600000 [*] Processing devicetree node at /ahb/apb [*] Processing devicetree node at /ahb/apb/spi@1e620000 [*] Bound sfc driver to /ahb/apb/spi@1e620000 [*] Processing devicetree node at /ahb/apb/spi@1e630000 [*] Bound sfc driver to /ahb/apb/spi@1e630000 [*] Processing devicetree node at /ahb/apb/spi@1e631000 [*] Bound sfc driver to /ahb/apb/spi@1e631000 [*] Processing devicetree node at /ahb/apb/memory-controller@1e6e0000 [*] Bound sdmc driver to /ahb/apb/memory-controller@1e6e0000 [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000 [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/clock [*] Bound clk driver to /ahb/apb/syscon@1e6e2000/clock [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/strapping [*] Bound strap driver to /ahb/apb/syscon@1e6e2000/strapping [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/superio [*] Bound sioctl driver to /ahb/apb/syscon@1e6e2000/superio [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/bridge-controller [*] Bound bridge-controller driver to /ahb/apb/syscon@1e6e2000/bridge-controller [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/debug-bridge-controller [*] Bound debugctl driver to /ahb/apb/syscon@1e6e2000/debug-bridge-controller [*] Processing devicetree node at /ahb/apb/syscon@1e6e2000/pcie-bridge-controller [*] Bound pciectl driver to /ahb/apb/syscon@1e6e2000/pcie-bridge-controller [*] Bound scu driver to /ahb/apb/syscon@1e6e2000 [*] Processing devicetree node at /ahb/apb/watchdog@1e785000 [*] Bound wdt driver to /ahb/apb/watchdog@1e785000 [*] Processing devicetree node at /ahb/apb/watchdog@1e785020 [*] Bound wdt driver to /ahb/apb/watchdog@1e785020 [*] Processing devicetree node at /ahb/apb/watchdog@1e785040 [*] Bound wdt driver to /ahb/apb/watchdog@1e785040 [*] Processing devicetree node at /ahb/apb/serial@1e787000 [*] Bound vuart driver to /ahb/apb/serial@1e787000 [*] Processing devicetree node at /ahb/apb/lpc@1e789000 [*] Processing devicetree node at /ahb/apb/lpc@1e789000/bridge-controller [*] Bound ilpcctl driver to /ahb/apb/lpc@1e789000/bridge-controller [*] Bound uart-mux driver to /ahb/apb/lpc@1e789000 [*] Initialising flash controller [*] fdt: Looking up device name 'fmc' [*] fdt: Locating node with device path '/ahb/apb/spi@1e620000' [*] ahb_readl: 0x1e6e2000: 0x00000001 [*] Initialised scu driver [*] Initialised clk driver [*] ahb_readl: 0x1e6e2070: 0xf120f287 [*] ahb_readl: 0x1e620010: 0x00002400 [*] ahb_readl: 0x1e620000: 0x8007002a [*] ahb_writel: 0x1e620000: 0x8007002a [*] ahb_writel: 0x1e620010: 0x00000400 [*] ahb_writel: 0x1e620094: 0x00000000 [*] Initialised sfc driver [*] Initialising flash chip [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000403 [*] ahb_readl: 0x20000000: 0x02020202 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000400 [*] LIBFLASH: Init status: 02 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000403 [*] ahb_readl: 0x20000000: 0x1020ba20 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000400 [*] LIBFLASH: Flash ID: 20.ba.20 (20ba20) [*] LIBFLASH: Found chip Micron N25Qx512Ax size 64M erase granule: 4K [*] LIBFLASH: Flash >16MB, enabling 4B mode... [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000403 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000400 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000403 [*] ahb_readl: 0x20000000: 0x02020202 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000400 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000403 [*] ahb_writel: 0x1e620010: 0x00000407 [*] ahb_writel: 0x1e620010: 0x00000400 [*] LIBFLASH: Enabling controller 4B mode... [*] ahb_readl: 0x1e620004: 0x00000701 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620004: 0x00000701 [*] Write-protecting all chip-selects [*] ahb_readl: 0x1e620000: 0x8007002a [*] ahb_writel: 0x1e620000: 0x8007002a [*] Exfiltrating BMC flash to stdout ................................................................ [*] ahb_readl: 0x1e620000: 0x8007002a [*] ahb_writel: 0x1e620000: 0x8007002a [*] Unbound instance of driver uart-mux [*] Unbound instance of driver ilpcctl [*] Unbound instance of driver vuart [*] Unbound instance of driver wdt [*] Unbound instance of driver wdt [*] Unbound instance of driver wdt [*] Unbound instance of driver scu [*] Unbound instance of driver pciectl [*] Unbound instance of driver debugctl [*] Unbound instance of driver bridge-controller [*] Unbound instance of driver sioctl [*] Unbound instance of driver strap [*] Unbound instance of driver clk [*] Unbound instance of driver sdmc [*] Unbound instance of driver sfc [*] Unbound instance of driver sfc [*] ahb_writel: 0x1e620010: 0x00002400 [*] Unbound instance of driver sfc [*] Unbound instance of driver trace ``` I did also try to get mmiotrace, which you mentioned in the GH issue (I have never worked with it before, but I am familiar with ftrace, so I am not 100% but reasonably certain that I did not hold it wrong), to work, but I could not make it emit any tracing data while either culvert or gigaflash were dumping ROM. Only when (un)loading the `ast` driver, lots of tracing data could be collected. I do have strace capture of `gigaflash` running for the first time after a reboot, but all the juicy bits seem to hide behind mmap() anyway, so I will only provide it upon request. Then it dawned on me that `culvert probe` before and after the gigaflash "unlock" might hold the key, and when diffing two runs' results (one before, when culvert could not dump anything, and one after, when it very well could) yielded this result: ``` $ diff -u1 culvert_probe_initial culvert_probe_after_dump --- culvert_probe_initial +++ culvert_probe_after_dump @@ -121,4 +121,4 @@ [*] ahb_readl: 0x1e6e2070: 0xf120f286 -[*] ahb_readl: 0x1e789100: 0x00000000 -ilpc: Permissive +[*] ahb_readl: 0x1e789100: 0x00000040 +ilpc: Restricted [*] ahb_readl: 0x1e6e2070: 0xf120f286 ``` I do not know how to interpret this, but here's goping you can tell if this might help solve the mystery also reported in the GH issue? :) > > 5.) Suppose I can restore the BMC's original SPI content and behavior - what's > > a recommended way to have the TFPT'd kernel boot into an OpenBMC rootfs > > *without* having it store on the BMC's main storage/overwriting SPI? > > If you're looking to deal with OpenBMC directly then this collection of > patches from Patrick will probably help: > > https://gerrit.openbmc.org/q/topic:%22no-rootfs%22 Thanks a bunch, this seems like a very useful reading list for when I sorted my immediate trouble with the board/BMC resulting of what I am about to detail at the end of this mail! ;) > > [...] > > 6.) Assuming this cannot be recovered in software - what are my chances of > > identifying the SPI flash on my board as such, and re-writing its contents > > using an affordable SPI programmer solution, given that I've never done > > anything like this with hardware before? :^) > > From the manual[2] I expect it's in the unmarked socket between the > PCIe x4 (27), M.2 (28) and PCIe x16 (29) slots. Thanks, I was afraid so - despite the diagram in the manual, the IC is not socketed, so I guess I'll have to find someone who'd be able to desolder it for me, or find a test clip that is compatible with that kind of "socket". Anyhow, like I hinted at above, that's not the end of today's episode: I did try if I could get culvert to *write* my OpenBMC flash in from my root prompt, and it readily complied on first try (after I'd gotten it to dump first). Unfortunately, I ran it with `-v -v` in effect, LOTS of debug output overwhelmed my generous scrollback buffer, and I could only preserve the last few thousands of lines it put out while writing and verifying flash. It looks a lot like this: ``` [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 .[*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_readl: 0x20000000: 0x02020202 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_readl: 0x20000000: 0x03030303 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_readl: 0x20000000: 0x00000000 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002403 [*] ahb_readl: 0x20000000: 0x81818181 [*] ahb_writel: 0x1e620010: 0x00002407 [*] ahb_writel: 0x1e620010: 0x00002400 . [*] LIBFLASH: Verifying... ................................................................................................................................................................................................................................................................ [*] Performing SoC reset [*] fdt: Looking up device name 'wdt2' [*] fdt: Locating node with device path '/ahb/apb/watchdog@1e785020' [*] ahb_readl: 0x1e78502c: 0x00000010 [*] wdt_readl: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] wdt_writel: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] ahb_writel: 0x1e78502c: 0x00000010 [*] ahb_readl: 0x1e78502c: 0x00000010 [*] wdt_readl: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] wdt_writel: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] ahb_writel: 0x1e78502c: 0x00000010 [*] wdt_writel: base: 0x1e785020, reg: 0x1c, val: 0x023ffffb [*] ahb_writel: 0x1e78503c: 0x023ffffb [*] ahb_readl: 0x1e78502c: 0x00000010 [*] wdt_readl: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] wdt_writel: base: 0x1e785020, reg: 0x04, val: 0x004c4b40 [*] ahb_writel: 0x1e785024: 0x004c4b40 [*] wdt_writel: base: 0x1e785020, reg: 0x08, val: 0x00004755 [*] ahb_writel: 0x1e785028: 0x00004755 [*] ahb_readl: 0x1e78502c: 0x00000010 [*] wdt_readl: base: 0x1e785020, reg: 0x0c, val: 0x00000010 [*] wdt_writel: base: 0x1e785020, reg: 0x0c, val: 0x00000013 [*] ahb_writel: 0x1e78502c: 0x00000013 [*] Waiting 6000000 microseconds for watchdog timer to expire [*] ahb_writel: 0x1e6e207c: 0x00000001 [*] wdt_writel: base: 0x1e785020, reg: 0x04, val: 0x00000000 [*] ahb_writel: 0x1e785024: 0x00000000 [*] Unbound instance of driver uart-mux [*] Unbound instance of driver ilpcctl [*] Unbound instance of driver vuart [*] Unbound instance of driver wdt [*] Unbound instance of driver wdt [*] Unbound instance of driver wdt [*] Unbound instance of driver scu [*] Unbound instance of driver pciectl [*] Unbound instance of driver debugctl [*] Unbound instance of driver bridge-controller [*] Unbound instance of driver sioctl [*] Unbound instance of driver strap [*] Unbound instance of driver clk [*] Unbound instance of driver sdmc [*] Unbound instance of driver sfc [*] Unbound instance of driver sfc [*] ahb_writel: 0x1e620010: 0x00002400 [*] Unbound instance of driver sfc [*] Unbound instance of driver trace /tmp/culvert -v -v write firmware < 190.48s user 69.45s system 97% cpu 4:25.94 total ``` The result is both very encouraging and also disappointing, because as you intially theorized, the BMC boots fine with OpenBMC flashed onto it - but none of its host system manmagement capabilities actually work. I do have the vanilla OpenBMC web application available now, with DHCP, DNS, NTP et al. working fine for the BMC, can log in via SSH, but all the peripherals the BMC ought to be able to manage and hook into do not work at present. The next step that I would want to take is that I find a way to revert to the BMC stock fw with having only OpenBMC running, since the host apparently cannot boot any more (same situation as with the stock BMC fw when u-boot had initialized, but no BMC system was allowed to boot up, afaict - the power button/contacts just do nothing) in this state. After that, I would like to establish a sane (and hopefully easy) way to convert the board's BMC firmware from OpenBMC to stock, and vice versa. Once I have established a surefire and straightfoward way to do what I have done in such meandering and clumsy attempts, I would like to learn more about how the "M" is actually put into this whole "BMC" thing, and see how far I can take that. The stock fw has some interesting description files regarding i2c configs that might come in handy, but I am just not educated enough (yet, I hope) to make real sense of it :) Can you perhaps offer me advice on how to flash arbitrary new SPI flash contents from either OpenBMC's u-boot or an OpenBMC root shell, or what I would need to look at in detail to learn how to do that? As always, I am very grateful for anyone's advice and time. Thank you! :) -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-28 18:37 ` Johannes Truschnigg @ 2024-05-29 0:52 ` Andrew Jeffery 2024-05-30 15:37 ` Johannes Truschnigg 0 siblings, 1 reply; 11+ messages in thread From: Andrew Jeffery @ 2024-05-29 0:52 UTC (permalink / raw) To: Johannes Truschnigg, openbmc On Tue, 2024-05-28 at 20:37 +0200, Johannes Truschnigg wrote: > Hi again! :) > > On Tue, May 28, 2024 at 09:56:38AM +0930, Andrew Jeffery wrote: > > > > So I guess I have at least six questions now: > > > > > > 1.) What happened when the kernel called it quits, u-boot reloaded and decided > > > to format some of its flash? > > > > Not sure, but you'd do well to boot a kernel that doesn't try to mount > > partitions from the flash. > > Understood. Is that always the case for OpenBMC kernel images with default > config? Depends on what you mean by default. Which platform did you build for? > > The subsequent attempts using a more up-to-date release of gigaflash worked, > and while fooling around with culvert and gigaflash (to check if gigaflash > always produced the same result and to find out if the switches that Gigabyte's > provided wrapper shellscript used to updaye fw actually made a difference), I > noticed that at some point, culvert suddenly could (correctly, bit-by-bit > identically!) dump the ROM, too! Right, culvert is likely missing some trick with initialising the flash controller. I'm trying to understand what that might be :) > > For posterity, this is what it looked like when gigaflash successfully dumped > the image for me: > > ``` > # Tools/gigaflash_x64 -dump test.img -cs 0 -2500 > gigaflash v2.0.10 > Failed to connect BMC, try to dump image! > --- Dump image from BMC... > Find ASPEED Device 1a03:2000 on 4:0.0 > MMIO Virtual Address: a4e84000 > Relocate IO Base: f000 > Found ASPEED Device 1a03:2500 rev. 41 > Static Memory Controller Information: > CS0 Flash Type is SPI > CS1 Flash Type is SPI > CS2 Flash Type is SPI > CS3 Flash Type is NOR > CS4 Flash Type is NOR > Boot CS is 0 > Option Information: > CS: 0 > Flash Type: SPI > [Warning] Don't AC OFF or Reboot System During BMC Firmware Update!! > [SOCFLASH] Flash ID : 20ba20 > Find Flash Chip #1: Numonyx N25Q512 > Backup Flash Chip O.K. > --- Dump image finished > --- Wait 90 secs for BMC ready... > ``` > > > I tried to arrive at a minimal reproducer on what I had to do after a power > cut to get `culvert read firmware` to work, and in the end, it seems to be > that one needs to run > > `gigaflash -dump some_filename -2500` > > ... once, and *only then* culvert can read the data as well. Okay, handy to know. That jives with my comment above. > > I did also try to get mmiotrace, which you mentioned in the GH issue (I have > never worked with it before, but I am familiar with ftrace, so I am not 100% > but reasonably certain that I did not hold it wrong), to work, but I could not > make it emit any tracing data while either culvert or gigaflash were dumping > ROM. Only when (un)loading the `ast` driver, lots of tracing data could be > collected. > > I do have strace capture of `gigaflash` running for the first time after a > reboot, but all the juicy bits seem to hide behind mmap() anyway, so I will > only provide it upon request. Well, consider this a request :D What it's mapping is of interest to me, along with what it's opening more broadly, and any calls to ioperm() or iopl(). > > Then it dawned on me that `culvert probe` before and after the gigaflash > "unlock" might hold the key, and when diffing two runs' results (one before, > when culvert could not dump anything, and one after, when it very well could) > yielded this result: > > ``` > $ diff -u1 culvert_probe_initial culvert_probe_after_dump > --- culvert_probe_initial > +++ culvert_probe_after_dump > @@ -121,4 +121,4 @@ > [*] ahb_readl: 0x1e6e2070: 0xf120f286 > -[*] ahb_readl: 0x1e789100: 0x00000000 > -ilpc: Permissive > +[*] ahb_readl: 0x1e789100: 0x00000040 > +ilpc: Restricted > [*] ahb_readl: 0x1e6e2070: 0xf120f286 > ``` So something (it's unclear to me whether the BMC has rebooted in your test here) has set the bit to disable writes via the iLPC2AHB bridge. The reason I ask for the strace above is the mmiotrace output in the github issue only shows a couple of accesses to the P2A. I'm wondering whether gigaflash is exploiting iLPC2AHB and LPC FW/MEM cycles instead (or perhaps I need to improve my understanding of the limits of mmiotrace). > > The result is both very encouraging and also disappointing, because as you > intially theorized, the BMC boots fine with OpenBMC flashed onto it - but none > of its host system manmagement capabilities actually work. I do have the > vanilla OpenBMC web application available now, with DHCP, DNS, NTP et al. > working fine for the BMC, can log in via SSH, but all the peripherals the BMC > ought to be able to manage and hook into do not work at present. Yes, this is expected. Configuring the BMC applications to match the integration into the board is the biggest challenge. > > > The next step that I would want to take is that I find a way to revert to the > BMC stock fw with having only OpenBMC running, since the host apparently cannot > boot any more (same situation as with the stock BMC fw when u-boot had > initialized, but no BMC system was allowed to boot up, afaict - the power > button/contacts just do nothing) in this state. After that, I would like to > establish a sane (and hopefully easy) way to convert the board's BMC firmware > from OpenBMC to stock, and vice versa. The least-effort hack would be to place the stock firmware at /run/initramfs/image-bmc and reboot OpenBMC - this will write the stock firmware image back to the flash for you. > > Once I have established a surefire and straightfoward way to do what I have > done in such meandering and clumsy attempts, I would like to learn more about > how the "M" is actually put into this whole "BMC" thing, and see how far I can > take that. The stock fw has some interesting description files regarding i2c > configs that might come in handy, but I am just not educated enough (yet, I > hope) to make real sense of it :) Yeah, the I2C configs will certainly help. Breaking into the stock firmware on the hardware and tracing things like GPIO accesses would go a long way. With the tools at hand it shouldn't be too difficult :) > > Can you perhaps offer me advice on how to flash arbitrary new SPI flash > contents from either OpenBMC's u-boot or an OpenBMC root shell, or what I would > need to look at in detail to learn how to do that? See the comment above regarding /run/initramfs/image-bmc. However you can also boot to an initrd and use flashcp. The main thing is you need to make sure no other accesses to the mtd device are taking place (hence suggesting the initrd environment). Andrew ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-29 0:52 ` Andrew Jeffery @ 2024-05-30 15:37 ` Johannes Truschnigg 2024-05-31 1:06 ` Andrew Jeffery 0 siblings, 1 reply; 11+ messages in thread From: Johannes Truschnigg @ 2024-05-30 15:37 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 6133 bytes --] Hi list! First things first, I wanted to share the good news that I can now boot the host with OpenBMC running the show on the BMC: Fiddling with GPIO #539 on the OpenBMC root shell in the right way makes the host power up a few seconds later. I do it like this for now: ``` echo 539 > /sys/class/gpio/export echo out > /sys/class/gpio/gpio539/direction echo 0 > /sys/class/gpio/gpio539/value sleep 1 echo 1 > /sys/class/gpio/gpio539/value ``` I'd like to say thanks to my friend Paul on IRC here in public, because without your patient guidance and steady flow of information and ideas, this 100% wouldn't have been possible!! :) For posterity and completeness, this is what culvert has to say about the BMC's state now that it runs OpenBMC: ``` root@grml ~ # /tmp/culvert -v -v probe [*] Found 5 registered bridge drivers [*] Trying bridge driver l2a [*] Failed to initialise L2A bridge: -95 [*] Trying bridge driver ilpc [*] Probing ilpc [*] Probing 0x2e for SuperIO [*] Unlocking SuperIO: 0 [*] Selecting SuperIO device 2 (SUART1): 0 [*] Found device 255 selected: 0 [*] Locking SuperIO [*] Probing 0x4e for SuperIO [*] Unlocking SuperIO: 0 [*] Selecting SuperIO device 2 (SUART1): 0 [*] Found device 255 selected: 0 [*] Locking SuperIO [*] SuperIO disabled [*] Trying bridge driver devmem [*] failed to initialise devmem bridge: -1 [*] Trying bridge driver debug-uart [*] Unrecognised argument list for debug interface (0) [*] Trying bridge driver p2a [*] Probing p2a [*] Probing for SoC revision registers [*] ahb_readl: 0x1e6e2004: 0xffffffff [*] ahb_readl: 0x1e6e207c: 0xffffffff [*] Found revision 0xffffffff [*] Revision 0xffffffff is unsupported [*] Failed P2A probe: -19 [*] Accessing the BMC's AHB via the ilpc bridge [*] Probing for SoC revision registers [*] ahb_readl: 0x1e6e2004: 0xffffffff [*] ahb_readl: 0x1e6e207c: 0xffffffff [*] Found revision 0xffffffff [*] Revision 0xffffffff is unsupported [*] Failed to probe SoC revision: -19 [*] Failed to probe SoC, exiting: -19 ``` I haven't tried to execute gigaflash on the booted host yet, but it is on my list of things to try next. On Wed, May 29, 2024 at 10:22:09AM +0930, Andrew Jeffery wrote: > On Tue, 2024-05-28 at 20:37 +0200, Johannes Truschnigg wrote: > > [...] > > Understood. Is that always the case for OpenBMC kernel images with default > > config? > > Depends on what you mean by default. Which platform did you build for? I built obmc-phosphor-image for evb-ast2500. > > [...] > > Right, culvert is likely missing some trick with initialising the flash > controller. I'm trying to understand what that might be :) I was hoping you were going to say that! 8-) > > [...] > > I do have strace capture of `gigaflash` running for the first time after a > > reboot, but all the juicy bits seem to hide behind mmap() anyway, so I will > > only provide it upon request. > > Well, consider this a request :D What it's mapping is of interest to > me, along with what it's opening more broadly, and any calls to > ioperm() or iopl(). I've uploaded the zstd-compressed trace result (it contains a lot of the actual ROM content dumped with how I called strace/gigaflash) to [0]. I really hope it helps chasing down what you're looking for! :) > > [...] > > The least-effort hack would be to place the stock firmware at > /run/initramfs/image-bmc and reboot OpenBMC - this will write the stock > firmware image back to the flash for you. Thanks; this reads like a very easy way to revert to stock - I will play around with OpenBMC on the physical thing some more before actually trying that out, though! > > Once I have established a surefire and straightfoward way to do what I have > > done in such meandering and clumsy attempts, I would like to learn more about > > how the "M" is actually put into this whole "BMC" thing, and see how far I can > > take that. The stock fw has some interesting description files regarding i2c > > configs that might come in handy, but I am just not educated enough (yet, I > > hope) to make real sense of it :) > > Yeah, the I2C configs will certainly help. > > Breaking into the stock firmware on the hardware and tracing things > like GPIO accesses would go a long way. With the tools at hand it > shouldn't be too difficult :) Yeah that would be cool, but I had established with Paul that the stock fw kernel doesn't come with debugfs and missing userspace tooling for tracing syscalls, so it will certainly be a challenge (at least for me, equipped with my current skillset, and the time constraints of having a day job with some other meatspace activity on top of that :D). > > Can you perhaps offer me advice on how to flash arbitrary new SPI flash > > contents from either OpenBMC's u-boot or an OpenBMC root shell, or what I would > > need to look at in detail to learn how to do that? > > See the comment above regarding /run/initramfs/image-bmc. However you > can also boot to an initrd and use flashcp. The main thing is you need > to make sure no other accesses to the mtd device are taking place > (hence suggesting the initrd environment). Thanks again also for this! One of my problems is that I have no idea what userland support utils/components even *exist* for that kind of embedded hw, so I am very much in the dark what my options could be without such hints. Meanwhile, due to a friendly user on a German web forum who led me to similar matters discussed about an ASRock Rack-series motherboard (which also comes with AST2500 for its BMC purposes), I think I have properly identified the relevant IC where the BMC's firmware actually resides, and it looks like SOIC-8 in direct vicinity to the AST2500 itself. I will have some new gear for my electronics drawer delivered soon to see how/if I can deal with it! :) [0]: https://johannes.truschnigg.info/tmp/strace_gigaflash_initial_.3305.zst -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-30 15:37 ` Johannes Truschnigg @ 2024-05-31 1:06 ` Andrew Jeffery 0 siblings, 0 replies; 11+ messages in thread From: Andrew Jeffery @ 2024-05-31 1:06 UTC (permalink / raw) To: Johannes Truschnigg, openbmc; +Cc: Zev Weiss On Thu, 2024-05-30 at 17:37 +0200, Johannes Truschnigg wrote: > Hi list! > > First things first, I wanted to share the good news that I can now boot the > host with OpenBMC running the show on the BMC: Fiddling with GPIO #539 on the > OpenBMC root shell in the right way makes the host power up a few seconds > later. I do it like this for now: > > ``` > echo 539 > /sys/class/gpio/export > echo out > /sys/class/gpio/gpio539/direction > echo 0 > /sys/class/gpio/gpio539/value > sleep 1 > echo 1 > /sys/class/gpio/gpio539/value > ``` > > I'd like to say thanks to my friend Paul on IRC here in public, because > without your patient guidance and steady flow of information and ideas, this > 100% wouldn't have been possible!! :) Nice work. For the record, the GPIO sysfs interface is deprecated and likely to go away soon. Useful for PoCs, but longer term you should look to the libgpiod APIs and tools[1]. [1]: https://git.kernel.org/pub/scm/libs/libgpiod/libgpiod.git/about/ > > > For posterity and completeness, this is what culvert has to say about the > BMC's state now that it runs OpenBMC: > > ``` > root@grml ~ # /tmp/culvert -v -v probe > [*] Found 5 registered bridge drivers > [*] Trying bridge driver l2a > [*] Failed to initialise L2A bridge: -95 > [*] Trying bridge driver ilpc > [*] Probing ilpc > [*] Probing 0x2e for SuperIO > [*] Unlocking SuperIO: 0 > [*] Selecting SuperIO device 2 (SUART1): 0 > [*] Found device 255 selected: 0 > [*] Locking SuperIO > [*] Probing 0x4e for SuperIO > [*] Unlocking SuperIO: 0 > [*] Selecting SuperIO device 2 (SUART1): 0 > [*] Found device 255 selected: 0 > [*] Locking SuperIO > [*] SuperIO disabled > [*] Trying bridge driver devmem > [*] failed to initialise devmem bridge: -1 > [*] Trying bridge driver debug-uart > [*] Unrecognised argument list for debug interface (0) > [*] Trying bridge driver p2a > [*] Probing p2a > [*] Probing for SoC revision registers > [*] ahb_readl: 0x1e6e2004: 0xffffffff > [*] ahb_readl: 0x1e6e207c: 0xffffffff > [*] Found revision 0xffffffff > [*] Revision 0xffffffff is unsupported > [*] Failed P2A probe: -19 > [*] Accessing the BMC's AHB via the ilpc bridge > [*] Probing for SoC revision registers > [*] ahb_readl: 0x1e6e2004: 0xffffffff > [*] ahb_readl: 0x1e6e207c: 0xffffffff > [*] Found revision 0xffffffff > [*] Revision 0xffffffff is unsupported > [*] Failed to probe SoC revision: -19 > [*] Failed to probe SoC, exiting: -19 > > ``` > > I haven't tried to execute gigaflash on the booted host yet, but it is on my > list of things to try next. What you can see from the output above is that OpenBMC has disabled all the hardware backdoors that culvert tries to exploit. The consequence of this is that gigaflash will (... should?) now also fail to function. You may want to play with these u-boot options: https://github.com/openbmc/u-boot/blob/v2019.04-aspeed-openbmc/arch/arm/mach-aspeed/Kconfig#L48-L85 Or we can add support to culvert for enabling and disabling the backdoors at runtime, and run it with /dev/mem enabled on the BMC (set mem.devmem=1 on the kernel commandline). Adding support to culvert should just be a matter of wiring up some command-line UI - what bits to flip and the necessary abstractions are already provided in the implementation. > > > > > [...] > > > I do have strace capture of `gigaflash` running for the first time after a > > > reboot, but all the juicy bits seem to hide behind mmap() anyway, so I will > > > only provide it upon request. > > > > Well, consider this a request :D What it's mapping is of interest to > > me, along with what it's opening more broadly, and any calls to > > ioperm() or iopl(). > > I've uploaded the zstd-compressed trace result (it contains a lot of the > actual ROM content dumped with how I called strace/gigaflash) to [0]. I really > hope it helps chasing down what you're looking for! :) Given some of the discussion on the culvert issue, it may not, but I'll take a look. And indeed they're using the P2A, so we'll need to consider another approach to tracing the accesses. > > > > > Once I have established a surefire and straightfoward way to do what I have > > > done in such meandering and clumsy attempts, I would like to learn more about > > > how the "M" is actually put into this whole "BMC" thing, and see how far I can > > > take that. The stock fw has some interesting description files regarding i2c > > > configs that might come in handy, but I am just not educated enough (yet, I > > > hope) to make real sense of it :) > > > > Yeah, the I2C configs will certainly help. > > > > Breaking into the stock firmware on the hardware and tracing things > > like GPIO accesses would go a long way. With the tools at hand it > > shouldn't be too difficult :) > > Yeah that would be cool, but I had established with Paul that the stock fw > kernel doesn't come with debugfs and missing userspace tooling for tracing > syscalls, so it will certainly be a challenge (at least for me, equipped with > my current skillset, and the time constraints of having a day job with some > other meatspace activity on top of that :D). If you can coax it past u-boot under qemu you can trace the hardware accesses via the qemu models (without requiring support from the OS). > Meanwhile, due to a friendly user on a German web forum who led me to similar > matters discussed about an ASRock Rack-series motherboard (which also comes > with AST2500 for its BMC purposes), I think I have properly identified the > relevant IC where the BMC's firmware actually resides, and it looks like > SOIC-8 in direct vicinity to the AST2500 itself. I will have some new gear for > my electronics drawer delivered soon to see how/if I can deal with it! :) For the record, Zev (who also helps out with culvert) is the OpenBMC maintainer for the ASRock Rack platforms, so should be able to assist you there as well. Andrew ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions 2024-05-20 17:32 OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions Johannes Truschnigg 2024-05-21 1:05 ` Andrew Jeffery @ 2024-07-07 9:19 ` Johannes Truschnigg 1 sibling, 0 replies; 11+ messages in thread From: Johannes Truschnigg @ 2024-07-07 9:19 UTC (permalink / raw) To: openbmc [-- Attachment #1: Type: text/plain, Size: 5325 bytes --] Hey list! Sorry for my long radio silence -- I've been busy (with other things, but also) quietly working at this porting project of mine, and made some imho rather cool progress. I have also, however, hit a few roadblocks I cannot seem to surmount on my own, and so I turn to you, dear reader, for advice and/or consolation. I have my own "meta-gigabyte" layer including a board-specific DTS for u-boot and linux-aspeed, with results of those that seem mostly OK. The ultra-barebones bmcweb function is there, and I have iKVM working. My plan was to get host power control and the ID button/LED working as a next objective - that way, the most important features from my POV would be there, and I would feel OK-ish with publishing what I got. For the past few weeks, I had been living under the impression that poking GPIO 27 the right way did just that (power up the host), and so I went on to reverse engineer the ID button/LED stuff with `gpiomon`. I figured out a number of functions this way: The Front Panel Header (FPH) Power Button corresponds to GPIO 24. The FPH Reset Button to GPIO 26. The FPH ID Button (as well as the board-mounted ID button) to GPIO 134. HDD LED activity seems to be wired to GPIO 111. The ID LED on the board itself is controlled via GPIO offset 49. Having poked at all these GPIOs somehow made my GPIO 27 interaction to boot up the host ineffective, however. That change persisted and survived even a cycle of going back to the stock firmware, and then flashing OpenBMC again. I have no idea what went wrong there, but I might have since managed to recover/revise the procedure, and can get the host to power again as a consequence: After a *ton* of experimentation, I think I also have figure out the proper minimal sequence of gpioset and gpioget invocations to make the host boot when the BMC is online; it seems to be this magic spell: # gpioget 0 25; gpioset -m time -D open-source -s 1 0 27=1; \ gpioset 0 25=0; gpioget 0 25 (I cannot help but notice that these pins are adjacent to the RESET and POWER buttons!) What I am struggling with now is how I should best make x86-power-control or a functional equivalent benefit from this knowledge, too, and how I can make it manage host power state in my stead. I was hoping that someone on this list would be both knowledgeable and kind enough to give me advice on how to proceed. The OpenBMC tree has a number of other machines where a shellscript that's a machine-specific part of "phosphor-state-manager" does the required `gpioget`/`gpioget` dance and then uses `busctl` to let the rest of the system know how things are (supposed to be) after that - is that what I should be purusing, too? Or would it just be a matter of properly naming (*is* there a canonical (sub)set of GPIO names?) the numeric IDs in the DTS, and I could expect things to magically fall into place on their own? If there's some implementation I should be looking at in particular, I'd be happy to do and then try mimic that. Also, since right now the FPH pins/buttons don't do anything apart from registering in gpiomon, I was wondering if it's normal to be the BMC's job to wait for these event firing, and then doing the proper GPIO magic to perform what the user asked for pressing the respective button. I guess that is what has to happen, right? If so, can anyone maybe tell me where to look at for an example of how this is set up for an existing, working machine? Furthermore, I am not sure which other operations I'll still have to figure out GPIO sequences for. Graceful shutdown (I expect this to have to generate an ACPI Power Button event in the host, or is there some other magic involved?) probably, "forced" shutdown/cutting power - but what else? And does anyone have any concrete advice on how to methodically tackle that (find out which GPIO needs which treatment to generate which event)? What I also wanted to learn is how I should best integrate my custom .dts and resulting .dtb in my machine-specific layer(s), so that both the kernel and u-boot "pick it up" correctly. The kernel part was relatively easy, since the build infra seems to automatically include any additional .dts named files placed in the proper location and be able to use that via setting 'KERNEL_DEVICETREE = "gigabyte-mc12-le0.dtb"' (in my instance), but u-boot does not, and I ended up overwriting arch/arm/dts/ast2500-evb.dts with my own in a `do_patch:append() {}` block and setting 'UBOOT_DEVICETREE = "ast2500-evb"' in my machine config. I guess the *intended*/preferred way is to upstream the (correct and sufficient - which I do not have yet) DTS artifacts into both kernel and u-boot, and not to have this problem in the first place? Assuming that's not an option for now, how should I *properly* integrate my local modification (actually: addition) so that OpenBMC upstream would be in a position to accept the "meta-gigabyte" layer including it with good conscience? (That is my eventual goal :)) Thanks a lot for reading this far, and for all the support provided to me off and on list to get this closer to actually working! :) -- with best regards: - Johannes Truschnigg ( johannes@truschnigg.info ) www: https://johannes.truschnigg.info/ [-- Attachment #2: signature.asc --] [-- Type: application/pgp-signature, Size: 833 bytes --] ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2024-07-08 23:21 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2024-05-20 17:32 OpenBMC replacing AMI AST2500 BMC fw on Gigabyte MC12-LE0 - questions Johannes Truschnigg 2024-05-21 1:05 ` Andrew Jeffery 2024-05-21 17:05 ` Johannes Truschnigg 2024-05-22 1:29 ` Andrew Jeffery 2024-05-27 18:09 ` Johannes Truschnigg 2024-05-28 0:26 ` Andrew Jeffery 2024-05-28 18:37 ` Johannes Truschnigg 2024-05-29 0:52 ` Andrew Jeffery 2024-05-30 15:37 ` Johannes Truschnigg 2024-05-31 1:06 ` Andrew Jeffery 2024-07-07 9:19 ` Johannes Truschnigg
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.