Re: or1k -M virt -hda and net.

Re: or1k -M virt -hda and net.

[Geert Uytterhoeven]

Hi Rob,

CC linux-sh

On Wed, Jan 8, 2025 at 11:40 PM Rob Landley <rob@landley.net> wrote:
> The sh2eb turtle board uses JCORE_EMAC which is still an out of tree
> driver. It's not secret, just too ugly to go upstream: done by a
> contractor and never cleaned up, full of half-finished IEEE-1588 support
> and so on. It used to be on
> https://web.archive.org/web/20200812035510/http://git.musl-libc.org/cgit/linux-sh/
> but alas Rich purged his trees when he stopped being maintainer and
> archive.org didn't crawl that far down.
>
> I have the patch locally if you care, but without the corresponding
> hardware, not much point attaching 1600 lines. (We sent several turtle
> boards to Glaubitz and his assistants last year, and at least one of the
> recipients asked for a copy of the ethernet driver patch, which I sent.

I still have to ask you for that driver, so yes I am interested in
the patch ;-)

> Still applies and works as of the last kernel I updated on the board,
> 6.18 I think? The update is a "pop the sd card, stick in laptop, copy
> files, move sd card back" kinda thing.

... which has been the main obstacle for me to integrate turtleboard
in my regular kernel testing workflow. There are only 24 hours in a day,
so usually there is no time left for juggling SD cards :-(

> I should do a linux image that
> can wget and kexec, but haven't yet... Anyway, doesn't help with this
> issue.)

Does kexec work on MMU-less J2?
On SH4, there was never an upstream kernel that worked with upstream
kexec-tools. The only one that works is the kexec binary from the old
landisk distro, which predates SH support in upstream kexec-tools,
and can only start a new kernel from a system that is running kernel
2.6.22...

BTW, this is seriously off-topic for openrisc and qemu, so please
reduce the CC list when continuing the linux-sh discussion..

Thanks!

Gr{oetje,eeting}s,

                        Geert

-- 
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: or1k -M virt -hda and net.

[Rob Landley]

On 1/9/25 02:49, Geert Uytterhoeven wrote:
> Hi Rob,
> 
> CC linux-sh
> 
> On Wed, Jan 8, 2025 at 11:40 PM Rob Landley <rob@landley.net> wrote:
>> The sh2eb turtle board uses JCORE_EMAC which is still an out of tree
>> driver. It's not secret, just too ugly to go upstream: done by a
>> contractor and never cleaned up, full of half-finished IEEE-1588 support
>> and so on. It used to be on
>> https://web.archive.org/web/20200812035510/http://git.musl-libc.org/cgit/linux-sh/
>> but alas Rich purged his trees when he stopped being maintainer and
>> archive.org didn't crawl that far down.
>>
>> I have the patch locally if you care, but without the corresponding
>> hardware, not much point attaching 1600 lines. (We sent several turtle
>> boards to Glaubitz and his assistants last year, and at least one of the
>> recipients asked for a copy of the ethernet driver patch, which I sent.
> 
> I still have to ask you for that driver, so yes I am interested in
> the patch ;-)

I sent it directly because we're spamming the lists enough already 
without large attachments. :)

>> Still applies and works as of the last kernel I updated on the board,
>> 6.18 I think? The update is a "pop the sd card, stick in laptop, copy
>> files, move sd card back" kinda thing.
> 
> ... which has been the main obstacle for me to integrate turtleboard
> in my regular kernel testing workflow. There are only 24 hours in a day,
> so usually there is no time left for juggling SD cards :-(

Yeah, I should do something about that. Using linux+kexec as a stage 2 
bootloader, linux could wget from/init initramfs and you could just 
stick the file on a web server. (I could also use tftp, but toybox or 
busybox httpd is trivial to set up on a high port...)

>> I should do a linux image that
>> can wget and kexec, but haven't yet... Anyway, doesn't help with this
>> issue.)
> 
> Does kexec work on MMU-less J2?

I thought I'd hacked something up once, but then there was a pandemic 
and an insurrection and I honestly don't recall the details...

The main limiting factor on software reboot in the FPGA version is the 
bitstream preloads values into SRAM blocks, which aren't preserved after 
boot. That's really all a bitstream _does_, initialize sram in the chip 
to known starting values, it's just some of those sram blocks are in 
"lookup tables" (LUTs) that act as logic gates, except all a LUT is 
REALLY doing internally is "this set of input bits = this output bit", 
hence the name lookup table, it's address = value lookup for single 
bits. But that's all and/or/nor gates do anyway, so... (There's also 
'switching fabric" which is "which wires connect to which 
inputs/outputs"... again SRAM controls it by setting bits to flip 
switches on and off to connect together the wires...)

Anyway, FPGAs also have big sram blocks in them (in the ice40 8k we're 
using, they come in 512 byte and 64k byte sizes, something like 2 dozen 
of the small ones and 2 of the big ones, xilinx has more and probably 
bigger), and the bitstream can have initialization values for those too.

In turtle, some of those preloaded address blocks get used as a boot 
ROM, which _could_ be marked read only and preserved (it's like a 1 line 
change in the VHDL, in that big address resolution if/else staircase) 
but there's a limited amount of sram in the board (especially 
conveniently placed near the ALU and so on), and what winds up happening 
is it's recycled as cache memory and such after boot, so when you go "I 
wanna reboot now" we ain't got that boot rom contents no more.

If you press the button it toggles the FPGA's hardware reset line which 
reloads the bitstream from the flash (in ice40 this is built in to the 
FPGA, in our xilinx designs the atmel chip does it), reinitializing the 
boot rom along the way. (Since we got the high speed quad data mode 
working it takes a fraction of a second.)

But rebooting from _software_ is only supported in ASIC, not FPGA. (Or 
possibly on the really big kintex chips that have resources out the 
wazoo, there may be a config toggle in the big j2 build? It's not in the 
ice40 j1 source, which is all I've looked at recently. That's the red 
square mounts, I don't think we've ever made a board less than 2 feet 
long that those plug into. We haven't sent them out to open source devs, 
they're hundreds of dollars each and need a heat sink and actual power 
supply, and only clock maybe twice as fast as the spartan 6 lx45 in a 
turtle. They have way way way more LUTs and such, of course...)

So to software reboot, we need a kexec method that doesn't go back 
through the boot rom, and I believe the kernel's built-in "kexec" 
mechanism worked at one point?

Because it's nommu (and the board currently hasn't got any hardware 
doing DMA, although we have some bitstream upgrades to add that in the 
works), you don't have to tear down the page tables and quiesce the A20 
line and similar nonsense, you basically just tell the IRQ controller to 
stop generating any interrupts, load the kernel into a high contiguous 
block (which can be pretty fscking hard if the system's been running a 
while due to fragmentation issues inherent in nommu systems that didn't 
reserve memory to start with), and jump to the decompression code that 
gunzips it down into the lower area of ram (smashing whatever's there). 
Then it jumps to the entry point and the kernel's init resets all the 
hardware as the drivers load. (I think the serial port is assumed to 
already be set to the right speed by the bootloader, the rest gets 
re-initialized by the drivers? Been a while...)

If all you want is "bootloader loads temporary kernel, then first thing 
temporary kernel does is kexec a file out of a mount point", 
fragmentation shouldn't be a problem because the top half of memory 
never got touched. (well, "echo 3 > /proc/sys/vm/drop_caches" for luck.) 
I think it just worked at one point? (Many moons ago. Might have had to 
hack the kernel config plumbing to get it ENABLED, but that's a modern 
kernel for you...)

> On SH4, there was never an upstream kernel that worked with upstream
> kexec-tools.

I didn't use kexec tools, there's a syscall. Have you tried the simple 
thing? Checking the man page, something like:

int main(int argc, char *argv[])
{
   int kern, initrd, cmdline;

   if (argc<4) exit(dprintf(2, "needs 3 arguments: kernel.img initrd.gz 
'command line'\n"));

   kexec_file_load(open(argv[1], O_RDONLY), open(ARGV[2], O_RDONLY), 
strlen(argv[3]), argv[3], KEXEC_FILE_UNLOAD);
}

You could theoretically even (and this is disgusting):

kexec <(wget -O- http://1.2.3.4:8888/kernel) <(wget -O- 
http://1.2.3.4:8888/initramfs) "$(wget -O- http://1.2.3.4:8888/cmdline)"

I don't think the turtle board comes with dhcp by default, but there's 
one in toybox's pending directory you could switch on for mkroot. (Or do 
you want a static busybox binary for turtle?)

> The only one that works is the kexec binary from the old
> landisk distro, which predates SH support in upstream kexec-tools,
> and can only start a new kernel from a system that is running kernel
> 2.6.22...

Never even looked at it. I think there's a kexec in klibc and another in 
busybox, but mostly there's a pair of syscalls one of which provides a 
simple API without needing to know what a kexec_segment is.

I note that I have ZERO memory of actually doing this, possibly I gave 
something like the above description to a co-worker and they did the 
actual debugging to make it work? (Or it was mid-pandemic, there are 
whole months that are just a blur...)

(The real problem is I send things to the other enginers, they use them 
for a while, it got archived when switching to a new project for a new 
customer, and then I have to go "I already solved this problem" and fish 
something out of my email outbox from 2021. They forgot about it because 
we migrated servers and archived the old wiki and THAT build system was 
on the mac trashcan not the new rackmount thing... Of course I can't 
complain when I'm the one currently failing to remember (mostly because 
I didn't blog about it, grepping my blog and email outbox for keywords 
is half my institutional knowledge, being in the same room in tokyo and 
talking to people in PERSON doesn't leave a record like that). We need a 
technical archivist, and I'm already wearing too many hats...)

> BTW, this is seriously off-topic for openrisc and qemu, so please
> reduce the CC list when continuing the linux-sh discussion..

Trimmed. (No idea how Stafford, Peter, and Jason feel about it either...)

> Thanks!
> 
> Gr{oetje,eeting}s,

Rob

P.S. There _is_ a DMA engine in the big kintex build, but it's this 
giant 64 channel implementation designed to pass data to/from a string 
of giant DSPs, and the DMA engine _alone_ is bigger than the j2 
processor, and the smallest it can configure down to with the current 
config plumbing is 16 channels. We have a much smaller/simpler generic 
DMA engine for j1 sketched out, but we wound up not needing it for the 
project we designed it for (you can do a LOT with hard realtime PIO and 
a small fifo buffer, and honestly if you control the spi clock you don't 
need ANY buffer because you control the clock), and that was on J1 for 
ice40 and would take some porting for j2 anyway because there's a 
different bus layer and arguing with leingen is a bit of a timesink...