Adding support for Fujitsu SPARC64 machines?

Adding support for Fujitsu SPARC64 machines?

[Meelis Roos]

I am playing with the idea of adding support for SPARC64 V and VI CPUs 
for sparclinux (actually maybe supervising a master student doing it).

Before we take it on, I would like to understand what needs to be done, 
what are the hard points and what are the risks. Below is my current 
understanding - please correct me where I am wrong. Also, if someone 
else might have some knowledge or docs that could help, please do tell.

* CPU support - 
http://www.fujitsu.com/global/products/computing/servers/unix/sparc-enterprise/downloads/documents/ 
has a bunch of developer docs for newer SPARC64 CPUs. Seems to be 
detailed enough for just the CPU.

* Machine support - I have PrimePower 250, 450, 650 and Sparc Enterprise 
M4000 available. Of them, 450 and M4000 have XSCF with full remote 
management with power and reset control, PP650 does not have it, and I 
do not know about PP250. M4000 is more complex so maybe PPP450 would 
be the best for start?

* PCI I/O - Solaris tells there is pcicmu0 at root: SAFARI 0x8 0x4000 
and px1 at root: SAFARI 0x1 0x700000 in the M4000. Am I correct if I
think support for this bridge needs to be implemented? There are some 
references to Safari in linux/arch/sparc/kernel code but Illumos seems 
to have more complex code.

(prtconf -pv sent to DaveM for prtconfs repo)

* Are there any other devices that need support? Console device at the 
least, maybe something else? Storage and network seem to be normal 
PCI.

* Is there any hypervisor protocol for talking to scf firmware from 
domains? scf management protocol is probably not too important for start 
since it should not be needed for boot/runtime support.

* Are there any devices know to have no documentation that would keep us 
from doing it?

* OpenBSD guys had a problem with some specific invalid hardware access 
causing fault in xscf firmware that needed to be cleared by service 
engineer, assuming there is a service contract. My Fujitsus are in my 
museum so there is no service contract. Has anyone heard if the current 
firmwares are more reliable? I'm running the latest on M4000 and 
whatever came with the PrimePowers.

*

-- 
Meelis Roos (mroos@linux.ee)

Re: Adding support for Fujitsu SPARC64 machines?

[David Miller]

From: Meelis Roos <mroos@linux.ee>
Date: Thu, 8 Jan 2015 00:55:08 +0200 (EET)

> * CPU support - 
> http://www.fujitsu.com/global/products/computing/servers/unix/sparc-enterprise/downloads/documents/ 
> has a bunch of developer docs for newer SPARC64 CPUs. Seems to be 
> detailed enough for just the CPU.

The OF based cpu scanner might need some tweaking, so that it
traverses properly from "cmp" down to "core" down to "cpu" to
find cpu nodes.

of_iterate_over_cpus() seems to just look for all nodes in the
tree of type "cpu" so you might be ok with no changes.

> * PCI I/O - Solaris tells there is pcicmu0 at root: SAFARI 0x8 0x4000 
> and px1 at root: SAFARI 0x1 0x700000 in the M4000. Am I correct if I
> think support for this bridge needs to be implemented? There are some 
> references to Safari in linux/arch/sparc/kernel code but Illumos seems 
> to have more complex code.

There are two different kinds of PCI controllers in the machine.

One PCI controller has a vendor ID of Fujitsu so it looks like it's
their own ASIC.  You therefore might need to write a PCI controller
driver for it.

The other is a Sun chip with device ID 80f8, it might be compatible
with the arch/sparc/kernel/pci_fire.c driver so simply adding an
entry like:

	{
		.name = "pci",
		.compatible = "pciex108e,80f8",
	},

to fire_match[] might work.

SAFARI is the system bus.

> * Are there any other devices that need support? Console device at the 
> least, maybe something else? Storage and network seem to be normal 
> PCI.

The console should probably be over the EBUS serial which is run by
the 'su' driver.  You'll have to get PCI working for this but meanwhile
a PROM console for output will work.

> * Is there any hypervisor protocol for talking to scf firmware from 
> domains? scf management protocol is probably not too important for start 
> since it should not be needed for boot/runtime support.

I would not even be thinking about this until you have the rest of
the machine working.

Re: Adding support for Fujitsu SPARC64 machines?

[Mark Kettenis]

From: David Miller <davem@davemloft.net>
Data: Mon, 12 Jan 2015 20:53:30

> From: Meelis Roos <mroos@linux.ee>
> Date: Thu, 8 Jan 2015 00:55:08 +0200 (EET)
>
> > * PCI I/O - Solaris tells there is pcicmu0 at root: SAFARI 0x8 0x4000 
> > and px1 at root: SAFARI 0x1 0x700000 in the M4000. Am I correct if I
> > think support for this bridge needs to be implemented? There are some 
> > references to Safari in linux/arch/sparc/kernel code but Illumos seems 
> > to have more complex code.
> 
> There are two different kinds of PCI controllers in the machine.
> 
> One PCI controller has a vendor ID of Fujitsu so it looks like it's
> their own ASIC.  You therefore might need to write a PCI controller
> driver for it.

"pci10cf,138f" and "pci10cf,1390" are actually variants of good old
Psycho, so in OpenBSD it is handled by the psycho(4) driver.  There's
no IOMMU, but there are no devices behind it that need do DMA anyway.

> The other is a Sun chip with device ID 80f8, it might be compatible
> with the arch/sparc/kernel/pci_fire.c driver so simply adding an
> entry like:
> 
> {
> 	.name = "pci",
> 	      .compatible = "pciex108e,80f8",
> },
> 
> to fire_match[] might work.

There are some subtle differences between "Oberon" and "Fire", the
most prominent one being how the target CPU is encoded in the
interrupt mapping registers,  Handled by pyro(4) in OpenBSD.

>> * OpenBSD guys had a problem with some specific invalid hardware access 
>> causing fault in xscf firmware that needed to be cleared by service 
>> engineer, assuming there is a service contract. My Fujitsus are in my 
>> museum so there is no service contract. Has anyone heard if the current 
>> firmwares are more reliable? I'm running the latest on M4000 and 
>> whatever came with the PrimePowers.

The problems seemed to be caused by trying to push the serial port too
hard.  Pretending the serial chip had no FIFO, pushing the characters
one at a time made the problem disappear.  Seems like the firmware on
the machine was detecting some sort of overflow and overreacted a bit
by faulting the entire chassis ;).

I know that there are newer firmware versions for the M3000 that allow
customers to clear these faults themselves.  You might want to check
whether the M4000 firmware allows this as well before starting to hack
on it.

I never encountered problems like that on the PrimePower machines.

Cheers,

Mark

Re: Adding support for Fujitsu SPARC64 machines?

[David Miller]

From: Mark Kettenis <mark.kettenis@xs4all.nl>
Date: Mon, 12 Jan 2015 22:48:59 +0100 (CET)

> From: David Miller <davem@davemloft.net>
> Data: Mon, 12 Jan 2015 20:53:30
> 
>> From: Meelis Roos <mroos@linux.ee>
>> Date: Thu, 8 Jan 2015 00:55:08 +0200 (EET)
>>
>> > * PCI I/O - Solaris tells there is pcicmu0 at root: SAFARI 0x8 0x4000 
>> > and px1 at root: SAFARI 0x1 0x700000 in the M4000. Am I correct if I
>> > think support for this bridge needs to be implemented? There are some 
>> > references to Safari in linux/arch/sparc/kernel code but Illumos seems 
>> > to have more complex code.
>> 
>> There are two different kinds of PCI controllers in the machine.
>> 
>> One PCI controller has a vendor ID of Fujitsu so it looks like it's
>> their own ASIC.  You therefore might need to write a PCI controller
>> driver for it.
> 
> "pci10cf,138f" and "pci10cf,1390" are actually variants of good old
> Psycho, so in OpenBSD it is handled by the psycho(4) driver.  There's
> no IOMMU, but there are no devices behind it that need do DMA anyway.

Indeed, only the EBUS is behind this PCI controller.

Re: Adding support for Fujitsu SPARC64 machines?

[David Miller]

From: Mark Kettenis <mark.kettenis@xs4all.nl>
Date: Mon, 12 Jan 2015 22:48:59 +0100 (CET)

> From: David Miller <davem@davemloft.net>
> Data: Mon, 12 Jan 2015 20:53:30
> 
>> The other is a Sun chip with device ID 80f8, it might be compatible
>> with the arch/sparc/kernel/pci_fire.c driver so simply adding an
>> entry like:
>> 
>> {
>> 	.name = "pci",
>> 	      .compatible = "pciex108e,80f8",
>> },
>> 
>> to fire_match[] might work.
> 
> There are some subtle differences between "Oberon" and "Fire", the
> most prominent one being how the target CPU is encoded in the
> interrupt mapping registers,  Handled by pyro(4) in OpenBSD.

Meelis, this is handled by sun4u_compute_tid() in arch/sparc/kernel/irq_64.c

Re: Adding support for Fujitsu SPARC64 machines?

[Xose Vazquez Perez]

Meelis Roos wrote:

> I am playing with the idea of adding support for SPARC64 V and VI CPUs 
> for sparclinux (actually maybe supervising a master student doing it)

> http://www.fujitsu.com/global/products/computing/servers/unix/sparc-enterprise/downloads/documents/ 
> has a bunch of developer docs for newer SPARC64 CPUs. Seems to be 
> detailed enough for just the CPU.

FYI, there are more chips docs at:
- Fujitsu SPARC64 V, VI, VII, VIIIfx and IXfx Extensions: http://www.fujitsu.com/global/solutions/business-technology/tc/catalog/
- Fujitsu SPARC64 X and X+ Specification: http://www.fujitsu.com/global/products/computing/servers/unix/sparc/downloads/documents/