Aspeed SuperIO runtime management

Aspeed SuperIO runtime management

[Zev Weiss]

Hello all,

I was hoping to gather any thoughts in the community on how best to deal
with a problem I've encountered on my latest OpenBMC platform port (but
which I think might be relevant to some other systems as well).

For reasons I don't fully understand but that I think are orthogonal to
this particular issue, the platform in question can't use the Aspeed
VUART, and so instead uses two SUARTs configured back-to-back via the
UART mux to provide the host's serial console.  The host's firmware thus
enables its UART early in the host boot sequence, which requires that
the AST2500's built-in SuperIO device be enabled (SCU70[20]=0).
Unfortunately that exposes the BMC to some of the CVE-2019-6260
("pantsdown") vulnerabilities, which is a pretty big downside, and one
that I'd like to minimize as much as I can.

The SuperIO only really *needs* to be enabled during the window of time
in which the host firmware performs the UART-enable sequence; once it's
up and running I can manually disable it without any adverse effects.
So what I'd ideally like is to have the BMC enabling and disabling the
SuperIO at runtime, turning it on only when it's expected to be needed
and then turning it back off so as to minimize the exposure to known
security holes (while in general I wouldn't like the BMC to consider the
BIOS/UEFI code as "trusted", it's hopefully at least less actively
hostile than whatever might be running when the host OS is booted).

To that end, what I've currently got consists of:

 1. A kernel tweak (currently hacked onto the aspeed-socinfo driver) to
    expose the SuperIO enable/disable state as a read/write sysfs file,
    and

 2. A patch to x86-power-control using that file to enable the SuperIO
    when the host's POST-complete signal is deasserted (and disable it
    when it's asserted).

Aside from being a bit of a kludge (and a fairly special-purpose one at
that), the major drawback with this approach is that it seems kind of
inherently racy.  When the host resets and the POST-complete signal
deasserts, there's nothing synchronizing the BMC and the host to ensure
that the BMC does in fact enable the SuperIO before the host tries to
access it when it goes to enable the UART.  In the stress-testing I've
done (including swamping the BMC with artificial CPU & interrupt load) I
haven't ever seen it "lose" the race, but I don't have a terribly
accurate sense of how tight the window of time really is.

So what I'm wondering here is:

 1. Does anyone know of any better ways of handling this problem?

 2. If not and this is the best option we've got, are there better
    implementation options that might be palatable for potential
    upstreaming (more appropriate places to put the kernel side, a way
    to make the userspace side less of a hard-coded hack, etc.), or is
    this doomed by its nature to live out-of-tree?


Thanks,
Zev

Re: Aspeed SuperIO runtime management

[Michael Richardson]

[-- Attachment #1: Type: text/plain, Size: 1111 bytes --]


Zev Weiss <zweiss@equinix.com> wrote:
    > For reasons I don't fully understand but that I think are orthogonal to
    > this particular issue, the platform in question can't use the Aspeed
    > VUART, and so instead uses two SUARTs configured back-to-back via the
    > UART mux to provide the host's serial console.  The host's firmware thus
    > enables its UART early in the host boot sequence, which requires that
    > the AST2500's built-in SuperIO device be enabled (SCU70[20]=0).

I don't really understand much about these SUARTs and the SuperIO.

I guess the SuperIO is supposed to now be provisioned only by the BMC,
except for this exception.
I use the word provisioned here, because I think that the host would get to
configure the UARTs speed, stop bits, etc. via it's normal process.

My question is: what if the BMC just did all the provisioning necessary and
the host either just didn't, or perhaps if it really wants to do this, that
it be faked into writing to some other thing that isn't really those
registers.

    > 1. Does anyone know of any better ways of handling this problem?



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 511 bytes --]

Re: Aspeed SuperIO runtime management

[Andrew Jeffery]

On Wed, 2023-10-04 at 00:34 +0000, Zev Weiss wrote:
> Hello all,
> 
> I was hoping to gather any thoughts in the community on how best to deal
> with a problem I've encountered on my latest OpenBMC platform port (but
> which I think might be relevant to some other systems as well).
> 
> For reasons I don't fully understand but that I think are orthogonal to
> this particular issue, the platform in question can't use the Aspeed
> VUART, and so instead uses two SUARTs configured back-to-back via the
> UART mux to provide the host's serial console.  The host's firmware thus
> enables its UART early in the host boot sequence, which requires that
> the AST2500's built-in SuperIO device be enabled (SCU70[20]=0).

Is part of the issue that it's difficult to change the initialisation 
sequence in the host firmware here? I assume host firmware doesn't 
behave very well if you disable SuperIO entirely?

If host firmware doesn't completely bail out, would the PUARTs help?

Essentially you can configure UART1 and UART2 to behave as if they were
VUARTs in the sense that they're configurable from the BMC-side, but 
with the benefit (?) of still doing TTL serial signalling internally 
and being hooked up to the UART mux.

The big caveat is I haven't actually tried this myself so have no 
feel for the potential traps, and may in-fact be imagining the
functionality. However, see the description of the PUARTs at the end of
43.1 (pg 669) in ast2500v17.pdf

> Unfortunately that exposes the BMC to some of the CVE-2019-6260
> ("pantsdown") vulnerabilities, which is a pretty big downside, and one
> that I'd like to minimize as much as I can.
> 
> The SuperIO only really *needs* to be enabled during the window of time
> in which the host firmware performs the UART-enable sequence; once it's
> up and running I can manually disable it without any adverse effects.
> So what I'd ideally like is to have the BMC enabling and disabling the
> SuperIO at runtime, turning it on only when it's expected to be needed
> and then turning it back off so as to minimize the exposure to known
> security holes (while in general I wouldn't like the BMC to consider the
> BIOS/UEFI code as "trusted", it's hopefully at least less actively
> hostile than whatever might be running when the host OS is booted).
> 
> To that end, what I've currently got consists of:
> 
>  1. A kernel tweak (currently hacked onto the aspeed-socinfo driver) to
>     expose the SuperIO enable/disable state as a read/write sysfs file,
>     and
> 
>  2. A patch to x86-power-control using that file to enable the SuperIO
>     when the host's POST-complete signal is deasserted (and disable it
>     when it's asserted).
> 
> Aside from being a bit of a kludge (and a fairly special-purpose one at
> that), the major drawback with this approach is that it seems kind of
> inherently racy.  When the host resets and the POST-complete signal
> deasserts, there's nothing synchronizing the BMC and the host to ensure
> that the BMC does in fact enable the SuperIO before the host tries to
> access it when it goes to enable the UART.  In the stress-testing I've
> done (including swamping the BMC with artificial CPU & interrupt load) I
> haven't ever seen it "lose" the race, but I don't have a terribly
> accurate sense of how tight the window of time really is.

I assume it's not enough to live with maintaining BMC integrity but not
confidentiality by putting the iLPC2AHB into read-only mode and leaving
SuperIO enabled?

Andrew

Re: Aspeed SuperIO runtime management

[Andrew Jeffery]

On Wed, 2023-10-04 at 10:02 -0400, Michael Richardson wrote:
> 
> My question is: what if the BMC just did all the provisioning necessary and
> the host either just didn't

This is essentially how the VUARTs and PUARTs operate, but the trade-
off is you need some "platform ABI" between the host and the BMC so the
BMC can meet the host's assumptions on the peripherals.

Andrew

Re: Aspeed SuperIO runtime management

[Zev Weiss]

On Wed, Oct 04, 2023 at 03:52:53PM PDT, Andrew Jeffery wrote:
>On Wed, 2023-10-04 at 00:34 +0000, Zev Weiss wrote:
>> Hello all,
>>
>> I was hoping to gather any thoughts in the community on how best to deal
>> with a problem I've encountered on my latest OpenBMC platform port (but
>> which I think might be relevant to some other systems as well).
>>
>> For reasons I don't fully understand but that I think are orthogonal to
>> this particular issue, the platform in question can't use the Aspeed
>> VUART, and so instead uses two SUARTs configured back-to-back via the
>> UART mux to provide the host's serial console.  The host's firmware thus
>> enables its UART early in the host boot sequence, which requires that
>> the AST2500's built-in SuperIO device be enabled (SCU70[20]=0).
>
>Is part of the issue that it's difficult to change the initialisation
>sequence in the host firmware here? I assume host firmware doesn't
>behave very well if you disable SuperIO entirely?
>
>If host firmware doesn't completely bail out, would the PUARTs help?
>
>Essentially you can configure UART1 and UART2 to behave as if they were
>VUARTs in the sense that they're configurable from the BMC-side, but
>with the benefit (?) of still doing TTL serial signalling internally
>and being hooked up to the UART mux.
>
>The big caveat is I haven't actually tried this myself so have no
>feel for the potential traps, and may in-fact be imagining the
>functionality. However, see the description of the PUARTs at the end of
>43.1 (pg 669) in ast2500v17.pdf

Yeah, unfortunately I'm not able to change the host firmware, and it
doesn't take kindly to the SuperIO being disabled.  My attempts to get
it to use the PUART have likewise not led anywhere useful.

>
>> Unfortunately that exposes the BMC to some of the CVE-2019-6260
>> ("pantsdown") vulnerabilities, which is a pretty big downside, and one
>> that I'd like to minimize as much as I can.
>>
>> The SuperIO only really *needs* to be enabled during the window of time
>> in which the host firmware performs the UART-enable sequence; once it's
>> up and running I can manually disable it without any adverse effects.
>> So what I'd ideally like is to have the BMC enabling and disabling the
>> SuperIO at runtime, turning it on only when it's expected to be needed
>> and then turning it back off so as to minimize the exposure to known
>> security holes (while in general I wouldn't like the BMC to consider the
>> BIOS/UEFI code as "trusted", it's hopefully at least less actively
>> hostile than whatever might be running when the host OS is booted).
>>
>> To that end, what I've currently got consists of:
>>
>>  1. A kernel tweak (currently hacked onto the aspeed-socinfo driver) to
>>     expose the SuperIO enable/disable state as a read/write sysfs file,
>>     and
>>
>>  2. A patch to x86-power-control using that file to enable the SuperIO
>>     when the host's POST-complete signal is deasserted (and disable it
>>     when it's asserted).
>>
>> Aside from being a bit of a kludge (and a fairly special-purpose one at
>> that), the major drawback with this approach is that it seems kind of
>> inherently racy.  When the host resets and the POST-complete signal
>> deasserts, there's nothing synchronizing the BMC and the host to ensure
>> that the BMC does in fact enable the SuperIO before the host tries to
>> access it when it goes to enable the UART.  In the stress-testing I've
>> done (including swamping the BMC with artificial CPU & interrupt load) I
>> haven't ever seen it "lose" the race, but I don't have a terribly
>> accurate sense of how tight the window of time really is.
>
>I assume it's not enough to live with maintaining BMC integrity but not
>confidentiality by putting the iLPC2AHB into read-only mode and leaving
>SuperIO enabled?
>

I suppose it'd marginally better than just leaving the read-write door
wide open, but I with the options I can see at the moment I'd be more
comfortable with a race condition that my best efforts have been unable
to actually trigger if it means I can shut off the SuperIO entirely.


Thanks,
Zev