new cmdline parameter disable_cpu_features= (microcode update?)

new cmdline parameter disable_cpu_features= (microcode update?)

[Piotr Dąbrowski]

Hello!

I would like to ask about a certain thing closely related to the
kernel's Microcode Update Driver.

Some new features of CPUs (namely TSX/HLE) have caused many problems
in the past (Intel's Haswell) and still continue to be a bleeding edge
technology especially when used with not yet well tested software
[1][2][3].
I thought it would be very handy to be able to temporarily disable
unwanted features of the CPU from the kernel cmdline level. So that
underlying libraries and programs do not know they are available, and
fallback to standard code.

I was in the middle of creating a patch for the disable_cpu_features=
kernel parameter when I realized that ignoring them just in the kernel
will not solve the problem.
Unfortunately glibc does not depend on the CPU flags reported by the
kernel, but by itself queries the CPU directly for its features [4].
And so can any other user-space program.

It seems the only reliable system-wide solution is to force a CPU
microcode update. And here comes my question, as I have no experience
in this area:
Would it be possible to apply only partial (CPU flags bits) microcode
update? Or extract the whole thing, change the requested flags (and
the checksum), and re-apply it?
Is such thing even possible? And would it be a reasonable thing to do
(maybe the rest of the microcode would break badly without the factory
flags set)?
What do you think?

Thank you for any tips on this topic.

[1] glibc hardware lock elision issues:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=800574
[2] https://lists.debian.org/debian-glibc/2014/09/msg00076.html
Henrique de Moraes Holschuh <hmh@debian.org>: "I recommend that lock
elision support in glibc should be disabled for Debian jessie. This
thing doesn't look nearly stable enough in glibc 2.19, and any bugs it
might cause *will be subtle*."
[3] nvidia proprietary driver's hardware lock elision bug:
https://devtalk.nvidia.com/default/topic/893325/newest-and-beta-linux-driver-causing-segmentation-fault-core-dumped-on-all-skylake-platforms/
[4] https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/x86_64/multiarch/init-arch.h;hb=78bd7499af46d739ce94410eaeea006e874ca9e5#l175
(ex. line 175)

Best Regards,
Piotr Dąbrowski

Re: new cmdline parameter disable_cpu_features= (microcode update?)

[Borislav Petkov]

On Mon, Jan 04, 2016 at 11:55:57PM +0100, Piotr Dąbrowski wrote:
> Is such thing even possible? And would it be a reasonable thing to do
> (maybe the rest of the microcode would break badly without the factory
> flags set)?

You cannot change the microcode patches - they're supplied by the CPU
vendors as is and are signed/encrypted.

In an ideal world, we should have BIOS disable options for all CPU
features which could turn out to be problematic so that the user remains
in control. Without the need to be applying microcode patches even.

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.

Re: new cmdline parameter disable_cpu_features= (microcode update?)

[Piotr Dąbrowski]

Thank you for your reply.

> You cannot change the microcode patches - they're supplied by the CPU
> vendors as is and are signed/encrypted.

Is the microcode's header encrypted too?
I thought there are two Processor Flags fields ('pf') available [1].
Are they what I think they are?
Is the header signed too, or only the actual microcode blob below the
headers is?
Sorry if I get it all wrong and there is no use for further discussion.

Do you think there is any point in actually implementing the
kernel-only disable_cpu_features= option upstream
and then somehow convince the userland to respect flags reported by
the kernel instead of those from the CPU?

[1] arch/x86/include/asm/microcode_intel.h:
struct microcode_header_intel {
unsigned int            hdrver;
unsigned int            rev;
unsigned int            date;
unsigned int            sig;
unsigned int            cksum;
unsigned int            ldrver;
unsigned int            pf;
unsigned int            datasize;
unsigned int            totalsize;
unsigned int            reserved[3];
};
[...]
/* microcode format is extended from prescott processors */
struct extended_signature {
unsigned int            sig;
unsigned int            pf;
unsigned int            cksum;
};

Best Regards,
Piotr Dąbrowski

Re: new cmdline parameter disable_cpu_features= (microcode update?)

[Borislav Petkov]

On Tue, Jan 05, 2016 at 01:27:21AM +0100, Piotr Dąbrowski wrote:
> Is the microcode's header encrypted too?
> I thought there are two Processor Flags fields ('pf') available [1].
> Are they what I think they are?
> Is the header signed too, or only the actual microcode blob below the
> headers is?
> Sorry if I get it all wrong and there is no use for further discussion.

Yes, the microcode loader is completely wrong for what you're trying to
achieve. Just forget about it. :)

> Do you think there is any point in actually implementing the
> kernel-only disable_cpu_features= option upstream
> and then somehow convince the userland to respect flags reported by
> the kernel instead of those from the CPU?

I don't think there is any. Because you'd need to force *all* userspace
to not do CPUID but ask the kernel instead and that is a problem
because older kernels won't have the newer features enabled in, say
/proc/cpuinfo, and userspace would have to fallback to CPUID or older
tools won't have the code to check /proc/cpuinfo and would do CPUID
so...

This is going to be one helluva mess.

So I don't think we need to, or can, do anything here - the TSX f*ckup
was a nasty one and I'm not aware of any BIOS chicken bit with which
users can disable it. Which was a bad idea to not have it, in hindsight.
Otherwise there would *not* have been a microcode patch in the first
place. Which, for itself, was a pain to distribute and apply everywhere,
as you've seen.

And if we had a BIOS chicken bit, people would go into the BIOS, turn
TSX off, the CPUID bit would be clear too and userspace would've cleanly
fallen back to the old implementation and things would've worked
smoothly.

Now, if the kernel could control which CPUID bits are set and cleared,
then disable_cpu_features= would work. Unfortunately, this is not the
case. And this would require that all CPU features have corresponding
CPUID bits. Which is not the case either. :-\

Which brings me back to the BIOS chicken bits... :-)

-- 
Regards/Gruss,
    Boris.

ECO tip #101: Trim your mails when you reply.