[tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[tip-bot for H. Peter Anvin]

Commit-ID:  b3b42ac2cbae1f3cecbb6229964a4d48af31d382
Gitweb:     http://git.kernel.org/tip/b3b42ac2cbae1f3cecbb6229964a4d48af31d382
Author:     H. Peter Anvin <hpa@linux.intel.com>
AuthorDate: Sun, 16 Mar 2014 15:31:54 -0700
Committer:  H. Peter Anvin <hpa@linux.intel.com>
CommitDate: Fri, 11 Apr 2014 10:10:09 -0700

x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

The IRET instruction, when returning to a 16-bit segment, only
restores the bottom 16 bits of the user space stack pointer.  We have
a software workaround for that ("espfix") for the 32-bit kernel, but
it relies on a nonzero stack segment base which is not available in
32-bit mode.

Since 16-bit support is somewhat crippled anyway on a 64-bit kernel
(no V86 mode), and most (if not quite all) 64-bit processors support
virtualization for the users who really need it, simply reject
attempts at creating a 16-bit segment when running on top of a 64-bit
kernel.

Cc: Linus Torvalds <torvalds@linux-foundation.org>
Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Link: http://lkml.kernel.org/n/tip-kicdm89kzw9lldryb1br9od0@git.kernel.org
Cc: <stable@vger.kernel.org>
---
 arch/x86/kernel/ldt.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c
index ebc9873..af1d14a 100644
--- a/arch/x86/kernel/ldt.c
+++ b/arch/x86/kernel/ldt.c
@@ -229,6 +229,17 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode)
 		}
 	}
 
+	/*
+	 * On x86-64 we do not support 16-bit segments due to
+	 * IRET leaking the high bits of the kernel stack address.
+	 */
+#ifdef CONFIG_X86_64
+	if (!ldt_info.seg_32bit) {
+		error = -EINVAL;
+		goto out_unlock;
+	}
+#endif
+
 	fill_ldt(&ldt, &ldt_info);
 	if (oldmode)
 		ldt.avl = 0;

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andy Lutomirski]

On 04/11/2014 10:36 AM, tip-bot for H. Peter Anvin wrote:
> Commit-ID:  b3b42ac2cbae1f3cecbb6229964a4d48af31d382
> Gitweb:     http://git.kernel.org/tip/b3b42ac2cbae1f3cecbb6229964a4d48af31d382
> Author:     H. Peter Anvin <hpa@linux.intel.com>
> AuthorDate: Sun, 16 Mar 2014 15:31:54 -0700
> Committer:  H. Peter Anvin <hpa@linux.intel.com>
> CommitDate: Fri, 11 Apr 2014 10:10:09 -0700
> 
> x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels
> 
> The IRET instruction, when returning to a 16-bit segment, only
> restores the bottom 16 bits of the user space stack pointer.  We have
> a software workaround for that ("espfix") for the 32-bit kernel, but
> it relies on a nonzero stack segment base which is not available in
> 32-bit mode.
> 
> Since 16-bit support is somewhat crippled anyway on a 64-bit kernel
> (no V86 mode), and most (if not quite all) 64-bit processors support
> virtualization for the users who really need it, simply reject
> attempts at creating a 16-bit segment when running on top of a 64-bit
> kernel.
> 
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
> Link: http://lkml.kernel.org/n/tip-kicdm89kzw9lldryb1br9od0@git.kernel.org
> Cc: <stable@vger.kernel.org>

If this is what I think it is (hi, Spender), then it is probably only
useful for 3.14.y and not earlier kernels.

--Andy

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 11:12 AM, Andy Lutomirski wrote:
> 
> If this is what I think it is (hi, Spender), then it is probably only
> useful for 3.14.y and not earlier kernels.
> 

Not really.  The kernel stack address is sensitive regardless of kASLR;
in fact, it is completely orthogonal to kASLR.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

Is this bug really still present in modern CPUs?  This change breaks
running 16-bit apps in Wine.  I have a few really old games I like to
play on occasion, and I don't have a copy of Win 3.11 to put in a VM.

On Fri, Apr 11, 2014 at 1:36 PM, tip-bot for H. Peter Anvin
<tipbot@zytor.com> wrote:
> Commit-ID:  b3b42ac2cbae1f3cecbb6229964a4d48af31d382
> Gitweb:     http://git.kernel.org/tip/b3b42ac2cbae1f3cecbb6229964a4d48af31d382
> Author:     H. Peter Anvin <hpa@linux.intel.com>
> AuthorDate: Sun, 16 Mar 2014 15:31:54 -0700
> Committer:  H. Peter Anvin <hpa@linux.intel.com>
> CommitDate: Fri, 11 Apr 2014 10:10:09 -0700
>
> x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels
>
> The IRET instruction, when returning to a 16-bit segment, only
> restores the bottom 16 bits of the user space stack pointer.  We have
> a software workaround for that ("espfix") for the 32-bit kernel, but
> it relies on a nonzero stack segment base which is not available in
> 32-bit mode.
>
> Since 16-bit support is somewhat crippled anyway on a 64-bit kernel
> (no V86 mode), and most (if not quite all) 64-bit processors support
> virtualization for the users who really need it, simply reject
> attempts at creating a 16-bit segment when running on top of a 64-bit
> kernel.
>
> Cc: Linus Torvalds <torvalds@linux-foundation.org>
> Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
> Link: http://lkml.kernel.org/n/tip-kicdm89kzw9lldryb1br9od0@git.kernel.org
> Cc: <stable@vger.kernel.org>
> ---
>  arch/x86/kernel/ldt.c | 11 +++++++++++
>  1 file changed, 11 insertions(+)
>
> diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c
> index ebc9873..af1d14a 100644
> --- a/arch/x86/kernel/ldt.c
> +++ b/arch/x86/kernel/ldt.c
> @@ -229,6 +229,17 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode)
>                 }
>         }
>
> +       /*
> +        * On x86-64 we do not support 16-bit segments due to
> +        * IRET leaking the high bits of the kernel stack address.
> +        */
> +#ifdef CONFIG_X86_64
> +       if (!ldt_info.seg_32bit) {
> +               error = -EINVAL;
> +               goto out_unlock;
> +       }
> +#endif
> +
>         fill_ldt(&ldt, &ldt_info);
>         if (oldmode)
>                 ldt.avl = 0;
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 11:27 AM, Brian Gerst wrote:
> Is this bug really still present in modern CPUs?  This change breaks
> running 16-bit apps in Wine.  I have a few really old games I like to
> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.

It is not a bug, per se, but an architectural definition issue, and it
is present in all x86 processors from all vendors.

Yes, it does break running 16-bit apps in Wine, although Wine could be
modified to put 16-bit apps in a container.  However, this is at best a
marginal use case.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

On Fri, Apr 11, 2014 at 2:29 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> On 04/11/2014 11:27 AM, Brian Gerst wrote:
>> Is this bug really still present in modern CPUs?  This change breaks
>> running 16-bit apps in Wine.  I have a few really old games I like to
>> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.
>
> It is not a bug, per se, but an architectural definition issue, and it
> is present in all x86 processors from all vendors.
>
> Yes, it does break running 16-bit apps in Wine, although Wine could be
> modified to put 16-bit apps in a container.  However, this is at best a
> marginal use case.


Marginal or not, it is still userspace breakage.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

On Fri, Apr 11, 2014 at 11:27 AM, Brian Gerst <brgerst@gmail.com> wrote:
> Is this bug really still present in modern CPUs?  This change breaks
> running 16-bit apps in Wine.  I have a few really old games I like to
> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.

Ok, so you actually do this on x86-64, and it currently works? For
some reason I thought that 16-bit windows apps already didn't work.

Because if we have working users of this, then I don't think we can do
the "we don't support 16-bit segments", or at least we need to make it
runtime configurable.

             Linus

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

On Fri, Apr 11, 2014 at 2:41 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, Apr 11, 2014 at 11:27 AM, Brian Gerst <brgerst@gmail.com> wrote:
>> Is this bug really still present in modern CPUs?  This change breaks
>> running 16-bit apps in Wine.  I have a few really old games I like to
>> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.
>
> Ok, so you actually do this on x86-64, and it currently works? For
> some reason I thought that 16-bit windows apps already didn't work.
>
> Because if we have working users of this, then I don't think we can do
> the "we don't support 16-bit segments", or at least we need to make it
> runtime configurable.
>
>              Linus

I haven't tested it recently but I do know it has worked on 64-bit
kernels.  There is no reason for it not to, the only thing not
supported in long mode is vm86.  16-bit protected mode is unchanged.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 11:41 AM, Linus Torvalds wrote:
> 
> Ok, so you actually do this on x86-64, and it currently works? For
> some reason I thought that 16-bit windows apps already didn't work.
> 

Some will work, because not all 16-bit software care about the upper
half of ESP getting randomly corrupted.

That is the "functionality bit" of the problem.  The other bit, of
course, is that that random corruption is the address of the kernel stack.

> Because if we have working users of this, then I don't think we can do
> the "we don't support 16-bit segments", or at least we need to make it
> runtime configurable.

I'll let you pick what the policy should be here.  I personally think
that we have to be able to draw a line somewhere sometimes (Microsoft
themselves haven't supported running 16-bit binaries for several Windows
generations now), but it is your policy, not mine.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com> wrote:
>
> I haven't tested it recently but I do know it has worked on 64-bit
> kernels.  There is no reason for it not to, the only thing not
> supported in long mode is vm86.  16-bit protected mode is unchanged.

Afaik 64-bit windows doesn't support 16-bit binaries, so I just
assumed Wine wouldn't do it either on x86-64. Not for any real
technical reasons, though.

HOWEVER. I'd like to hear something more definitive than "I haven't
tested recently". The "we don't break user space" is about having
actual real *users*, not about test programs.

Are there people actually using 16-bit old windows programs under
wine? That's what matters.

                Linus

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andy Lutomirski]

On 04/11/2014 11:29 AM, H. Peter Anvin wrote:
> On 04/11/2014 11:27 AM, Brian Gerst wrote:
>> Is this bug really still present in modern CPUs?  This change breaks
>> running 16-bit apps in Wine.  I have a few really old games I like to
>> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.
> 
> It is not a bug, per se, but an architectural definition issue, and it
> is present in all x86 processors from all vendors.
> 
> Yes, it does break running 16-bit apps in Wine, although Wine could be
> modified to put 16-bit apps in a container.  However, this is at best a
> marginal use case.

I wonder if there's an easy-ish good-enough fix:

Allocate some percpu space in the fixmap.  (OK, this is ugly, but
kvmclock already does it, so it's possible.)  To return to 16-bit
userspace, make sure interrupts are off, copy the whole iret descriptor
to the current cpu's fixmap space, change rsp to point to that space,
and then do the iret.

This won't restore the correct value to the high bits of [er]sp, but it
will at least stop leaking anything interesting to userspace.

--Andy

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 02:16 PM, Andy Lutomirski wrote:
> On 04/11/2014 11:29 AM, H. Peter Anvin wrote:
>> On 04/11/2014 11:27 AM, Brian Gerst wrote:
>>> Is this bug really still present in modern CPUs?  This change breaks
>>> running 16-bit apps in Wine.  I have a few really old games I like to
>>> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.
>>
>> It is not a bug, per se, but an architectural definition issue, and it
>> is present in all x86 processors from all vendors.
>>
>> Yes, it does break running 16-bit apps in Wine, although Wine could be
>> modified to put 16-bit apps in a container.  However, this is at best a
>> marginal use case.
> 
> I wonder if there's an easy-ish good-enough fix:
> 
> Allocate some percpu space in the fixmap.  (OK, this is ugly, but
> kvmclock already does it, so it's possible.)  To return to 16-bit
> userspace, make sure interrupts are off, copy the whole iret descriptor
> to the current cpu's fixmap space, change rsp to point to that space,
> and then do the iret.
> 
> This won't restore the correct value to the high bits of [er]sp, but it
> will at least stop leaking anything interesting to userspace.
> 

This would fix the infoleak, at the cost of allocating a chunk of memory
for each CPU.  It doesn't fix the functionality problem.

If we're going to do a workaround I would prefer to do something that
fixes both, but it is highly nontrivial.

This is a writeup I did to a select audience before this was public:

> Hello,
> 
> It appears we have an information leak on x86-64 by which at least bits
> [31:16] of the kernel stack address leaks to user space (some silicon
> including the 64-bit Pentium 4 leaks [63:16]).  This is due to the the
> behavior of IRET when returning to a 16-bit segment: IRET restores only
> the bottom 16 bits of the stack pointer.
> 
> This is known on 32 bits and we, in fact, have a workaround for it
> ("espfix") there.  We do not, however, have the equivalent on 64 bits,
> nor does it seem that it is very easy to construct a workaround (see below.)
> 
> This is both a functionality problem (16-bit code gets the upper bits of
> %esp corrupted when the kernel is invoked) and an information leak.  The
> 32-bit workaround was labeled as a fix for the functionality problem,
> but it of course also addresses the leak.
> 
> On 64 bits, the easiest mitigation seems to be to make modify_ldt()
> refuse to install a 16-bit segment when running on a 64-bit kernel.
> 16-bit support is already somewhat crippled on 64 bits since there is no
> V86 support; obviously, for "full service" support we can always set up
> a virtual machine -- most (but sadly, not all) 64-bit parts are also
> virtualization capable.
> 
> I would have suggested rejecting modify_ldt() entirely, to reduce attack
> surface, except that some early versions of 32-bit NPTL glibc use
> modify_ldt() to exclusion of all other methods of establishing the
> thread pointer, so in order to stay compatible with those we would need
> to allow 32-bit segments via modify_ldt() still.
> 
> However, there is no doubt this will break some legitimate users of
> 16-bit segments, e.g. Wine for 16-bit Windows apps (which don't work on
> 64-bit Windows either, for what it is worth.)
> 
> We may very well have other infoleaks that dwarf this, but the kernel
> stack address is a relatively high value item for exploits.
> 
> Some workarounds I have considered:
> 
> a. Using paging in a similar way to the 32-bit segment base workaround
> 
> This one requires a very large swath of virtual user space (depending on
> allocation policy, as much as 4 GiB per CPU.)  The "per CPU" requirement
> comes in as locking is not feasible -- as we return to user space there
> is nowhere to release the lock.
> 
> b. Return to user space via compatibility mode
> 
> As the kernel lives above the 4 GiB virtual mark, a transition through
> compatibility mode is not practical.  This would require the kernel to
> reserve virtual address space below the 4 GiB mark, which may interfere
> with the application, especially an application launched as a 64-bit
> application.
> 
> c. Trampoline in kernel space
> 
> A trampoline in kernel space is not feasible since all ring transition
> instructions capable of returning to 16-bit mode require the use of the
> stack.
> 
> d. Trampoline in user space
> 
> A return to the vdso with values set up in registers r8-r15 would enable
> a trampoline in user space.  Unfortunately there is no way
> to do a far JMP entirely with register state so this would require
> touching user space memory, possibly in an unsafe manner.
> 
> The most likely variant is to use the address of the 16-bit user stack
> and simply hope that this is a safe thing to do.
> 
> This appears to be the most feasible workaround if a workaround is
> deemed necessary.
> 
> e. Transparently run 16-bit code segments inside a lightweight VMM
> 
> The complexity of this solution versus the realized value is staggering.
> It also doesn't work on non-virtualization-capable hardware (including
> running on top of a VMM which doesn't support nested virtualization.)
> 
> 	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

On Fri, Apr 11, 2014 at 2:16 PM, Andy Lutomirski <luto@amacapital.net> wrote:
>
> I wonder if there's an easy-ish good-enough fix:

Heh. Yes. Check the thread on lkml about three weeks ago under the
subject "x86-64: Information leak: kernel stack address leaks to user
space". It had exactly that as a suggestion.

Anyway, I ended up pulling the current change - let's see if anybody even cares.

And if somebody *does* care, maybe we can just do a trivial sysctl. If
you are running 16-bit apps under wine, the default kernel setup
already stops you: the 'mmap_min_addr' being non-zero means that that
already will not run.

But yeah, I personally don't care about the high bits of rsp one whit,
since that has never worked on x86-64. But the information leak needs
to be plugged, and a percpu stack can fix that.

I'm a bit worried that a percpu stack can cause issues with NMI's,
which already have too much complexity in them, so I don't think it's
*entirely* trivial to do. And the exception that the 'iretq' can take
adds more complexity wrt kernel stack pointer games. Which is why I'm
not at all sure it's worth it.

              Linus

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andy Lutomirski]

On 04/11/2014 02:24 PM, H. Peter Anvin wrote:
> On 04/11/2014 02:16 PM, Andy Lutomirski wrote:
>> I wonder if there's an easy-ish good-enough fix:
>>
>> Allocate some percpu space in the fixmap.  (OK, this is ugly, but
>> kvmclock already does it, so it's possible.)  To return to 16-bit
>> userspace, make sure interrupts are off, copy the whole iret descriptor
>> to the current cpu's fixmap space, change rsp to point to that space,
>> and then do the iret.
>>
>> This won't restore the correct value to the high bits of [er]sp, but it
>> will at least stop leaking anything interesting to userspace.
>>
> 
> This would fix the infoleak, at the cost of allocating a chunk of memory
> for each CPU.  It doesn't fix the functionality problem.
> 
> If we're going to do a workaround I would prefer to do something that
> fixes both, but it is highly nontrivial.
> 
> This is a writeup I did to a select audience before this was public:
> 
>> Hello,
>>
>> This is both a functionality problem (16-bit code gets the upper bits of
>> %esp corrupted when the kernel is invoked) and an information leak.  The
>> 32-bit workaround was labeled as a fix for the functionality problem,
>> but it of course also addresses the leak.

How big of a functionality problem is it?  Apparently it doesn't break
16-bit code on wine.

Since the high bits of esp have been corrupted on x86_64 since the
beginning, there's no regression issue here if an eventual fix writes
less meaningful crap to those bits -- I see no real reason to try to put
the correct values in there.


>> I would have suggested rejecting modify_ldt() entirely, to reduce attack
>> surface, except that some early versions of 32-bit NPTL glibc use
>> modify_ldt() to exclusion of all other methods of establishing the
>> thread pointer, so in order to stay compatible with those we would need
>> to allow 32-bit segments via modify_ldt() still.

I actually use modify_ldt for amusement: it's the only way I know of to
issue real 32-bit syscalls from 64-bit userspace.  Yes, this isn't
really a legitimate use case.

>>
>> a. Using paging in a similar way to the 32-bit segment base workaround
>>
>> This one requires a very large swath of virtual user space (depending on
>> allocation policy, as much as 4 GiB per CPU.)  The "per CPU" requirement
>> comes in as locking is not feasible -- as we return to user space there
>> is nowhere to release the lock.

Why not just 4k per CPU?  Write the pfn to the pte, invlpg, update rsp,
iret.  This leaks the CPU number, but that's all.

To me, this sounds like the easiest solution, so long as rsp is known to
be sufficiently far from a page boundary.

These ptes could even be read-only to limit the extra exposure to
known-address attacks.

If you want a fully correct solution, you can use a fancier allocation
policy that can fit quite a few cpus per 4G :)

>>
>> d. Trampoline in user space
>>
>> A return to the vdso with values set up in registers r8-r15 would enable
>> a trampoline in user space.  Unfortunately there is no way
>> to do a far JMP entirely with register state so this would require
>> touching user space memory, possibly in an unsafe manner.
>>
>> The most likely variant is to use the address of the 16-bit user stack
>> and simply hope that this is a safe thing to do.
>>
>> This appears to be the most feasible workaround if a workaround is
>> deemed necessary.

Eww.

--Andy

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 02:53 PM, Andy Lutomirski wrote:
> 
> How big of a functionality problem is it?  Apparently it doesn't break
> 16-bit code on wine.
> 

It breaks *some* 16-bit code.  This is actually the reason that 32 bits
has the espfix workaround - it wasn't identified as an infoleak at the time.

> Since the high bits of esp have been corrupted on x86_64 since the
> beginning, there's no regression issue here if an eventual fix writes
> less meaningful crap to those bits -- I see no real reason to try to put
> the correct values in there.

It is a regression vs. the 32-bit kernel, and if we're going to support
16-bit code we should arguably support 16-bit code correctly.

This is actually how I stumbled onto this problem in the first place: it
broke a compiler test suite for gcc -m16 I was working on.  The
workaround for *that* was to run in a VM instead.

>>> I would have suggested rejecting modify_ldt() entirely, to reduce attack
>>> surface, except that some early versions of 32-bit NPTL glibc use
>>> modify_ldt() to exclusion of all other methods of establishing the
>>> thread pointer, so in order to stay compatible with those we would need
>>> to allow 32-bit segments via modify_ldt() still.
> 
> I actually use modify_ldt for amusement: it's the only way I know of to
> issue real 32-bit syscalls from 64-bit userspace.  Yes, this isn't
> really a legitimate use case.

That's actually wrong on no less than two levels:

1. You can issue real 32-bit system calls from 64-bit user space simply
   by invoking int $0x80; it works in 64-bit mode as well.

2. Even if you want to be in 32-bit mode you can simply call via
   __USER32_CS, you don't need an LDT entry.

> Why not just 4k per CPU?  Write the pfn to the pte, invlpg, update rsp,
> iret.  This leaks the CPU number, but that's all.
> 
> To me, this sounds like the easiest solution, so long as rsp is known to
> be sufficiently far from a page boundary.
> 
> These ptes could even be read-only to limit the extra exposure to
> known-address attacks.
> 
> If you want a fully correct solution, you can use a fancier allocation
> policy that can fit quite a few cpus per 4G :)

It's damned hard, because you don't have a logical place to
*deallocate*.  That's what ends up killing you.

Also, you will need to port over the equivalent to the espfix recovery
code from 32 bits (what happens if IRET takes an exception), so it is
nontrivial.

>>> d. Trampoline in user space
>>>
>>> A return to the vdso with values set up in registers r8-r15 would enable
>>> a trampoline in user space.  Unfortunately there is no way
>>> to do a far JMP entirely with register state so this would require
>>> touching user space memory, possibly in an unsafe manner.
>>>
>>> The most likely variant is to use the address of the 16-bit user stack
>>> and simply hope that this is a safe thing to do.
>>>
>>> This appears to be the most feasible workaround if a workaround is
>>> deemed necessary.
> 
> Eww.

I don't think any of the options are anything but.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andy Lutomirski]

On Fri, Apr 11, 2014 at 2:59 PM, H. Peter Anvin <hpa@linux.intel.com> wrote:
> On 04/11/2014 02:53 PM, Andy Lutomirski wrote:
>>
>> How big of a functionality problem is it?  Apparently it doesn't break
>> 16-bit code on wine.
>>
>
> It breaks *some* 16-bit code.  This is actually the reason that 32 bits
> has the espfix workaround - it wasn't identified as an infoleak at the time.
>
>> Since the high bits of esp have been corrupted on x86_64 since the
>> beginning, there's no regression issue here if an eventual fix writes
>> less meaningful crap to those bits -- I see no real reason to try to put
>> the correct values in there.
>
> It is a regression vs. the 32-bit kernel, and if we're going to support
> 16-bit code we should arguably support 16-bit code correctly.
>
> This is actually how I stumbled onto this problem in the first place: it
> broke a compiler test suite for gcc -m16 I was working on.  The
> workaround for *that* was to run in a VM instead.
>
>>>> I would have suggested rejecting modify_ldt() entirely, to reduce attack
>>>> surface, except that some early versions of 32-bit NPTL glibc use
>>>> modify_ldt() to exclusion of all other methods of establishing the
>>>> thread pointer, so in order to stay compatible with those we would need
>>>> to allow 32-bit segments via modify_ldt() still.
>>
>> I actually use modify_ldt for amusement: it's the only way I know of to
>> issue real 32-bit syscalls from 64-bit userspace.  Yes, this isn't
>> really a legitimate use case.
>
> That's actually wrong on no less than two levels:
>
> 1. You can issue real 32-bit system calls from 64-bit user space simply
>    by invoking int $0x80; it works in 64-bit mode as well.
>
> 2. Even if you want to be in 32-bit mode you can simply call via
>    __USER32_CS, you don't need an LDT entry.

I just looked up my hideous code.  I was doing this to test the
now-deleted int 0xcc vsyscall stuff.  I used modify_ldt because either
I didn't realize that __USER32_CS was usable or I didn't think it was
ABI.  Or I was just being silly.

But yes, breaking my hack would not matter. :)

--Andy

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 03:15 PM, Andy Lutomirski wrote:
> 
> I just looked up my hideous code.  I was doing this to test the
> now-deleted int 0xcc vsyscall stuff.  I used modify_ldt because either
> I didn't realize that __USER32_CS was usable or I didn't think it was
> ABI.  Or I was just being silly.
> 
> But yes, breaking my hack would not matter. :)
> 

Either way, it wouldn't break it.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

On Fri, Apr 11, 2014 at 2:50 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>
>> I haven't tested it recently but I do know it has worked on 64-bit
>> kernels.  There is no reason for it not to, the only thing not
>> supported in long mode is vm86.  16-bit protected mode is unchanged.
>
> Afaik 64-bit windows doesn't support 16-bit binaries, so I just
> assumed Wine wouldn't do it either on x86-64. Not for any real
> technical reasons, though.
>
> HOWEVER. I'd like to hear something more definitive than "I haven't
> tested recently". The "we don't break user space" is about having
> actual real *users*, not about test programs.
>
> Are there people actually using 16-bit old windows programs under
> wine? That's what matters.
>
>                 Linus

I just verified that the game does still run on a 64-bit kernel
(3.13.8-200.fc20.x86_64).  It needed an older version of Wine, but
that's a Wine regression and not kernel related.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

So Wine regressed and noone noticed? They doesn't sound like an active user base.

On April 11, 2014 9:44:22 PM PDT, Brian Gerst <brgerst@gmail.com> wrote:
>On Fri, Apr 11, 2014 at 2:50 PM, Linus Torvalds
><torvalds@linux-foundation.org> wrote:
>> On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com>
>wrote:
>>>
>>> I haven't tested it recently but I do know it has worked on 64-bit
>>> kernels.  There is no reason for it not to, the only thing not
>>> supported in long mode is vm86.  16-bit protected mode is unchanged.
>>
>> Afaik 64-bit windows doesn't support 16-bit binaries, so I just
>> assumed Wine wouldn't do it either on x86-64. Not for any real
>> technical reasons, though.
>>
>> HOWEVER. I'd like to hear something more definitive than "I haven't
>> tested recently". The "we don't break user space" is about having
>> actual real *users*, not about test programs.
>>
>> Are there people actually using 16-bit old windows programs under
>> wine? That's what matters.
>>
>>                 Linus
>
>I just verified that the game does still run on a 64-bit kernel
>(3.13.8-200.fc20.x86_64).  It needed an older version of Wine, but
>that's a Wine regression and not kernel related.

-- 
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Sat, Apr 12, 2014 at 10:18:25AM -0700, H. Peter Anvin wrote:
> So Wine regressed and noone noticed? They doesn't sound like an active
> user base.

Btw, wouldn't this obscure use case simply work in a KVM guest with a
kernel <= 3.14?

Because if so, we simply cut it at 3.14, everything newer has the leak
fix and people who still want to play phone games on a x86 machine, can
do so in a guest with an older kernel. Everybody's happy.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

Run a 32-bit VM.  The 32-bit kernel does this right.

I suspect it would also work fine in a Qemu user mode guest (is this supported by KVM?), in a ReactOS VM, or some other number of combinations.

The real question is how many real users are actually affected.

On April 12, 2014 12:35:41 PM PDT, Borislav Petkov <bp@alien8.de> wrote:
>On Sat, Apr 12, 2014 at 10:18:25AM -0700, H. Peter Anvin wrote:
>> So Wine regressed and noone noticed? They doesn't sound like an
>active
>> user base.
>
>Btw, wouldn't this obscure use case simply work in a KVM guest with a
>kernel <= 3.14?
>
>Because if so, we simply cut it at 3.14, everything newer has the leak
>fix and people who still want to play phone games on a x86 machine, can
>do so in a guest with an older kernel. Everybody's happy.

-- 
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Sat, Apr 12, 2014 at 12:44:42PM -0700, H. Peter Anvin wrote:
> Run a 32-bit VM.  The 32-bit kernel does this right.

Yes, even better.

> I suspect it would also work fine in a Qemu user mode guest (is
> this supported by KVM?), in a ReactOS VM, or some other number of
> combinations.

Right.

So basically, there a lot of different virt scenarios which can all take
care of those use cases *without* encumbering some insane solutions on
64-bit.

> The real question is how many real users are actually affected.

And if they are, virtualize them, for chrissake. It is time we finally
used virt for maybe one of its major use cases - virtualize old/obscure
hw. It should be pretty reliable by now.

:-P

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

For this particular game, not 16-bit in general.   The installer, also
16-bit, runs perfectly.  Already filed wine bug 35977.

On Sat, Apr 12, 2014 at 1:18 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> So Wine regressed and noone noticed? They doesn't sound like an active user base.
>
> On April 11, 2014 9:44:22 PM PDT, Brian Gerst <brgerst@gmail.com> wrote:
>>On Fri, Apr 11, 2014 at 2:50 PM, Linus Torvalds
>><torvalds@linux-foundation.org> wrote:
>>> On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com>
>>wrote:
>>>>
>>>> I haven't tested it recently but I do know it has worked on 64-bit
>>>> kernels.  There is no reason for it not to, the only thing not
>>>> supported in long mode is vm86.  16-bit protected mode is unchanged.
>>>
>>> Afaik 64-bit windows doesn't support 16-bit binaries, so I just
>>> assumed Wine wouldn't do it either on x86-64. Not for any real
>>> technical reasons, though.
>>>
>>> HOWEVER. I'd like to hear something more definitive than "I haven't
>>> tested recently". The "we don't break user space" is about having
>>> actual real *users*, not about test programs.
>>>
>>> Are there people actually using 16-bit old windows programs under
>>> wine? That's what matters.
>>>
>>>                 Linus
>>
>>I just verified that the game does still run on a 64-bit kernel
>>(3.13.8-200.fc20.x86_64).  It needed an older version of Wine, but
>>that's a Wine regression and not kernel related.
>
> --
> Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

On Sat, Apr 12, 2014 at 4:11 PM, Borislav Petkov <bp@alien8.de> wrote:
> On Sat, Apr 12, 2014 at 12:44:42PM -0700, H. Peter Anvin wrote:
>> Run a 32-bit VM.  The 32-bit kernel does this right.
>
> Yes, even better.
>
>> I suspect it would also work fine in a Qemu user mode guest (is
>> this supported by KVM?), in a ReactOS VM, or some other number of
>> combinations.
>
> Right.
>
> So basically, there a lot of different virt scenarios which can all take
> care of those use cases *without* encumbering some insane solutions on
> 64-bit.
>
>> The real question is how many real users are actually affected.
>
> And if they are, virtualize them, for chrissake. It is time we finally
> used virt for maybe one of its major use cases - virtualize old/obscure
> hw. It should be pretty reliable by now.
>
> :-P
>
> --
> Regards/Gruss,
>     Boris.
>
> Sent from a fat crate under my desk. Formatting is fine.
> --

My experience with kvm so far is that is slow and clunky.  It may be
OK for a server environment, but interactively it's difficult to use.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Sat, Apr 12, 2014 at 04:34:14PM -0400, Brian Gerst wrote:
> My experience with kvm so far is that is slow and clunky. It may be OK
> for a server environment, but interactively it's difficult to use.

Are you saying, you've run your game in a guest and perf. is sucky?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Brian Gerst]

On Sat, Apr 12, 2014 at 4:59 PM, Borislav Petkov <bp@alien8.de> wrote:
> On Sat, Apr 12, 2014 at 04:34:14PM -0400, Brian Gerst wrote:
>> My experience with kvm so far is that is slow and clunky. It may be OK
>> for a server environment, but interactively it's difficult to use.
>
> Are you saying, you've run your game in a guest and perf. is sucky?
>

Performance is bad in general, running a 32-bit Fedora 20 guest.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Sat, Apr 12, 2014 at 05:13:40PM -0400, Brian Gerst wrote:
> Performance is bad in general, running a 32-bit Fedora 20 guest.

So this means you haven't tried the game in the guest yet, so that we
can know for sure that a guest doesn't solve your problem or what?

Btw, which game is that and can I get it somewhere to try it here
locally?

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

On Sat, Apr 12, 2014 at 12:44 PM, H. Peter Anvin <hpa@zytor.com> wrote:
> Run a 32-bit VM.  The 32-bit kernel does this right.

I really don't think that's the answer.

If people really run these 16-bit programs, we need to allow it.
Clearly it used to work.

Just make the unconditional "don't allow 16-bit segments" be a sysconf entry.

          Linus

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/12/2014 02:53 PM, Linus Torvalds wrote:
> On Sat, Apr 12, 2014 at 12:44 PM, H. Peter Anvin <hpa@zytor.com> wrote:
>> Run a 32-bit VM.  The 32-bit kernel does this right.
> 
> I really don't think that's the answer.
> 
> If people really run these 16-bit programs, we need to allow it.
> Clearly it used to work.
> 
> Just make the unconditional "don't allow 16-bit segments" be a sysconf entry.
> 

Well, is there more than one user, really... that's my question.

But yes, we can make it configurable, but the default should almost
certainly be off.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andi Kleen]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Fri, Apr 11, 2014 at 11:27 AM, Brian Gerst <brgerst@gmail.com> wrote:
>> Is this bug really still present in modern CPUs?  This change breaks
>> running 16-bit apps in Wine.  I have a few really old games I like to
>> play on occasion, and I don't have a copy of Win 3.11 to put in a VM.
>
> Ok, so you actually do this on x86-64, and it currently works? For
> some reason I thought that 16-bit windows apps already didn't work.

No, it always worked. I spent some time on this early in the x86-64 port
and it flushed out some bugs in the early segment handling. x86-64 
is perfectly compatible to this.

Also was always proud that Linux 64 was more compatible old Windows
binaries than Win 64...

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Andi Kleen]

"H. Peter Anvin" <hpa@zytor.com> writes:
>
> But yes, we can make it configurable, but the default should almost
> certainly be off.

Why? Either it works or it doesn't.

If it works it doesn't make any sense to have a sysctl.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

It leaks security sensitive information to userspace and corrupts the upper half of ESP because it lacks the equivalent of the espfix workaround.

On April 12, 2014 7:56:48 PM PDT, Andi Kleen <andi@firstfloor.org> wrote:
>"H. Peter Anvin" <hpa@zytor.com> writes:
>>
>> But yes, we can make it configurable, but the default should almost
>> certainly be off.
>
>Why? Either it works or it doesn't.
>
>If it works it doesn't make any sense to have a sysctl.
>
>-Andi

-- 
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

On Sat, Apr 12, 2014 at 7:56 PM, Andi Kleen <andi@firstfloor.org> wrote:
>
> Why? Either it works or it doesn't.
>
> If it works it doesn't make any sense to have a sysctl.

BS.

It "works" exactly like mmap() at NULL "works".

It is a potential security leak, because x86-64 screwed up the
architecture definition in this area. So it should definitely be
disabled by default, exactly like mmap_min_addr is non-zero by
default.

            Linus

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 04/11/2014 02:53 PM, Andy Lutomirski wrote:
> 
> If you want a fully correct solution, you can use a fancier allocation
> policy that can fit quite a few cpus per 4G :)
> 

The more I think about this, I think this might actually be a reasonable
option, *IF* someone is willing to deal with actually implementing it.

The difference versus my "a" alternative is rather than mapping the
existing kernel stack into an alternate part of the address space would
be that we would have a series of ministacks that is only large enough
that we can handle the IRET data *and* big enough to handle any
exceptions that IRET may throw, until we can switch back to the real
kernel stack.  Tests would have to be added to the appropriate exception
paths, as early as possible.  We would then *copy* the IRET data to the
ministack before returning.  Each ministack would be mapped 65536 times.

If we can get away with 64 bytes per CPU, then we can get away with 4
GiB of address space per 1024 CPUs, so if MAX_CPUS is 16384 we would
need 64 GiB of address space... which is not unreasonable on 64 bits.
The total memory consumption would be about 81 bytes per CPU for the
ministacks plus page tables (just over 16K per 1K CPUs.)  Again, fairly
reasonable, but a *lot* of complexity.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Ingo Molnar]

* Borislav Petkov <bp@alien8.de> wrote:

> On Sat, Apr 12, 2014 at 05:13:40PM -0400, Brian Gerst wrote:
> > Performance is bad in general, running a 32-bit Fedora 20 guest.
> 
> So this means you haven't tried the game in the guest yet, so that 
> we can know for sure that a guest doesn't solve your problem or 
> what?
> 
> Btw, which game is that and can I get it somewhere to try it here 
> locally?

Apparently the game in question is "Exile: Escape from the pit":

  http://osdir.com/ml/wine-bugs/2014-04/msg01159.html

Thanks,

        Ingo

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Ingo Molnar]

* H. Peter Anvin <hpa@linux.intel.com> wrote:

> On 04/11/2014 11:41 AM, Linus Torvalds wrote:
> > 
> > Ok, so you actually do this on x86-64, and it currently works? For
> > some reason I thought that 16-bit windows apps already didn't work.
> > 
> 
> Some will work, because not all 16-bit software care about the upper
> half of ESP getting randomly corrupted.
> 
> That is the "functionality bit" of the problem.  The other bit, of
> course, is that that random corruption is the address of the kernel stack.
> 
> > Because if we have working users of this, then I don't think we can do
> > the "we don't support 16-bit segments", or at least we need to make it
> > runtime configurable.
> 
> I'll let you pick what the policy should be here.  I personally 
> think that we have to be able to draw a line somewhere sometimes 
> (Microsoft themselves haven't supported running 16-bit binaries for 
> several Windows generations now), but it is your policy, not mine.

I think the mmap_min_addr model works pretty well:

 - it defaults to secure

 - allow a security policy to grant an exception to a known package, 
   built by the distro

 - end user can also grant an exception

This essentially punts any 'makes the system less secure' exceptions 
to the distro and the end user.

Thanks,

	Ingo

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Alexandre Julliard]

Linus Torvalds <torvalds@linux-foundation.org> writes:

> On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>
>> I haven't tested it recently but I do know it has worked on 64-bit
>> kernels.  There is no reason for it not to, the only thing not
>> supported in long mode is vm86.  16-bit protected mode is unchanged.
>
> Afaik 64-bit windows doesn't support 16-bit binaries, so I just
> assumed Wine wouldn't do it either on x86-64. Not for any real
> technical reasons, though.
>
> HOWEVER. I'd like to hear something more definitive than "I haven't
> tested recently". The "we don't break user space" is about having
> actual real *users*, not about test programs.
>
> Are there people actually using 16-bit old windows programs under
> wine? That's what matters.

Yes, there is still a significant number of users, and we still
regularly get bug reports about specific 16-bit apps. It would be really
nice if we could continue to support them on x86-64, particularly since
Microsoft doesn't ;-)

-- 
Alexandre Julliard
julliard@winehq.org

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Mon, Apr 14, 2014 at 09:21:13AM +0200, Ingo Molnar wrote:
> Apparently the game in question is "Exile: Escape from the pit":
> 
>   http://osdir.com/ml/wine-bugs/2014-04/msg01159.html

Ah, thanks.

Well, FWIW, you can get the game for free:

http://www.spiderwebsoftware.com/exile/winexile.html

I did run it on an old windoze guest I had lying around. Performance
feels like native considering this game is from the previous century :-)

https://en.wikipedia.org/wiki/Exile_%28video_game_series%29

Wikipedia says you can get it on steam too, which runs linux and stuff.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Ingo Molnar]

* Borislav Petkov <bp@alien8.de> wrote:

> On Mon, Apr 14, 2014 at 09:21:13AM +0200, Ingo Molnar wrote:
> > Apparently the game in question is "Exile: Escape from the pit":
> > 
> >   http://osdir.com/ml/wine-bugs/2014-04/msg01159.html
> 
> Ah, thanks.
> 
> Well, FWIW, you can get the game for free:
> 
> http://www.spiderwebsoftware.com/exile/winexile.html
> 
> I did run it on an old windoze guest I had lying around. Performance
> feels like native considering this game is from the previous century :-)

In fact it's from the previous millenium :-)

Thanks,

	Ingo

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

For both of these, though, it is really kind of broken that it is a global switch, whereas typically only one application on the whole system needs it, so it would be much better to have application-specific controls.  How to do that is another matter...

On April 14, 2014 12:27:56 AM PDT, Ingo Molnar <mingo@kernel.org> wrote:
>
>* H. Peter Anvin <hpa@linux.intel.com> wrote:
>
>> On 04/11/2014 11:41 AM, Linus Torvalds wrote:
>> > 
>> > Ok, so you actually do this on x86-64, and it currently works? For
>> > some reason I thought that 16-bit windows apps already didn't work.
>> > 
>> 
>> Some will work, because not all 16-bit software care about the upper
>> half of ESP getting randomly corrupted.
>> 
>> That is the "functionality bit" of the problem.  The other bit, of
>> course, is that that random corruption is the address of the kernel
>stack.
>> 
>> > Because if we have working users of this, then I don't think we can
>do
>> > the "we don't support 16-bit segments", or at least we need to make
>it
>> > runtime configurable.
>> 
>> I'll let you pick what the policy should be here.  I personally 
>> think that we have to be able to draw a line somewhere sometimes 
>> (Microsoft themselves haven't supported running 16-bit binaries for 
>> several Windows generations now), but it is your policy, not mine.
>
>I think the mmap_min_addr model works pretty well:
>
> - it defaults to secure
>
> - allow a security policy to grant an exception to a known package, 
>   built by the distro
>
> - end user can also grant an exception
>
>This essentially punts any 'makes the system less secure' exceptions 
>to the distro and the end user.
>
>Thanks,
>
>	Ingo

-- 
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Sven Joachim]

On 2014-04-14 09:48 +0200, Alexandre Julliard wrote:

> Linus Torvalds <torvalds@linux-foundation.org> writes:
>
>> On Fri, Apr 11, 2014 at 11:45 AM, Brian Gerst <brgerst@gmail.com> wrote:
>>>
>>> I haven't tested it recently but I do know it has worked on 64-bit
>>> kernels.  There is no reason for it not to, the only thing not
>>> supported in long mode is vm86.  16-bit protected mode is unchanged.
>>
>> Afaik 64-bit windows doesn't support 16-bit binaries, so I just
>> assumed Wine wouldn't do it either on x86-64. Not for any real
>> technical reasons, though.
>>
>> HOWEVER. I'd like to hear something more definitive than "I haven't
>> tested recently". The "we don't break user space" is about having
>> actual real *users*, not about test programs.
>>
>> Are there people actually using 16-bit old windows programs under
>> wine? That's what matters.

It seems that at least some 32-bit programs are also broken, since after
upgrading the kernel to 3.14.3 I can no longer start my old chess
database program:

,----
| % file CB70.exe 
| CB70.exe: PE32 executable (GUI) Intel 80386, for MS Windows
| % LANG=C wine CB70.exe
| modify_ldt: Invalid argument
| modify_ldt: Invalid argument
| modify_ldt: Invalid argument
| modify_ldt: Invalid argument
| modify_ldt: Invalid argument
`----

And here it just hangs, with wineboot.exe taking 100% CPU.  I had to
kill first wineboot.exe and then CB70.exe. :-(

> Yes, there is still a significant number of users, and we still
> regularly get bug reports about specific 16-bit apps. It would be really
> nice if we could continue to support them on x86-64, particularly since
> Microsoft doesn't ;-)

I would rather not set up a virtual machine just for wine (I don't have
Windows anymore).

Cheers,
       Sven

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Borislav Petkov]

On Wed, May 07, 2014 at 11:18:49AM +0200, Sven Joachim wrote:
> I would rather not set up a virtual machine just for wine (I don't
> have Windows anymore).

What about reactos? (I'm not saying this shouldn't be addressed,
regardless...)

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Linus Torvalds]

[-- Attachment #1: Type: text/plain, Size: 1036 bytes --]

On Wed, May 7, 2014 at 2:18 AM, Sven Joachim <svenjoac@gmx.de> wrote:
>
> It seems that at least some 32-bit programs are also broken, since after
> upgrading the kernel to 3.14.3 I can no longer start my old chess
> database program:

So for backporting (and for 3.15) maybe this (TOTALLY UNTESTED) patch
would be acceptable.

It adds a "/proc/sys/abi/ldt16" sysctl that defaults to zero (off). If
you hit this issue and care about your old Windows program more than
you care about a kernel stack address information leak, you can do

   echo 1 > /proc/sys/abi/ldt16

as root (add it to your startup scripts), and you should be ok.

Afaik, 16-bit programs under wine already need

  echo 0 > /proc/sys/vm/mmap_min_addr

because they want to map things at address 0, so this isn't a new concept.

I would like to repeat that this is totally untested. And the sysct
table is only added if you have COMPAT support enabled on x86-64, but
I assume anybody who runs old windows binaries very much does that ;)

                        Linus

[-- Attachment #2: patch.diff --]
[-- Type: text/plain, Size: 1389 bytes --]

 arch/x86/kernel/ldt.c        | 4 +++-
 arch/x86/vdso/vdso32-setup.c | 8 ++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c
index af1d14a9ebda..dcbbaa165bde 100644
--- a/arch/x86/kernel/ldt.c
+++ b/arch/x86/kernel/ldt.c
@@ -20,6 +20,8 @@
 #include <asm/mmu_context.h>
 #include <asm/syscalls.h>
 
+int sysctl_ldt16 = 0;
+
 #ifdef CONFIG_SMP
 static void flush_ldt(void *current_mm)
 {
@@ -234,7 +236,7 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode)
 	 * IRET leaking the high bits of the kernel stack address.
 	 */
 #ifdef CONFIG_X86_64
-	if (!ldt_info.seg_32bit) {
+	if (!ldt_info.seg_32bit && !sysctl_ldt16) {
 		error = -EINVAL;
 		goto out_unlock;
 	}
diff --git a/arch/x86/vdso/vdso32-setup.c b/arch/x86/vdso/vdso32-setup.c
index 00348980a3a6..e1f220e3ca68 100644
--- a/arch/x86/vdso/vdso32-setup.c
+++ b/arch/x86/vdso/vdso32-setup.c
@@ -39,6 +39,7 @@
 #ifdef CONFIG_X86_64
 #define vdso_enabled			sysctl_vsyscall32
 #define arch_setup_additional_pages	syscall32_setup_pages
+extern int sysctl_ldt16;
 #endif
 
 /*
@@ -249,6 +250,13 @@ static struct ctl_table abi_table2[] = {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec
 	},
+	{
+		.procname	= "ldt16",
+		.data		= &sysctl_ldt16,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec
+	},
 	{}
 };
 

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 05/07/2014 09:57 AM, Linus Torvalds wrote:
> 
> Afaik, 16-bit programs under wine already need
> 
>   echo 0 > /proc/sys/vm/mmap_min_addr
> 
> because they want to map things at address 0, so this isn't a new concept.
> 

I think that applies to DOSEMU, but not to Wine.

Sven: if you have the ability to build your own kernel, could you also
try the "x86/espfix" branch of the git tree:

https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/

(clone URLs:)
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git

... to make sure the proper solution works for you?

I'm somewhat curious if this program you have is actually a 32-bit
program or if it is really a 16-bit program wrapped in a 32-bit
installer of some kind.  Hard to know without seeing the program in
question.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Alexandre Julliard]

"H. Peter Anvin" <hpa@zytor.com> writes:

> On 05/07/2014 09:57 AM, Linus Torvalds wrote:
>> 
>> Afaik, 16-bit programs under wine already need
>> 
>>   echo 0 > /proc/sys/vm/mmap_min_addr
>> 
>> because they want to map things at address 0, so this isn't a new concept.
>> 
>
> I think that applies to DOSEMU, but not to Wine.

Yes, there are a few exceptions, but most Win16 apps run fine without
mapping address 0.

> I'm somewhat curious if this program you have is actually a 32-bit
> program or if it is really a 16-bit program wrapped in a 32-bit
> installer of some kind.  Hard to know without seeing the program in
> question.

It could be a mix of both, there are various thunking mechanisms that
allow 32-bit binaries to use 16-bit components. This was pretty common
in the Win95 days.

-- 
Alexandre Julliard
julliard@winehq.org

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Sven Joachim]

On 2014-05-07 19:09 +0200, H. Peter Anvin wrote:

> On 05/07/2014 09:57 AM, Linus Torvalds wrote:
>> 
>> Afaik, 16-bit programs under wine already need
>> 
>>   echo 0 > /proc/sys/vm/mmap_min_addr
>> 
>> because they want to map things at address 0, so this isn't a new concept.
>> 
>
> I think that applies to DOSEMU, but not to Wine.

DOSEMU does no longer need it either.  If vm.mmap_min_addr is > 0, it
turns on CPU emulation, which it has to use anyway due to lack of vm86
mode.

> Sven: if you have the ability to build your own kernel, could you also
> try the "x86/espfix" branch of the git tree:
>
> https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/
>
> (clone URLs:)
> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
>
> ... to make sure the proper solution works for you?

Works fine here, thanks.

> I'm somewhat curious if this program you have is actually a 32-bit
> program or if it is really a 16-bit program wrapped in a 32-bit
> installer of some kind.  Hard to know without seeing the program in
> question.

The main application (a chess database program) is 32-bit, but it comes
with several 16-bit analysis engines that are loaded on startup (I see
them in lsof output), so that's the situation described by Alexandre.

Cheers,
       Sven

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

Actually it could use KVM instead of CPU emulation on nearly all modern processors...

On May 7, 2014 11:43:59 PM PDT, Sven Joachim <svenjoac@gmx.de> wrote:
>On 2014-05-07 19:09 +0200, H. Peter Anvin wrote:
>
>> On 05/07/2014 09:57 AM, Linus Torvalds wrote:
>>> 
>>> Afaik, 16-bit programs under wine already need
>>> 
>>>   echo 0 > /proc/sys/vm/mmap_min_addr
>>> 
>>> because they want to map things at address 0, so this isn't a new
>concept.
>>> 
>>
>> I think that applies to DOSEMU, but not to Wine.
>
>DOSEMU does no longer need it either.  If vm.mmap_min_addr is > 0, it
>turns on CPU emulation, which it has to use anyway due to lack of vm86
>mode.
>
>> Sven: if you have the ability to build your own kernel, could you
>also
>> try the "x86/espfix" branch of the git tree:
>>
>> https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git/
>>
>> (clone URLs:)
>> git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
>> https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git
>>
>> ... to make sure the proper solution works for you?
>
>Works fine here, thanks.
>
>> I'm somewhat curious if this program you have is actually a 32-bit
>> program or if it is really a 16-bit program wrapped in a 32-bit
>> installer of some kind.  Hard to know without seeing the program in
>> question.
>
>The main application (a chess database program) is 32-bit, but it comes
>with several 16-bit analysis engines that are loaded on startup (I see
>them in lsof output), so that's the situation described by Alexandre.
>
>Cheers,
>       Sven

-- 
Sent from my mobile phone.  Please pardon brevity and lack of formatting.

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 05/08/2014 06:50 AM, H. Peter Anvin wrote:
> Actually it could use KVM instead of CPU emulation on nearly all modern processors...

Of course, at that point you might just run qemu-kvm instead of DOSEMU,
since I seem to recall that DOSEMU is still a real version of DOS.  I
have to admit to mostly using DOSBOX for games and KVM for anything else
that needs DOS these days... just what happens to work best for me.

	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 05/08/2014 06:50 AM, H. Peter Anvin wrote:
> Actually it could use KVM instead of CPU emulation on nearly all modern processors...

That being said, it would be cool if someone would either port the
lredir backend (MFS) into Qemu, or finish the 9P front end I started
writing at one point, but probably will never have time to finish.
	
	-hpa

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[Josh Boyer]

On Wed, May 7, 2014 at 12:57 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
> On Wed, May 7, 2014 at 2:18 AM, Sven Joachim <svenjoac@gmx.de> wrote:
>>
>> It seems that at least some 32-bit programs are also broken, since after
>> upgrading the kernel to 3.14.3 I can no longer start my old chess
>> database program:

Now that this has hit 3.14.y stable, we have another report of this
commit breaking something in Wine, this time MS Access 2000:

https://bugzilla.redhat.com/show_bug.cgi?id=1096725 (reporter now CC'd)

> So for backporting (and for 3.15) maybe this (TOTALLY UNTESTED) patch
> would be acceptable.

I don't think your patch went anywhere.  I have no idea if you want to
push that or revert or just tell people to run old apps in 32-bit
guests at this point.  Just forwarding on the information.

josh

Re: [tip:x86/urgent] x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

[H. Peter Anvin]

On 05/12/2014 06:16 AM, Josh Boyer wrote:
> On Wed, May 7, 2014 at 12:57 PM, Linus Torvalds
> <torvalds@linux-foundation.org> wrote:
>> On Wed, May 7, 2014 at 2:18 AM, Sven Joachim <svenjoac@gmx.de> wrote:
>>>
>>> It seems that at least some 32-bit programs are also broken, since after
>>> upgrading the kernel to 3.14.3 I can no longer start my old chess
>>> database program:
> 
> Now that this has hit 3.14.y stable, we have another report of this
> commit breaking something in Wine, this time MS Access 2000:
> 
> https://bugzilla.redhat.com/show_bug.cgi?id=1096725 (reporter now CC'd)
> 
>> So for backporting (and for 3.15) maybe this (TOTALLY UNTESTED) patch
>> would be acceptable.
> 
> I don't think your patch went anywhere.  I have no idea if you want to
> push that or revert or just tell people to run old apps in 32-bit
> guests at this point.  Just forwarding on the information.
> 

Linus is off the net at the moment.  Someone needs to take his patch and
test it/clean it up.

	-hpa

[tip:x86/urgent] x86-64, modify_ldt: Make support for 16-bit segments a runtime option

[tip-bot for Linus Torvalds]

Commit-ID:  fa81511bb0bbb2b1aace3695ce869da9762624ff
Gitweb:     http://git.kernel.org/tip/fa81511bb0bbb2b1aace3695ce869da9762624ff
Author:     Linus Torvalds <torvalds@linux-foundation.org>
AuthorDate: Wed, 14 May 2014 16:33:54 -0700
Committer:  H. Peter Anvin <hpa@linux.intel.com>
CommitDate: Wed, 14 May 2014 16:33:54 -0700

x86-64, modify_ldt: Make support for 16-bit segments a runtime option

Checkin:

b3b42ac2cbae x86-64, modify_ldt: Ban 16-bit segments on 64-bit kernels

disabled 16-bit segments on 64-bit kernels due to an information
leak.  However, it does seem that people are genuinely using Wine to
run old 16-bit Windows programs on Linux.

A proper fix for this ("espfix64") is coming in the upcoming merge
window, but as a temporary fix, create a sysctl to allow the
administrator to re-enable support for 16-bit segments.

It adds a "/proc/sys/abi/ldt16" sysctl that defaults to zero (off). If
you hit this issue and care about your old Windows program more than
you care about a kernel stack address information leak, you can do

   echo 1 > /proc/sys/abi/ldt16

as root (add it to your startup scripts), and you should be ok.

The sysctl table is only added if you have COMPAT support enabled on
x86-64, but I assume anybody who runs old windows binaries very much
does that ;)

Signed-off-by: H. Peter Anvin <hpa@linux.intel.com>
Link: http://lkml.kernel.org/r/CA%2B55aFw9BPoD10U1LfHbOMpHWZkvJTkMcfCs9s3urPr1YyWBxw@mail.gmail.com
Cc: <stable@vger.kernel.org>
---
 arch/x86/kernel/ldt.c        | 4 +++-
 arch/x86/vdso/vdso32-setup.c | 8 ++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

diff --git a/arch/x86/kernel/ldt.c b/arch/x86/kernel/ldt.c
index af1d14a..dcbbaa1 100644
--- a/arch/x86/kernel/ldt.c
+++ b/arch/x86/kernel/ldt.c
@@ -20,6 +20,8 @@
 #include <asm/mmu_context.h>
 #include <asm/syscalls.h>
 
+int sysctl_ldt16 = 0;
+
 #ifdef CONFIG_SMP
 static void flush_ldt(void *current_mm)
 {
@@ -234,7 +236,7 @@ static int write_ldt(void __user *ptr, unsigned long bytecount, int oldmode)
 	 * IRET leaking the high bits of the kernel stack address.
 	 */
 #ifdef CONFIG_X86_64
-	if (!ldt_info.seg_32bit) {
+	if (!ldt_info.seg_32bit && !sysctl_ldt16) {
 		error = -EINVAL;
 		goto out_unlock;
 	}
diff --git a/arch/x86/vdso/vdso32-setup.c b/arch/x86/vdso/vdso32-setup.c
index 0034898..e1f220e 100644
--- a/arch/x86/vdso/vdso32-setup.c
+++ b/arch/x86/vdso/vdso32-setup.c
@@ -39,6 +39,7 @@
 #ifdef CONFIG_X86_64
 #define vdso_enabled			sysctl_vsyscall32
 #define arch_setup_additional_pages	syscall32_setup_pages
+extern int sysctl_ldt16;
 #endif
 
 /*
@@ -249,6 +250,13 @@ static struct ctl_table abi_table2[] = {
 		.mode		= 0644,
 		.proc_handler	= proc_dointvec
 	},
+	{
+		.procname	= "ldt16",
+		.data		= &sysctl_ldt16,
+		.maxlen		= sizeof(int),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec
+	},
 	{}
 };