Warning about kernel 4.2 performance

Warning about kernel 4.2 performance

[Andrew Bird]

Hi all,
    I think it best to warn everyone that requires the maximum performance for Dosemu and is using 32bit linux to get hardware assistance via cpuemu=off (the default on 32bit), that if you install a 4.2 kernel or greater the system call that Dosemu uses to get you that extra speed is being turned off by default on upstream kernels. It is up to the individual distros to enable it specifically. This will, depending upon what your DOS app does, cause a considerable performance drop (for me that's about about a factor of 10) as Dosemu falls back to cpuemu=vm86. 

In Ubuntu Wily they have not enabled the vm86() system call, which I expect is because they overlooked the fact a new kernel config switch was added, which means if you need the speed you'll have to start building your own kernels. I raised a bug on Ubuntu's Launchpad tracker asking for the syscall to be compiled in, but runtime disabled by default and you should do the same for your distro if it's important to you too.
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1499089


-- 
Andrew Bird <ajb@spheresystems.co.uk>

Re: Warning about kernel 4.2 performance

[solarflow99]

thats a shame, I haven't checked but i'll bet Fedora will go this way as well.



On Sat, Oct 3, 2015 at 12:56 AM, Andrew Bird <ajb@spheresystems.co.uk> wrote:
> Hi all,
>     I think it best to warn everyone that requires the maximum performance for Dosemu and is using 32bit linux to get hardware assistance via cpuemu=off (the default on 32bit), that if you install a 4.2 kernel or greater the system call that Dosemu uses to get you that extra speed is being turned off by default on upstream kernels. It is up to the individual distros to enable it specifically. This will, depending upon what your DOS app does, cause a considerable performance drop (for me that's about about a factor of 10) as Dosemu falls back to cpuemu=vm86.
>
> In Ubuntu Wily they have not enabled the vm86() system call, which I expect is because they overlooked the fact a new kernel config switch was added, which means if you need the speed you'll have to start building your own kernels. I raised a bug on Ubuntu's Launchpad tracker asking for the syscall to be compiled in, but runtime disabled by default and you should do the same for your distro if it's important to you too.
> https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1499089
>
>
> --
> Andrew Bird <ajb@spheresystems.co.uk>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance

[Andrew Bird]

I don't use fedora any more myself, but a quick check online shows that Fedora 23 has kernel 4.2. Unfortunately I was unable to find out whether the kconfig on 32bit version enables the vm86() syscall or not. Does anyone have a running Fedora 23 32bit system they could check, please?


On Sat, 3 Oct 2015 21:27:44 -0700
solarflow99 <solarflow99@gmail.com> wrote:

> thats a shame, I haven't checked but i'll bet Fedora will go this way as well.
> 
> 
> 
> On Sat, Oct 3, 2015 at 12:56 AM, Andrew Bird <ajb@spheresystems.co.uk> wrote:
> > Hi all,
> >     I think it best to warn everyone that requires the maximum performance for Dosemu and is using 32bit linux to get hardware assistance via cpuemu=off (the default on 32bit), that if you install a 4.2 kernel or greater the system call that Dosemu uses to get you that extra speed is being turned off by default on upstream kernels. It is up to the individual distros to enable it specifically. This will, depending upon what your DOS app does, cause a considerable performance drop (for me that's about about a factor of 10) as Dosemu falls back to cpuemu=vm86.
> >
> > In Ubuntu Wily they have not enabled the vm86() system call, which I expect is because they overlooked the fact a new kernel config switch was added, which means if you need the speed you'll have to start building your own kernels. I raised a bug on Ubuntu's Launchpad tracker asking for the syscall to be compiled in, but runtime disabled by default and you should do the same for your distro if it's important to you too.
> > https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1499089
> >
> >
> > --
> > Andrew Bird <ajb@spheresystems.co.uk>
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andrew Bird <ajb@spheresystems.co.uk>

Re: Warning about kernel 4.2 performance

[Felix Miata]

Andrew Bird composed on 2015-10-04 08:16 (UTC+0100):

> I don't use fedora any more myself, but a quick check online shows that
> Fedora 23 has kernel 4.2. Unfortunately I was unable to find out whether
> the kconfig on 32bit version enables the vm86() syscall or not. Does
> anyone have a running Fedora 23 32bit system they could check, please?

I have several. Also, openSUSE Tumbleweed is due to switch to 4.2 very soon,
maybe tomorrow night, and I have more of them than Fedoras. Exactly what
needs to be done to get the answer you wish?
-- 
"The wise are known for their understanding, and pleasant
words are persuasive." Proverbs 16:21 (New Living Translation)

 Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata  ***  http://fm.no-ip.com/

Re: Warning about kernel 4.2 performance

[Andrew Bird]

Hi Felix,
   Well on Ubuntu the config used for each installed kernel is found in /boot e.g.

/boot/config-`uname -r`

But that name or location might be different on other systems. You should grep for VM86, e.g.

grep VM86 /boot/config-`uname -r`

Depending on which kernel version you are checking the response may be different e.g. on my 3.19 system I get

ajb@polly:~$ grep VM86 /boot/config-`uname -r`
CONFIG_VM86=y

I expect a 4.2 32bit system to have a variable called X86_LEGACY_VM86 if it's set to 'n' then the syscall is not enabled on that system. If it's important to you then please report it as a regression in the distro's bugtracker.

Thanks

On Sun, 4 Oct 2015 04:35:49 -0400
Felix Miata <mrmazda@earthlink.net> wrote:

> Andrew Bird composed on 2015-10-04 08:16 (UTC+0100):
> 
> > I don't use fedora any more myself, but a quick check online shows that
> > Fedora 23 has kernel 4.2. Unfortunately I was unable to find out whether
> > the kconfig on 32bit version enables the vm86() syscall or not. Does
> > anyone have a running Fedora 23 32bit system they could check, please?
> 
> I have several. Also, openSUSE Tumbleweed is due to switch to 4.2 very soon,
> maybe tomorrow night, and I have more of them than Fedoras. Exactly what
> needs to be done to get the answer you wish?
> -- 
> "The wise are known for their understanding, and pleasant
> words are persuasive." Proverbs 16:21 (New Living Translation)
> 
>  Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!
> 
> Felix Miata  ***  http://fm.no-ip.com/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andrew Bird <ajb@spheresystems.co.uk>

Re: Warning about kernel 4.2 performance (revised)

[Andrew Bird]

Hi Felix,
   Well on Ubuntu the config used for each installed kernel is found in /boot e.g.

/boot/config-`uname -r`

But that name or location might be different on other systems. You should grep for VM86, e.g.

grep VM86 /boot/config-`uname -r`

Depending on which kernel version you are checking the response may be different e.g. on my 3.19 system I get

ajb@polly:~$ grep VM86 /boot/config-`uname -r`
CONFIG_VM86=y

If it's not set then the syscall is not enabled on that system. If it's important to you then please report it as a regression in the distro's bugtracker.

e.g. reading a Ubuntu 4.2 32bit config file the response is

# CONFIG_VM86 is not set

Which is the setting gained from the default value X86_LEGACY_VM86 in Kconfig



Thanks

On Sun, 4 Oct 2015 04:35:49 -0400
Felix Miata <mrmazda@earthlink.net> wrote:

> Andrew Bird composed on 2015-10-04 08:16 (UTC+0100):
> 
> > I don't use fedora any more myself, but a quick check online shows that
> > Fedora 23 has kernel 4.2. Unfortunately I was unable to find out whether
> > the kconfig on 32bit version enables the vm86() syscall or not. Does
> > anyone have a running Fedora 23 32bit system they could check, please?
> 
> I have several. Also, openSUSE Tumbleweed is due to switch to 4.2 very soon,
> maybe tomorrow night, and I have more of them than Fedoras. Exactly what
> needs to be done to get the answer you wish?
> -- 
> "The wise are known for their understanding, and pleasant
> words are persuasive." Proverbs 16:21 (New Living Translation)
> 
>  Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!
> 
> Felix Miata  ***  http://fm.no-ip.com/
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andrew Bird <ajb@spheresystems.co.uk>

Re: Warning about kernel 4.2 performance

[Stas Sergeev]

04.10.2015 10:16, Andrew Bird пишет:
> I don't use fedora any more myself, but a quick check online shows that Fedora 23 has kernel 4.2. Unfortunately I was unable to find out whether the kconfig on 32bit version enables the vm86() syscall or not. Does anyone have a running Fedora 23 32bit system they could check, please?
>
What's the use?
Fedora never provided dosemu, so they obviously
always have vm86 disabled.
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance (revised)

[Felix Miata]

Andrew Bird composed on 2015-10-04 11:32 (UTC+0100):

> ajb@polly:~$ grep VM86 /boot/config-`uname -r`
> CONFIG_VM86=y

# uname -a
Linux gx280 4.2.1-300.fc23.i686+PAE #1 SMP Mon Sep 21 22:32:31 UTC 2015 i686 i686 i386 GNU/Linux
# grep VM86 /boot/config-`uname -r`
CONFIG_VM86=y
-- 
"The wise are known for their understanding, and pleasant
words are persuasive." Proverbs 16:21 (New Living Translation)

 Team OS/2 ** Reg. Linux User #211409 ** a11y rocks!

Felix Miata  ***  http://fm.no-ip.com/

Re: Warning about kernel 4.2 performance (revised)

[Stas Sergeev]

04.10.2015 20:26, Felix Miata пишет:
> Andrew Bird composed on 2015-10-04 11:32 (UTC+0100):
>
>> ajb@polly:~$ grep VM86 /boot/config-`uname -r`
>> CONFIG_VM86=y
> # uname -a
> Linux gx280 4.2.1-300.fc23.i686+PAE #1 SMP Mon Sep 21 22:32:31 UTC 2015 i686 i686 i386 GNU/Linux
> # grep VM86 /boot/config-`uname -r`
> CONFIG_VM86=y
Interesting, why do they enable it?
According to this thread:
https://lists.fedoraproject.org/pipermail/devel/2015-July/212084.html
they don't even know why themselves. :)
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance

[solarflow99]

I had tried to include it in Fedora, but licensing and binary
redistribution prevented that, so it went to its sister project
rpmfusion all made of Fedora developers and packages specially for
rhel/Fedora.

Re: Warning about kernel 4.2 performance

[Andrew Bird]

I'm pretty sure the licensing is sorted out for dosemu2 and there are no precompiled binaries included as long as it's packaged without freedos. Or is there some other issue? Maybe it could now get packaged for Fedora / RHEL, but I don't know the mechanics of the process.

On Sun, 4 Oct 2015 13:45:06 -0700
solarflow99 <solarflow99@gmail.com> wrote:

> I had tried to include it in Fedora, but licensing and binary
> redistribution prevented that, so it went to its sister project
> rpmfusion all made of Fedora developers and packages specially for
> rhel/Fedora.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andrew Bird <ajb@spheresystems.co.uk>

Re: Warning about kernel 4.2 performance (revised)

[Andrew Bird]

Mmm, that thread is full of the same old rhetoric (i.e. it must be a risk as no one can be bothered/has time to check it and users must be prevented from hurting themselves even if they've already jumped through hoops to enable mm.vm_addr=0, which is a known security risk, so that dosemu can function with cpuemu=off). I do fail to see why the default can't have the vm86() syscall compiled in, but disabled by default at run time. It wouldn't affect anyone except those that want to turn it on at run time. (sorry about the rant, it wasn't directed at anyone here, just letting off steam!)

Regarding RHEL kernels, I've used CentOS 3, 4, 5 and 6 successfully with Dosemu cpuemu=off, so CONFIG_VM86=y was set on those.

 
On Sun, 4 Oct 2015 20:36:42 +0300
Stas Sergeev <stsp@list.ru> wrote:

> 04.10.2015 20:26, Felix Miata пишет:
> > Andrew Bird composed on 2015-10-04 11:32 (UTC+0100):
> >
> >> ajb@polly:~$ grep VM86 /boot/config-`uname -r`
> >> CONFIG_VM86=y
> > # uname -a
> > Linux gx280 4.2.1-300.fc23.i686+PAE #1 SMP Mon Sep 21 22:32:31 UTC 2015 i686 i686 i386 GNU/Linux
> > # grep VM86 /boot/config-`uname -r`
> > CONFIG_VM86=y
> Interesting, why do they enable it?
> According to this thread:
> https://lists.fedoraproject.org/pipermail/devel/2015-July/212084.html
> they don't even know why themselves. :)
> --
> To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Andrew Bird <ajb@spheresystems.co.uk>
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance

[Stas Sergeev]

05.10.2015 12:33, Andrew Bird пишет:
> I'm pretty sure the licensing is sorted out for dosemu2 and there are no precompiled binaries included as long as it's packaged without freedos.
You can package without FreeDOS of course, but this
is currently a non-workable solution.
If dosemu could at least do a network install of freedos,
then maybe this makes sense. But I don't think there is
an established practice of the network installers in a
free distros, so I suspect even this won't help (but network
install would be very nice to have anyway).
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance (revised)

[Stas Sergeev]

05.10.2015 12:47, Andrew Bird пишет:
> Mmm, that thread is full of the same old rhetoric (i.e. it must be a risk as no one can be bothered/has time to check it and users
> must be prevented from hurting themselves even if they've already jumped through hoops to enable mm.vm_addr=0, which is a known security
> risk, so that dosemu can function with cpuemu=off). I do fail to see why the default can't have the vm86() syscall compiled in, but
> disabled by default at run time.
As Felix pointed, this is exactly what happens.
But really, having never delivering dosemu, keeping vm86
ON is pretty useless for them, so I won't bet on this to
stay forever. But now as it can be disabled at run-time -
the chances are big they'll no longer re-visit this subject
and leave it as is.
Note that AFAIK dosemu can't use vm86 on fedora even if
enabled both compile-time and run-time, because selinux
then prevents mapping zero page even if you enabled it in
mmap_min_addr (but I may be wrong, someone needs to double-check
also this). The security threat may come from the fact
that you need to disable selinux.


> Regarding RHEL kernels, I've used CentOS 3, 4, 5 and 6 successfully with Dosemu cpuemu=off, so CONFIG_VM86=y was set on those.
Have you disabled selinux?
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance (revised)

[Andrew Bird]

On Mon, 5 Oct 2015 14:20:41 +0300
Stas Sergeev <stsp@list.ru> wrote:

> 05.10.2015 12:47, Andrew Bird пишет:
> > Mmm, that thread is full of the same old rhetoric (i.e. it must be a risk as no one can be bothered/has time to check it and users
> > must be prevented from hurting themselves even if they've already jumped through hoops to enable mm.vm_addr=0, which is a known security
> > risk, so that dosemu can function with cpuemu=off). I do fail to see why the default can't have the vm86() syscall compiled in, but
> > disabled by default at run time.
> As Felix pointed, this is exactly what happens.
> But really, having never delivering dosemu, keeping vm86
> ON is pretty useless for them, so I won't bet on this to
> stay forever. But now as it can be disabled at run-time -
> the chances are big they'll no longer re-visit this subject
> and leave it as is.
> Note that AFAIK dosemu can't use vm86 on fedora even if
> enabled both compile-time and run-time, because selinux
> then prevents mapping zero page even if you enabled it in
> mmap_min_addr (but I may be wrong, someone needs to double-check
> also this). The security threat may come from the fact
> that you need to disable selinux.
> 
> 
Yes, I understand and if it stays that way that's great, I only wish Ubuntu were the same :-(

> > Regarding RHEL kernels, I've used CentOS 3, 4, 5 and 6 successfully with Dosemu cpuemu=off, so CONFIG_VM86=y was set on those.
> Have you disabled selinux?
I no longer use CentOS, but my notes indicate that I did disable it at the time.
This page
http://blog.namei.org/2008/02/15/mmap_min_addr-setting-may-mitigate-vmsplice-exploit/
suggests an selinux policy written for a specific app can be used to avoid the sledgehammer of disabling selinux, though I have no idea how to do that.

--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: Warning about kernel 4.2 performance (revised)

[Bart Oldeman]

Hi,

we may have better luck with KVM for the forseeable future than with
VM86. I submitted a proof-of-concept PR here:
https://github.com/stsp/dosemu2/pull/90, based on this article:
http://lwn.net/Articles/658511/

At least on Westmere+ CPUs (first gen Core i3/i5/i7) KVM can run
"unrestricted guest" mode which behaves like real mode and seems just
as fast as vm86() but it traps less than vm86 so some things that
dosemu depends upon (e.g. that INTs are trapped) do not work. That can
be worked around though with some more coding.

(Yes I am still alive but Stas has for more time than me to work on
DOSEMU so I wish him good luck with dosemu2 so that's why I sent the
PR to dosemu2).

Regards,
Bart

Re: Warning about kernel 4.2 performance (revised)

[Paul Crawford]

Hi Bart,
> At least on Westmere+ CPUs (first gen Core i3/i5/i7) KVM can run
> "unrestricted guest" mode which behaves like real mode and seems just
> as fast as vm86() but it traps less than vm86 so some things that
> dosemu depends upon (e.g. that INTs are trapped) do not work. That can
> be worked around though with some more coding.

In our case the ability to get hardware interrupts routed through to the 
DOS emulator's code in a prompt manner is VERY important. In fact that 
is one major reason we have generally not played with the 64-bit version 
very much in the past.

By "prompt" I mean less than 10us or so which was no trouble in the 
native 286/DOS era, and today should not be a problem given a typical 
CPU can execute some 30k instructions in that time, but we know that is 
not how software bloat and indirection go :(

Regards,
Paul

Re: Warning about kernel 4.2 performance (revised)

[Stas Sergeev]

14.10.2015 22:10, Paul Crawford пишет:
> Hi Bart,
>> At least on Westmere+ CPUs (first gen Core i3/i5/i7) KVM can run
>> "unrestricted guest" mode which behaves like real mode and seems just
>> as fast as vm86() but it traps less than vm86 so some things that
>> dosemu depends upon (e.g. that INTs are trapped) do not work. That can
>> be worked around though with some more coding.
>
> In our case the ability to get hardware interrupts routed through to 
> the DOS emulator's code in a prompt manner is VERY important.
If you mean the hardware interrupts from real hardware
(rather than from the emulated hw), then the only way to
keep this alive is to write an UIO backend:
https://www.kernel.org/doc/htmldocs/uio-howto/about.html#how_uio_works
And that should be written by someone who at least has
the test case, which means you. :)
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Warning about kernel 4.2 performance (revised)

[solarflow99]

this is great if this provides an easier way to make dos programs work
in 64 bit mode.  In the thread it sounds like vm86 might still be
used, so not sure which way it should go, but whatever is easier will
probably be the best way since dosemu doesn't have as much development
time these days.

Switching to KVM would change things a lot, dosemu2 would make sense for that.

Re: Warning about kernel 4.2 performance (revised)

[Stas Sergeev]

16.10.2015 05:07, solarflow99 пишет:
> this is great if this provides an easier way to make dos programs work
> in 64 bit mode.  In the thread it sounds like vm86 might still be
> used, so not sure which way it should go,
vm86 never worked in 64bit mode.
This is actually the reason I didn't rush re-implementing it
in the kernel: I don't even use it myself, so I felt very guilty
even defending it in lkml.
So I guess the kvm efforts that Bart started, will eventually
do the trick better than vm86 ever could. But this will require
some coding efforts to get right. Much less in dosemu2 than
in dosemu1, because dosemu2 was already freed from most
vm86 ugly dependencies (but not all, as it now appears).

> Switching to KVM would change things a lot, dosemu2 would make sense for that.
For some workloads (heavy games, gfx effects etc) dosemu2
without vm86 is faster than dosemu1 with vm86. Try
Destruction Derby, Settlers2, NFS-SE or even doom2 to
see the difference (vm86 is not in a big use for these games
though, that's why the difference will be serious).
--
To unsubscribe from this list: send the line "unsubscribe linux-msdos" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html