Re: Memory issues and USB support? (and OpenWatcom)

Re: Memory issues and USB support? (and OpenWatcom)

[David O'Shea]

Hi Alan, all,

On Tue, Jun 6, 2017 at 12:11 AM, Alan Cox <alan@llwyncelyn.cymru> wrote:
> The bigger limit is that
> for small mode you've only got 64K offsets from each segment register.

That kind of leads me to something else I wanted to ask about: I take
it the requirement to use small mode is because of the bcc compiler?
I saw various mentions of looking at using OpenWatcom in the past
(e.g. http://www.spinics.net/lists/linux-8086/msg00417.html), and I
didn't see these reach any showstoppers or anyone saying it was a
terrible idea, nor success.  Is moving to OpenWatcom something that
would be welcome, so long as it was truly an improvement (and didn't
require performing compiles on DOS :) )?

> In the Linux 8086 case the big problem is the kernel stack for each
> process comes from that 64K.

On entry to the kernel, couldn't one change SS:SP to point to some
other memory area?  It looks like the parameters to system calls are
in registers rather than on the stack.  Is it just a case of not
having done this extra work of managing more bits of memory yet, or is
there an architectural limitation here?

Apologies if any of the above are FAQs!

>> As it is, with LIM EMS
>> 3.2, even if you can only map in a single 64KB page at a time, you
>> could at least do that for some of their memory.  I guess you also get
>> some memory protection from this, right?
>
> Not a lot no. Most of this comes in with later processors.

I was kind of thinking that a process whose pages are in EMS are
unlikely to get corrupted by a process running off the end of an
array.  I realise that I shouldn't have used the term "memory
protection" though!

Thanks,
David

Re: Memory issues and USB support? (and OpenWatcom)

[Alan Cox]

> That kind of leads me to something else I wanted to ask about: I take
> it the requirement to use small mode is because of the bcc compiler?

Moving to large code isn't a problem, but large data means far pointers
and that means a huge slowdown in performance. Instead ELKS has some
facilities to allow things like disk cache to be in a different segment.

For userspace large models also mean you can't do swapping until you have
286 protected mode.

> > In the Linux 8086 case the big problem is the kernel stack for each
> > process comes from that 64K.  
> 
> On entry to the kernel, couldn't one change SS:SP to point to some
> other memory area?  It looks like the parameters to system calls are
> in registers rather than on the stack.  Is it just a case of not
> having done this extra work of managing more bits of memory yet, or is
> there an architectural limitation here?

SS and DS have to be the same in small and tiny mode. A syscall switches
to the kernel code and data so the kernel has 64K of data to play with.
It uses a load of it on kernel stacks it could avoid at a small cost on
context switches. If that were fixed there would be plenty of kernel data
space.

> I was kind of thinking that a process whose pages are in EMS are
> unlikely to get corrupted by a process running off the end of an
> array.  I realise that I shouldn't have used the term "memory
> protection" though!

It's less likely one would be scribbled on, but in fact if you give a
16bit process 64K code and 64K data it has to corrupt segment registers
to scribble outside of its space and that makes it more reliable than you
might expect.

Alan

Re: Memory issues and USB support? (and OpenWatcom)

[David O'Shea]

On Mon, Jun 12, 2017 at 12:24 AM, Alan Cox <alan@llwyncelyn.cymru> wrote:
> Moving to large code isn't a problem, but large data means far pointers
> and that means a huge slowdown in performance.

Ah, that hadn't occurred to me, but it makes sense.  Thanks!

> For userspace large models also mean you can't do swapping until you have
> 286 protected mode.

I assume the issues are that you can't page segments in on demand
because there's no MMU, so whenever you schedule the process you'd
have to page all of it in, and the process now has physical memory
addresses (rather than just addresses relative to some registers that
the kernel can change), so you always have to page it to the same
location in memory?

I presume that this is why the MS/PC-DOS DOSSHELL (and no doubt there
were various other tools that did the same) was a "task swapper" which
swapped your entire process to/from disk when you switched processes?

To support larger programs, would it be worth sacrificing the ability
to swap those particular programs?

And regardless of moving away from tiny model, I gather that moving to
OpenWatcom just for the optimisations and ANSI C would potentially be
worthwhile?

> It's less likely one would be scribbled on, but in fact if you give a
> 16bit process 64K code and 64K data it has to corrupt segment registers
> to scribble outside of its space and that makes it more reliable than you
> might expect.

Oh of course!

Thanks!
David

Re: Memory issues and USB support? (and OpenWatcom)

[Alan Cox]

On Mon, 12 Jun 2017 13:35:49 +0930
"David O'Shea" <dcoshea@gmail.com> wrote:

> On Mon, Jun 12, 2017 at 12:24 AM, Alan Cox <alan@llwyncelyn.cymru> wrote:
> > Moving to large code isn't a problem, but large data means far pointers
> > and that means a huge slowdown in performance.  
> 
> Ah, that hadn't occurred to me, but it makes sense.  Thanks!
> 
> > For userspace large models also mean you can't do swapping until you have
> > 286 protected mode.  
> 
> I assume the issues are that you can't page segments in on demand
> because there's no MMU, so whenever you schedule the process you'd
> have to page all of it in, and the process now has physical memory
> addresses (rather than just addresses relative to some registers that
> the kernel can change), so you always have to page it to the same
> location in memory?

Mostly the latter. If you want the full glorious horror story take a look
at how windows16 does call stack frame fixup for 16bit apps.

(Short version - each stack frame pushes bp. Windows 16 inc's bp before
pushing and decs after if the function is far, thus allowing the windows
OS code to walk the stack frames and modify each far return according to
what is and isn't in memory and where it moved)

We could modify the compiler to do that, but it's up there with coding
while drunk on 'good idea' rating.

> I presume that this is why the MS/PC-DOS DOSSHELL (and no doubt there
> were various other tools that did the same) was a "task swapper" which
> swapped your entire process to/from disk when you switched processes?

That was more because a DOS program owns all of memory so you don't know
how much to give it.

> 
> To support larger programs, would it be worth sacrificing the ability
> to swap those particular programs?

No - those are the ones you need to swap the most.

> And regardless of moving away from tiny model, I gather that moving to
> OpenWatcom just for the optimisations and ANSI C would potentially be
> worthwhile?

16bit gcc is probably better.

Alan

Re: Memory issues and USB support? (and OpenWatcom)

[David O'Shea]

On Wed, Jun 14, 2017 at 10:53 AM, Alan Cox <alan@llwyncelyn.cymru> wrote:
> Mostly the latter. If you want the full glorious horror story take a look
> at how windows16 does call stack frame fixup for 16bit apps.
>
> (Short version - each stack frame pushes bp. Windows 16 inc's bp before
> pushing and decs after if the function is far, thus allowing the windows
> OS code to walk the stack frames and modify each far return according to
> what is and isn't in memory and where it moved)

Oh, nice, sort of!  I found
https://blogs.msdn.microsoft.com/oldnewthing/20110316-00/?p=11203
which explains it in more words, but I probably would have understood
your explanation if I had thought about it some more :)

> We could modify the compiler to do that, but it's up there with coding
> while drunk on 'good idea' rating.

Yeah, it was pointed out in the comments on the above article that if
you corrupt your stack you're going to make the kernel corrupt some
other memory.

I presume that modifying far pointers to code within the code (rather
than within the stack as is covered above) on Win16 relies on having a
relocation table, and ELKS binaries don't have relocation tables since
they're small model, correct?

> That was more because a DOS program owns all of memory so you don't know
> how much to give it.

Oh yeah, of course, thanks!

>> And regardless of moving away from tiny model, I gather that moving to
>> OpenWatcom just for the optimisations and ANSI C would potentially be
>> worthwhile?
>
> 16bit gcc is probably better.

I was thinking that at least OpenWatcom is maintained, but I didn't
think to check if anyone had resurrected 16-bit GCC in the last 3
months :)

Thanks!
David