Somewhat confused with Interrupts/TSRs

Somewhat confused with Interrupts/TSRs

[John Coppens]

Hello people.

I accepted the task of trying to get a very old datalogging program
working for a friend. It is needed to read an older sensor, and
apparently no other software is available.

It's a bit large (170 kB exe) to disassemble completely, but cursory
investigation shows it was written in Turbo Pascal. Apparently the CRT
unit was included, which makes executable use internal keyboard access
instead of MsDOS functions.

All this is important, because I want to simulate keypresses and the TSR
doesn't seem to work with any CRT using Pascal program.

I followed the program to be using Int16 for key input, which probably
explains why no Int28 or Int1C calls are made anymore (needed for the
TSR to work)...

I tried to follow where 16h goes to, but, using debug in DOSemu, this
leads to a HLT instruction...

Does anyone know of this (TSR) problem or can anyone suggest a solution?

Thanks!
John

Re: Somewhat confused with Interrupts/TSRs

[Mike McCarty]

John Coppens wrote:
> Hello people.
> 
> I accepted the task of trying to get a very old datalogging program
> working for a friend. It is needed to read an older sensor, and
> apparently no other software is available.
> 
> It's a bit large (170 kB exe) to disassemble completely, but cursory
> investigation shows it was written in Turbo Pascal. Apparently the CRT
> unit was included, which makes executable use internal keyboard access
> instead of MsDOS functions.
> 
> All this is important, because I want to simulate keypresses and the TSR
> doesn't seem to work with any CRT using Pascal program.

I don't understand this statement. Do you mean that there is another
TP program which uses the CRT module and which must run with this
TSR, but does not?

> I followed the program to be using Int16 for key input, which probably
> explains why no Int28 or Int1C calls are made anymore (needed for the
> TSR to work)...

I also don't follow this. The Int16 BIOS calls don't disturb the
clock interrupts, AFAIK. I've used them many times.

> I tried to follow where 16h goes to, but, using debug in DOSemu, this
> leads to a HLT instruction...

Sounds like the TSR may be trying to guard itself against debug. I wrote
some device drivers 'way back on contract, and the people I worked
for wanted to prevent them from running when debug was being used.

> Does anyone know of this (TSR) problem or can anyone suggest a solution?

You could try stuffing the BIOS type-ahead buffer. Is there a reason
you don't have source? Also, ISTM that you may have uncovered
an incompatibility. Are you running actual MSDOS, or freedos?
You might try running actual MSDOS rather than freedos, still
using dosemu.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Somewhat confused with Interrupts/TSRs

[John Coppens]

On Fri, 30 Nov 2007 12:41:38 -0600
Mike McCarty <Mike.McCarty@sbcglobal.net> wrote:

> John Coppens wrote:
> > Hello people.
> > 
> > I accepted the task of trying to get a very old datalogging program
> > working for a friend. It is needed to read an older sensor, and
> > apparently no other software is available.
> > 
> > It's a bit large (170 kB exe) to disassemble completely, but cursory
> > investigation shows it was written in Turbo Pascal. Apparently the CRT
> > unit was included, which makes executable use internal keyboard access
> > instead of MsDOS functions.
> > 
> > All this is important, because I want to simulate keypresses and the
> > TSR doesn't seem to work with any CRT using Pascal program.
> 
> I don't understand this statement. Do you mean that there is another
> TP program which uses the CRT module and which must run with this
> TSR, but does not?

Hi Mike. Sorry for all the confusion. It's the TSR that has to generate
the keypresses, and the TSR works fine. Eg. I can call 'debug', send a
dump command, and exit again, all via the TSR.

The sensor logger is the one that seems to be compiled with 'uses CRT' in
Pascal, which means that the executable uses Int16 instead of the DOS
routines to read the keyboard (and to write the screen).

> > I followed the program to be using Int16 for key input, which probably
> > explains why no Int28 or Int1C calls are made anymore (needed for the
> > TSR to work)...
> 
> I also don't follow this. The Int16 BIOS calls don't disturb the
> clock interrupts, AFAIK. I've used them many times.

Yes... The logger calls Int16, which works fine, except that in that case
the TSR stops working, because both 'hooks' to 'stimulate' the TSR (Int
1C and 28) stop working (see below). Note that both 1Ch and 28h are
'voluntary' interrupts, called only when DOS is idle. So while in the
(BIOS's) Int16, DOS won't be able to call them.

> > I tried to follow where 16h goes to, but, using debug in DOSemu, this
> > leads to a HLT instruction...
> 
> Sounds like the TSR may be trying to guard itself against debug. I wrote
> some device drivers 'way back on contract, and the people I worked
> for wanted to prevent them from running when debug was being used.

No. It's not the TSR. Run xdosemu, and then debug (no TSR running). The
16h interrupt vector points to F800:682E. At that address is a very
simple routine:

F800:682E FB            STI
F800:682F F6C4EF        TEST    AH,EF
F800:6832 7405          JZ      6839
F800:6834 EA16C000F0    JMP     F000:C016
F800:6839 9C            PUSHF
F800:683A 9A16C000F0    CALL    F000:C016
F800:683F 74F8          JZ      6839
F800:6841 CF            IRET

And at F000:C016 is a HLT. I suspect this is part of the mechanism dosemu
uses to emulate.

> > Does anyone know of this (TSR) problem or can anyone suggest a
> > solution?
> 
> You could try stuffing the BIOS type-ahead buffer. Is there a reason
> you don't have source? Also, ISTM that you may have uncovered
> an incompatibility. Are you running actual MSDOS, or freedos?
> You might try running actual MSDOS rather than freedos, still
> using dosemu.

That's what I'm doing in the TSR. And works fine, too. With other
programs or Pascal programs without CRT compiled in. But it stops working
with any Pascal program with CRT. Eg.:

program Simple;
uses
  CRT;
var
  w: string;

begin
  readln(w);
  writeln('I''ve read:', w);
end.

It seems that's because, with CRT, the program uses Int16 to read
characters, which means DOS can't activate either Ints 1Ch or 28h (and
thus the TSR remains unable to stuff keys). I'm guessing this would be
exactly the same with MsDOS instead of FreeDOS.

Hope this clears it up...
John

Re: Somewhat confused with Interrupts/TSRs

[James Courtier-Dutton]

John Coppens wrote:
> 
> Yes... The logger calls Int16, which works fine, except that in that case
> the TSR stops working, because both 'hooks' to 'stimulate' the TSR (Int
> 1C and 28) stop working (see below). Note that both 1Ch and 28h are
> 'voluntary' interrupts, called only when DOS is idle. So while in the
> (BIOS's) Int16, DOS won't be able to call them.

The program does use Int16, function AH=0x10.
You would need to also hook Int16 so that it "stimulates" your TSR program.

> 
> 
> And at F000:C016 is a HLT. I suspect this is part of the mechanism dosemu
> uses to emulate.
> 
That is correct, the HLT causes the dosemu to callout of DOS back into
the dosemu program to process the BIOS call.

I would probably take this approach:
Modify dosemu, or dosbox so that the BIOS call reads keyboard input from
a unix pipe. Then you can simply cat characters into the pipe from a
shell and the dos program will receive them.

Then you would not need a TSR at all!

James

Re: Somewhat confused with Interrupts/TSRs

[Mike McCarty]

John Coppens wrote:

Did you intend to take this to private e-mail?

> On Fri, 30 Nov 2007 12:41:38 -0600
> Mike McCarty <Mike.McCarty@sbcglobal.net> wrote:
> 
>> John Coppens wrote:
>>> Hello people.

[snip]

>>> All this is important, because I want to simulate keypresses and the
>>> TSR doesn't seem to work with any CRT using Pascal program.
>> I don't understand this statement. Do you mean that there is another
>> TP program which uses the CRT module and which must run with this
>> TSR, but does not?
> 
> Hi Mike. Sorry for all the confusion. It's the TSR that has to generate
> the keypresses, and the TSR works fine. Eg. I can call 'debug', send a
> dump command, and exit again, all via the TSR.
> 
> The sensor logger is the one that seems to be compiled with 'uses CRT' in
> Pascal, which means that the executable uses Int16 instead of the DOS
> routines to read the keyboard (and to write the screen).

Ah, I see. So, the TSR is stuffing the BIOS TA buffer, but you aren't
able to get it from the *other* program. AIUI the TSR isn't running
because the DOS IDLE interrupts aren't being used. This is, as you say,
because you use the BIOS keyboard interrupts.

>>> I followed the program to be using Int16 for key input, which probably
>>> explains why no Int28 or Int1C calls are made anymore (needed for the
>>> TSR to work)...
>> I also don't follow this. The Int16 BIOS calls don't disturb the
>> clock interrupts, AFAIK. I've used them many times.
> 
> Yes... The logger calls Int16, which works fine, except that in that case
> the TSR stops working, because both 'hooks' to 'stimulate' the TSR (Int
> 1C and 28) stop working (see below). Note that both 1Ch and 28h are
> 'voluntary' interrupts, called only when DOS is idle. So while in the
> (BIOS's) Int16, DOS won't be able to call them.

Which INT16 services are you using? [grabs old MSDOS programming book]
Service 00H	Wait until a characer is ready for input, put into AL
         01H	Read status, return flag and char if one ready
         02H	Return keyboard flags

Ok, I guess you're calling service 00H, and it is "hang and wait".

I take issue with your statement that INT 1CH is optional. As I stated
before, it is the system clock interrupt (software level). INT 28 is
DOS_IDLE as you stated. But even if DOS_IDLE is not being called,
INT 1C should be.

>>> I tried to follow where 16h goes to, but, using debug in DOSemu, this
>>> leads to a HLT instruction...
>> Sounds like the TSR may be trying to guard itself against debug. I wrote
>> some device drivers 'way back on contract, and the people I worked
>> for wanted to prevent them from running when debug was being used.
> 
> No. It's not the TSR. Run xdosemu, and then debug (no TSR running). The
> 16h interrupt vector points to F800:682E. At that address is a very
> simple routine:
> 
> F800:682E FB            STI
> F800:682F F6C4EF        TEST    AH,EF

This code looks for an extended keyboard service.

> F800:6832 7405          JZ      6839

We get here for all "normal" services, except service 00H.
So, services 00H and 10H wind up here. 00H is read keyboard.

> F800:6834 EA16C000F0    JMP     F000:C016
> F800:6839 9C            PUSHF
> F800:683A 9A16C000F0    CALL    F000:C016
> F800:683F 74F8          JZ      6839
> F800:6841 CF            IRET
> 
> And at F000:C016 is a HLT. I suspect this is part of the mechanism dosemu
> uses to emulate.

Ah. Interesting. Perhaps the HLT instruction is mis-emulated!
It should halt until an interrupt occurs, then exit. It appears
that the hardware clock interrupts are not being passed through
to the emulation level to cause the HLT to exit. As a NASTY work
around, you could patch out the STI instruction. This leaves
you with a window during which the code might hang, due to the
keyboard interrupt arriving just before the HLT. If the clock
interrupt eventually does get passed through, then it would
eventually recover. You could do this in a little program which
runs just before the TSR gets run (and installs itself) during boot.
Hopefully, this area is not in emulated ROM, prohibiting writes.
If it is, then you could copy the entire BIOS INT 16H routine
into ANOTHER TSR which then omits the STI, and revector
to that.

Another possibility is to patch the program using CRT to use
service 01H in a loop until it finds "character ready" and then
use service 00H to read it, which hopefully then doesn't enter
this code. Or put that into another TSR (as above) which emulates
service 00H by using 01H until it gets "ready" and then use
00H to read it and return. This may be the best way, as it has
very little interaction with other programs, and is less "fiddly".
This little TSR could be just a few bytes long, actually. The
code would look like this:

	resize memory END_MEM
	disable interrupts
	save old INT 16H vector
	set self as new INT 16H handler
	enable interrupts
	TSR

handler:
	if service is not 00H
		jump to old INT 16H vector
loop:
	load AH with 01H
	push flags to emulate interrupt
	call old INT 16H vector
	if flag indicates not ready
		jmp to loop
	load AH with 00H
	push flags to emulate interrupt
	call old INT 16H vector
	IRET
END_MEM:

Another possibility is to patch the program using CRT to use
the corresponding MSDOS call INT 21H service 08H to read a
character w/o echo. This opens you up to ^C and ^BREAK messing
you up, however. Do you already trap those? You'd need an
INT 23H handler.

[...]

> Hope this clears it up...

I think I've got a better handle. This looks like an oversight
in dosemu which causes an interaction when used with the INT 16H
BIOS keyboard interrupt service.

http://www.ctyme.com/intr/int-16.htm

Your best bet may be the little TSR I described above.

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Somewhat confused with Interrupts/TSRs

[Mike McCarty]

John Coppens wrote:

Did you intend only to answer via private e-mail?

> On Fri, 30 Nov 2007 16:31:29 -0600
> Mike McCarty <Mike.McCarty@sbcglobal.net> wrote:
> 
> 
>> I take issue with your statement that INT 1CH is optional. As I stated
>> before, it is the system clock interrupt (software level). INT 28 is
>> DOS_IDLE as you stated. But even if DOS_IDLE is not being called,
>> INT 1C should be.
> 
> I stand corrected, Mike... I _did_ read that somewhere, but now I

I didn't mean to be "in your face". Sorry it sounded that way.

> disassembled the short code of Int08, and Int1C is called _there_, at
> the end. So, unless interrupts are disabled, the TSR should be called
> (which isn't happening). I'll investigate along that path.

Right... INT 08H is the hardware clock. All the INT 08H handler should
do really is keep the hardware happy and ready to make another timer
tick, and then pass things along to the software timer system, by using
the system timer interrupt INT 1CH, which is the software level clock.

INT 1CH keeps the TOD etc. It is not optional. One should not hook
INT 08H as it needs immediate access to the hardware. If one wants
to mess with the system timer I/F and then pass back after having
done some other work, one should hook INT 1CH.

> <...>
>>> And at F000:C016 is a HLT. I suspect this is part of the mechanism
>>> dosemu uses to emulate.
>> Ah. Interesting. Perhaps the HLT instruction is mis-emulated!
>> It should halt until an interrupt occurs, then exit. It appears
>> that the hardware clock interrupts are not being passed through
>> to the emulation level to cause the HLT to exit. 
> 
> Privately, James confirmed that the HLT instruction is used by DOSemu to
> emulate the BIOS functions (that's why the BIOS is so small - my $.02)

Not surprising. This looks like an unintended consequence.
INT 08H is tied to a physical interrupt line controlled by
the PIC. When the HLT instruction is executed with interrupts
disabled, it exits the halted state when an interrupt occurs.

http://library.n0i.net/hardware/i8086opcodes/#HLT

>> Another possibility is to patch the program using CRT to use
>> the corresponding MSDOS call INT 21H service 08H to read a
>> character w/o echo. This opens you up to ^C and ^BREAK messing
>> you up, however. Do you already trap those? You'd need an
>> INT 23H handler.
> 
> Yes, I do... 
> 
> I think it boils down to the fact that for some reason Interrupts seem to
> be disabled while Int16 is called, else the TSR would be called as it

That particular bit of code needs to be interrupt protected, I suspect.
The issue is this: two bits of code are using/modifying the same
variables. The keyboard interrupt handler (the one which responds to the
physical interrupt line associated with the 8048 programmed
as the keyboard interface processor) places characters into the
BIOS TA buffer. It looks at head and tail, and if there is room
it puts a character in, otherwise it sets up the sound system
to make a "beep". It also modifies the head pointer in the BIOS
area when it puts a character into the TA buffer.

The INT 16H handler should look at tail and head, and use this to
ascertain whether the TA buffer be holding anything. If TA is holding,
then the INT 16H handler should take the character at tail, and update
tail. If it makes a decision that it does not have a character
available, then it needs to wait. It does this by executing HLT. When a
hardware interrupt occurs, then the processor exits halted state.

Without interrupts being disabled, a race state would occur. The INT 16H
code would check, and see that no character is available, making a
decision to halt. If, before the HLT instruction is executed,
the keyboard interrupt occurs, then the character gets placed into the
TA buffer, and the interrupt event is now over. When the processor later
executes the HLT, then the keyboard event would already be past.

> should be. This may be a problem resulting from how things are simulated
> in DOSemu, in which case the whole thing may actually work on the DOS
> machine, ;-) 

It looks to me like some at least of the emulated hardware interrupt
events are not getting passed up the emulation chain to the HLT pending
halt state to cause it to exit.

I haven't read the code, just guessing from the behavior.

> I'll test that tomorrow... Thanks for all the hints!

Quite welcome!

> 
> John
> 
> 


-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!

Re: Somewhat confused with Interrupts/TSRs

[Mike McCarty]

John Coppens wrote:

[...]

> I think it boils down to the fact that for some reason Interrupts seem to
> be disabled while Int16 is called, else the TSR would be called as it
> should be. This may be a problem resulting from how things are simulated
> in DOSemu, in which case the whole thing may actually work on the DOS
> machine, ;-) 
> 
> I'll test that tomorrow... Thanks for all the hints!

What came of it?

Mike
-- 
p="p=%c%s%c;main(){printf(p,34,p,34);}";main(){printf(p,34,p,34);}
Oppose globalization and One World Governments like the UN.
This message made from 100% recycled bits.
You have found the bank of Larn.
I can explain it for you, but I can't understand it for you.
I speak only for myself, and I am unanimous in that!