Emulated PCI devices?

Emulated PCI devices?

[Mouse]

I'm working with dosemu and would like to add an emulated PCI device to
the emulated DOS machine.  Obviously, I know enough about the hardware
I want to emulate to believe I can write an emulation of it.

But I'm not sure where to hook it in.  (I'm working based on commit
18f6f5cdf1beceae8c7532718bfaf423e4a44f6a.)  I found
src/base/async/pci_bios.c and src/base/dev/misc/pci.c, but they appear
to be all about giving the emulated machine access to real PCI hardware
on the real machine.  That's not what I want; I'm trying to supply the
emulated machine with hardware that is not actually present.

There is src/env/video/matrox.c, which appears to be monkeying with PCI
stuff, but even that checks the real machine's /proc/pci and doesn't
run unless there's real hardware backing it (see matroxProbe(), which
incidentally is incorrectly commented as being MGAProbe).

Is this something that's already got hooks supporting it, or am I
breaking new ground here?

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: Emulated PCI devices?

[Stas Sergeev]

19.10.2016 00:45, Mouse пишет:
> I'm working with dosemu and would like to add an emulated PCI device to
> the emulated DOS machine.  Obviously, I know enough about the hardware
> I want to emulate to believe I can write an emulation of it.
>
> But I'm not sure where to hook it in.  (I'm working based on commit
> 18f6f5cdf1beceae8c7532718bfaf423e4a44f6a.)
Good luck.

>    I found
> src/base/async/pci_bios.c and src/base/dev/misc/pci.c, but they appear
> to be all about giving the emulated machine access to real PCI hardware
> on the real machine.  That's not what I want; I'm trying to supply the
> emulated machine with hardware that is not actually present.
>
> There is src/env/video/matrox.c, which appears to be monkeying with PCI
> stuff, but even that checks the real machine's /proc/pci and doesn't
> run unless there's real hardware backing it (see matroxProbe(), which
> incidentally is incorrectly commented as being MGAProbe).
>
> Is this something that's already got hooks supporting it, or am I
> breaking new ground here?
There was never any need to emulate the PCI device
for DOS. There are hardly too many DOS drivers for the
PCI devices. Maybe you can instead emulate the ISA version
of that device, which should be much simpler than adding
the PCI bus emulator. Or maybe you simply want to use qemu
because I can't think of the real use-cases where you want
to emulate some modern device under dosemu or dosbox.

Re: Emulated PCI devices?

[Mouse]

>> I'm working with dosemu and would like to add an emulated PCI device
>> to the emulated DOS machine.  [...]

>> Is this something that's already got hooks supporting it, or am I
>> breaking new ground here?

> There was never any need to emulate the PCI device for DOS.  There
> are hardly too many DOS drivers for the PCI devices.

In general, perhaps not. :-)

> Maybe you can instead emulate the ISA version of that device,

As far as I know, no ISA version has ever existed - and, indeed, the
particular device in question has just recently been EOLed by its
maker.

> Or maybe you simply want to use qemu

It may come to that; I'm working with dosemu because that was the first
DOS emulator we tried that we could make the software run in.  And, in
some respects, I prefer the way dosemu emulates DOS in much the same
sense that wine emulates Windows, instead of emulating hardware proper
and not caring what software is running on it.

> because I can't think of the real use-cases where you want to emulate
> some modern device under dosemu or dosbox.

Well, I daresay they are not common, but I've got one. :-)

I can't go into too much detail (NDAs and such), but it involves a
legacy application, running under DOS, which depends on a particular
(relatively rare and expensive) piece of hardware.  I got involved in
an effort to port/rewrite to make it run under something more modern -
but then the hardware was EOLed and the full port/rewrite is likely to
take longer than the company can keep going on their EOL-buy hardware
quantity.  Thus, we are looking at adding a little glue logic and
running the legacy code (unchanged or with minor tweaks) under
emulation, as a bridge until the full move to something more modern is
completed.

We'd rather keep the application basically unchanged (so that the same
binary can run on real machines with the special hardware, or on the
emulator), or I'd just rip out all the special hardware interfacing and
replace it with traps to the emulator.  I may have to do that with a
run-time switch based on a test for emulation anyway, but I figured
emulating the device was an approach worth at least looking at.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: Emulated PCI devices?

[Stas Sergeev]

19.10.2016 03:03, Mouse пишет:
>>> I'm working with dosemu and would like to add an emulated PCI device
>>> to the emulated DOS machine.  [...]
>>> Is this something that's already got hooks supporting it, or am I
>>> breaking new ground here?
>> There was never any need to emulate the PCI device for DOS.  There
>> are hardly too many DOS drivers for the PCI devices.
> In general, perhaps not. :-)
>
>> Maybe you can instead emulate the ISA version of that device,
> As far as I know, no ISA version has ever existed - and, indeed, the
> particular device in question has just recently been EOLed by its
> maker.
>
>> Or maybe you simply want to use qemu
> It may come to that; I'm working with dosemu because that was the first
> DOS emulator we tried that we could make the software run in.  And, in
> some respects, I prefer the way dosemu emulates DOS in much the same
> sense that wine emulates Windows, instead of emulating hardware proper
dosemu does emulate the HW and uses freedos (or any other
dos you ask it to boot).
You can look into dosbox that "emulates" dos.

> and not caring what software is running on it.
>
>> because I can't think of the real use-cases where you want to emulate
>> some modern device under dosemu or dosbox.
> Well, I daresay they are not common, but I've got one. :-)
>
> I can't go into too much detail (NDAs and such), but it involves a
> legacy application, running under DOS, which depends on a particular
> (relatively rare and expensive) piece of hardware.
If you want the DOS driver to communicate to that hardware,
then dosemu is fully prepared to that, because no emulation is
needed then. If you want to emulate the "very expensive hw"
without actually using the real hw, then I wonder what's the point.

Re: Emulated PCI devices?

[Mouse]

>> [...] the way dosemu emulates DOS in much the same sense that wine
>> emulates Windows, instead of emulating hardware proper [...]
> dosemu does emulate the HW and uses freedos (or any other dos you ask
> it to boot).

To a point, sure.  But, for example, it takes file I/O traps and
provides its own implementation, backed by the host filesystem, rather
than letting the FreeDOS implementation talk to (presumably emulated)
disk hardware.

>> [...] it involves a legacy application, running under DOS, which
>> depends on a particular (relatively rare and expensive) piece of
>> hardware.
> If you want the DOS driver to communicate to that hardware, [...].
> If you want to emulate the "very expensive hw" without actually using
> the real hw, then I wonder what's the point.

The point is to get the application running without the fancy hardware
(which is now EOLed) without having to rewrite the portions of the
application that talk to it.  The company has done an EOL buy of that
hardware, but that gives them only enough to last about half the time
we expect it to take to do the complete port/rewrite.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: Emulated PCI devices?

[Stas Sergeev]

20.10.2016 05:36, Mouse пишет:
>>> [...] the way dosemu emulates DOS in much the same sense that wine
>>> emulates Windows, instead of emulating hardware proper [...]
>> dosemu does emulate the HW and uses freedos (or any other dos you ask
>> it to boot).
> To a point, sure.  But, for example, it takes file I/O traps and
> provides its own implementation, backed by the host filesystem, rather
> than letting the FreeDOS implementation talk to (presumably emulated)
> disk hardware.
No, it does nothing of that kind.
freedos redirects the filesystem accesses via the network
redirector API. If not for freedos (or any other DOS) doing this,
you'll end up using a hdimage - dosemu supports hdimages too.

>>> [...] it involves a legacy application, running under DOS, which
>>> depends on a particular (relatively rare and expensive) piece of
>>> hardware.
>> If you want the DOS driver to communicate to that hardware, [...].
>> If you want to emulate the "very expensive hw" without actually using
>> the real hw, then I wonder what's the point.
> The point is to get the application running without the fancy hardware
> (which is now EOLed) without having to rewrite the portions of the
> application that talk to it.  The company has done an EOL buy of that
> hardware, but that gives them only enough to last about half the time
> we expect it to take to do the complete port/rewrite.
OK, maybe this is your case.
Its just that usually when you have some expensive HW, emulating
it in software is not enough because the HW is usually doing something
useful, not just makes some program to work. :) If you don't need
any functionality of that HW other than to make some DOS prog happy,
then I am afraid dosemu is not prepared for that, and you'll need to
implement the PCI emulator (in which case you can try qemu).

Re: Emulated PCI devices?

[Mouse]

> Its just that usually when you have some expensive HW, emulating it
> in software is not enough because the HW is usually doing something
> useful, not just makes some program to work. :)  If you don't need
> any functionality of that HW other than to make some DOS prog happy,

Sort of.  It's a (relatively-)high-speed input device, and we plan to
replace it with either a pre-canned data file or a network connection
as a data source.  But, even once the host machine has the data, we
have to get it into the program somehow.  I could replace all the
relevant hardware accesses with traps to the emulator, but I could also
simulate the hardware it's expecting.  I thought the latter was at
least worth looking at; it would have the benefit that it would be
running an identical software image to a real machine with real
hardware, making some kinds of debugging and verification easier.

> then I am afraid dosemu is not prepared for that, and you'll need to
> implement the PCI emulator (in which case you can try qemu).

OK, I'll talk with the other people invovled and we'll decide what tack
to take: hack on the program, add PCI emulation to dosemu, switch
emulators, whatever.

Many thanks for all your help.  Even if we end up switching away from
it, dosemu has been extremely helpful in this endeavour!

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse@rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

Re: Emulated PCI devices?

[Stas Sergeev]

22.10.2016 16:34, Mouse пишет:
>> Its just that usually when you have some expensive HW, emulating it
>> in software is not enough because the HW is usually doing something
>> useful, not just makes some program to work. :)  If you don't need
>> any functionality of that HW other than to make some DOS prog happy,
> Sort of.  It's a (relatively-)high-speed input device, and we plan to
> replace it with either a pre-canned data file or a network connection
> as a data source.  But, even once the host machine has the data, we
> have to get it into the program somehow.  I could replace all the
> relevant hardware accesses with traps to the emulator, but I could also
> simulate the hardware it's expecting.
I wonder if there is a big difference.
Any hardware access you don't have ioperm permissions to,
makes it into an emulator trap. So I very much suspect you are
talking differently about the same thing. If you patch every
in/out insn of your prog into hlt, nothing will really change other
than it will be more difficult to simulate.

>> then I am afraid dosemu is not prepared for that, and you'll need to
>> implement the PCI emulator (in which case you can try qemu).
> OK, I'll talk with the other people invovled and we'll decide what tack
> to take: hack on the program, add PCI emulation to dosemu, switch
> emulators, whatever.
Since dosemu is really a virtual machine (not like ntvdm or wine,
for example dosemu can run real windows31), and so is qemu,
you can get most of everything you had with dosemu, with qemu
too. Its just that dosemu sets up many things for you automatically,
which makes people think of it as of wine or ntvdm, but its not.
With qemu you'll have to do a lot of manual setup to get the
host FS access for example, but at the end you can get the
same results.
OTOH adding the PCI emulator to dosemu is not exceptionally
difficult, provided that it already has the PCI bios and emulation
of the config space ports. Its not like implementing everything
from scratches because you likely only need to emulate the
device config space. I suppose the device emulation itself will
take much more time anyway, esp if it uses DMA.