[Qemu-devel] : Memory problem with Qemu/SeaBIOS

[Qemu-devel] : Memory problem with Qemu/SeaBIOS

[Alain Ribière]

[-- Attachment #1.1: Type: text/plain, Size: 1285 bytes --]

Hello,

I have a memory trouble with an old DOS OS from Digital Research called Concurrent DOS.
It worked fine till Qemu v0.11.1.
I use Qemu to make a legacy application run without any modification.

Starting with Qemu 0.12, my legacy application has not enough memory to run.
Actually it seems that the SeaBIOS doesn't use the same memory mapping than PC-BIOS used to, causing trouble to my application.

Is there any way to get SeaBIOS use (more or less) the same memory amount and memory mapping than the older PC-BIOS ?

I tried to recomplie SeaBIOS 1.7.0 with less options but the result is the same.
Here is my .config file :

CONFIG_RELOCATE_INIT=y
CONFIG_ATA=y
CONFIG_FLOPPY=y
CONFIG_PS2PORT=y
CONFIG_SERIAL=y
CONFIG_LPT=y
CONFIG_DRIVES=y
CONFIG_OPTIONROMS=y
CONFIG_PMM=y
CONFIG_BOOT=y
CONFIG_KEYBOARD=y
CONFIG_KBD_CALL_INT15_4F=y
CONFIG_MOUSE=y
CONFIG_NO_VGABIOS=y
CONFIG_DEBUG_LEVEL=0

I also join two print screen of the same C-DOS utility. One with Qemu 1.0 and SeaBIOS 1.7.0 and the other with Qemu 0.10.0. The amount of memory is quite different between the versions.
If you want to try, get this floppy :
https://docs.google.com/open?id=0B7mz0vq6Rpb7Q0tYbmxMVzVqTms
and then type :
STOP

If you have any idea...


Thanks,
Alain

[-- Attachment #1.2: Type: text/html, Size: 1570 bytes --]

[-- Attachment #2: stop_qemu_0_10.png --]
[-- Type: image/png, Size: 7703 bytes --]

[-- Attachment #3: stop_qemu_1_0.png --]
[-- Type: image/png, Size: 7749 bytes --]

Re: [Qemu-devel] [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Kevin O'Connor]

On Mon, May 07, 2012 at 08:58:45AM -0700, Alain Ribière wrote:
> Hello,
> 
> I have a memory trouble with an old DOS OS from Digital Research called Concurrent DOS.
> It worked fine till Qemu v0.11.1.
> I use Qemu to make a legacy application run without any modification.
> 
> Starting with Qemu 0.12, my legacy application has not enough memory to run.
> Actually it seems that the SeaBIOS doesn't use the same memory mapping than PC-BIOS used to, causing trouble to my application.
> 
> Is there any way to get SeaBIOS use (more or less) the same memory amount and memory mapping than the older PC-BIOS ?
> 
> I tried to recomplie SeaBIOS 1.7.0 with less options but the result is the same.

When reporting an issue with SeaBIOS, please include the log obtained
when running qemu with "-chardev stdio,id=seabios -device
isa-debugcon,iobase=0x402,chardev=seabios".

I took a quick look at this, and I don't think this is a memory size
issue - the screenshots actually show that running under seabios makes
more memory available (15664 vs 15648).  Instead, I'd guess concurrent
dos is not making a "banked window" that is large enough.  I don't
know why the memory segments between 0x0b00 and 0x3400 aren't being
used.  There is no legacy bios interface to reserve memory under 1meg,
so I'm sure SeaBIOS isn't doing this wrong.  Indeed, SeaBIOS is much
more careful about touching ram under 1meg than Bochs bios is.

So, I'd guess something unrelated is confusing concurrent dos and the
above is just a symptom of the problem.  Unfortunately, I don't know
of any way to track that down without source code for the image and
spending a large amount of debugging time.

-Kevin

[Qemu-devel] Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Alain Ribière]

[-- Attachment #1.1: Type: text/plain, Size: 2436 bytes --]

Thanks for your answer.


Here is the debug log in attachement.
I disabled the debug on the screen shot I sent because I noticed I could get a bit more memory without it.
With the debug on, I got :
Banked Window  416K at 3800:0 segment
and a free memory of 15680K

I'll try to find what confuses C-DOS. It reacts a bit strangely... 
When I add or remove options from the SeaBIOS, the memory avalaible 
change and the size of banked window changes too. So I hoped there was a way to have enough memory to run my application...
Otherwise, I will have to stuck with Qemu 0.11 and PC-BIOS.

Do you know if there is any DOS utility to get the memory map ?

Thanks,

Alain 

________________________________
On Mon, May 07, 2012 at 08:58:45AM -0700, Alain Ribière wrote:
> Hello,
> 
> I have a memory trouble with an old DOS OS from Digital Research
 called Concurrent DOS.
> It worked fine till Qemu v0.11.1.
> I
 use Qemu to make a legacy application run without any modification.
> 
> Starting with Qemu 0.12, my legacy application has not enough memory to run.
> Actually it seems that the SeaBIOS doesn't use the same memory mapping than PC-BIOS used to, causing trouble to my application.
> 
> Is there any way to get SeaBIOS use (more or less) the same memory amount and memory mapping than the older PC-BIOS ?
> 
> I tried to recomplie SeaBIOS 1.7.0 with less options but the result is the same.

When reporting an issue with SeaBIOS, please include the log obtained
when running qemu with "-chardev stdio,id=seabios -device
isa-debugcon,iobase=0x402,chardev=seabios".

I took a quick look at this, and I don't think this is a memory size
issue - the screenshots actually show that running under seabios makes
more memory available (15664 vs 15648).  Instead, I'd guess concurrent
dos is not making a
 "banked window" that is large enough.  I don't
know why the memory segments between 0x0b00 and 0x3400 aren't being
used.  There is no legacy bios interface to reserve memory under 1meg,
so I'm sure SeaBIOS isn't doing this wrong.  Indeed, SeaBIOS is much
more careful about touching ram under 1meg than Bochs bios is.

So, I'd guess something unrelated is confusing concurrent dos and the
above is just a symptom of the problem.  Unfortunately, I don't know
of any way to track that down without source code for the image and
spending a large amount of debugging time.

-Kevin

[-- Attachment #1.2: Type: text/html, Size: 5757 bytes --]

[-- Attachment #2: seabios.log --]
[-- Type: application/octet-stream, Size: 2275 bytes --]

Start bios (version 1.7.0-20120508_175951-viper)
Ram Size=0x01000000 (0x0000000000000000 high)
Relocating init from 0x000f3790 to 0x00feb6c0 (size 18460)
CPU Mhz=2568
=== PCI bus & bridge init ===
PCI: pci_bios_init_bus_rec bus = 0x0
=== PCI device probing ===
Found 6 PCI devices (max PCI bus is 00)
=== PCI new allocation pass #1 ===
PCI: check devices
=== PCI new allocation pass #2 ===
PCI: init bases bus 0 (primary)
  type io max 40 sum 50 base c000
    size       40: 1 bar(s),     c000 ->     c03f
    size       10: 1 bar(s),     c040 ->     c04f
  type mem max 20000 sum 31000 base febc0000
    size    20000: 1 bar(s), febc0000 -> febdffff
    size    10000: 1 bar(s), febe0000 -> febeffff
    size     1000: 1 bar(s), febf0000 -> febf0fff
  type prefmem max 2000000 sum 2000000 base fc000000
    size  2000000: 1 bar(s), fc000000 -> fdffffff
PCI: map device bdf=00:00.0
PCI: map device bdf=00:01.0
PCI: map device bdf=00:01.1
  bar 4, addr c040, size 10 [io]
PCI: map device bdf=00:01.3
PCI: map device bdf=00:02.0
  bar 0, addr fc000000, size 2000000 [prefmem]
  bar 1, addr febf0000, size 1000 [mem]
  bar 6, addr febe0000, size 10000 [mem]
PCI: map device bdf=00:03.0
  bar 0, addr febc0000, size 20000 [mem]
  bar 1, addr c000, size 40 [io]
PCI: init bdf=00:00.0 id=8086:1237
PCI: init bdf=00:01.0 id=8086:7000
PCI: init bdf=00:01.1 id=8086:7010
PCI: init bdf=00:01.3 id=8086:7113
PCI: init bdf=00:02.0 id=1013:00b8
PCI: init bdf=00:03.0 id=8086:100e
PIIX3/PIIX4 init: elcr=00 0c
Found 1 cpu(s) max supported 1 cpu(s)
Scan for VGA option rom
Running option rom at c000:0003
Turning on vga text mode console
SeaBIOS (version 1.7.0-20120508_175951-viper)

PS2 keyboard initialized
Found 1 lpt ports
Found 1 serial ports
ATA controller 1 at 1f0/3f4/0 (irq 14 dev 9)
ATA controller 2 at 170/374/0 (irq 15 dev 9)
DVD/CD [ata1-0: QEMU DVD-ROM ATAPI-4 DVD/CD]
Scan for option roms
Returned 65536 bytes of ZoneHigh
e820 map has 5 items:
  0: 0000000000000000 - 000000000009fc00 = 1 RAM
  1: 000000000009fc00 - 00000000000a0000 = 2 RESERVED
  2: 00000000000f0000 - 0000000000100000 = 2 RESERVED
  3: 0000000000100000 - 0000000001000000 = 1 RAM
  4: 00000000fffc0000 - 0000000100000000 = 2 RESERVED
enter handle_19:
  NULL
Booting from Floppy...
Booting from 0000:7c00

[-- Attachment #3: stop_qemu_1_0_debug.jpg --]
[-- Type: image/jpeg, Size: 44947 bytes --]

Re: [Qemu-devel] Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Kevin O'Connor]

On Tue, May 08, 2012 at 09:47:56AM -0700, Alain Ribière wrote:
> Thanks for your answer.
> 
> Here is the debug log in attachement.
> I disabled the debug on the screen shot I sent because I noticed I could get a bit more memory without it.
> With the debug on, I got :
> Banked Window  416K at 3800:0 segment
> and a free memory of 15680K
> 
> I'll try to find what confuses C-DOS. It reacts a bit strangely... 
> When I add or remove options from the SeaBIOS, the memory avalaible 
> change and the size of banked window changes too. So I hoped there was a way to have enough memory to run my application...

The options don't impact SeaBIOS' memory usage, so this reafirms my
guess that what you are seeing is a secondary effect.  As a guess,
this could be stack usage - some old program call the bios with very
small stacks and if the stack overflows silent corruption can occur.
The fact that you're seeing different memory layouts with random
changes to config options could be due to different options causing
the compiler to layout the stack slightly differently and thus causing
slightly different corruption - just a guess.

The other thing I noticed with your C-DOS image is that it doesn't
know about the EBDA and uses that memory for its own purposes.  Thus,
the seabios runtime usage of the ebda along with c-dos could cause
conflicts.  I don't think this is the cause of your current issue
though as some moving the ebda in some quick tests I ran doesn't seem
to impact the output of "stop".

> Otherwise, I will have to stuck with Qemu 0.11 and PC-BIOS.
> 
> Do you know if there is any DOS utility to get the memory map ?

I'm not familiar with DOS utilities.  You could ask on the freedos
mailing list.

-Kevin

[Qemu-devel] Re : Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Alain Ribière]

[-- Attachment #1: Type: text/plain, Size: 4237 bytes --]

Tanks for your answer.


I have to apologize, I made a mistake. I rechecked my application and used git bisect.
So I found that this is not a SeaBIOS issue (actually it's more a C-DOS issues...). My problem comes from this Qemu commit :

49669fc551e0ccd2310a9584a9b7343a0bcae477 is the first bad commit
commit 49669fc551e0ccd2310a9584a9b7343a0bcae477
Author: Glauber Costa <glommer@redhat.com>
Date:   Thu Apr 23 15:20:55 2009 -0400

    suport device driver initialization model
    
    According to PnP specification, Appendix B, Option ROMs
    that support DDIM (device driver initialization model) should
    have their memory space writeable.
    
    KVM deviates from us here, by removing the IO_MEM_ROM flag,
    to allow for PCI option ROMs (they require DDIM). However,
    there's absolutely no reason we can't do the same.
    
    Signed-off-by: Glauber Costa <glommer@redhat.com>
    Signed-off-by: Anthony Liguori <aliguori@us.ibm.com>

:040000 040000 5186361da34236c2680af6ed568b5f5bdb2915ac c968308168e23ffa2678011ed2ff010389b528bd M      hw

diff --git a/hw/pc.c b/hw/pc.c
index 61f6e7b..340b313 100644
--- a/hw/pc.c
+++ b/hw/pc.c
@@ -920,7 +920,8 @@ static void pc_init1(ram_addr_t ram_size, int vga_ram_size,
 
     option_rom_offset = qemu_ram_alloc(0x20000);
     oprom_area_size = 0;
-    cpu_register_physical_memory(0xc0000, 0x20000, option_rom_offset);
+    cpu_register_physical_memory(0xc0000, 0x20000,
+                                 option_rom_offset | IO_MEM_ROM);
 
     if (using_vga) {
         /* VGA BIOS load */


Also I found the theorical memory map of the C-DOS in the documentation of my legacy application :
start         end

000000    9FFFF    Program area

0A0000    0BFFFF    Video memory

0C00000    0C7FFF    BIOS graphic card
0C8000    0CBFFF    BIOS cache controller
0CC000    0CFFFF    BIOS removable disk
0D0000    0D3FFF    BIOS network adapter
0D4000    0EFFFF    CDOS system
0F0000    0FFFFF    ROM BIOS

I wonder what is the CDOS system at 0D4000... It seems a bit strange.


I do not understand why this commit affects the C-DOS memory allocation.
But you're probably right, C-DOS is using memory addresses he shouldn't for it's own purpose causing conflicts...

If you have any idea.


Thanks, and sorry again for the wrong issue (I will remove the SeaBIOS mailing list in the future mails)...


Alain


________________________________

On Tue, May 08, 2012 at 09:47:56AM -0700, Alain Ribière wrote:
> Thanks for your answer.
> 
> Here is the debug log in attachement.
> I disabled the debug on the screen shot I sent because I noticed I could get a bit more memory without it.
> With the debug on, I got :
> Banked Window  416K at 3800:0 segment
> and a free memory of 15680K
> 
> I'll try to find what confuses C-DOS. It reacts a bit strangely... 
> When I add or remove options from the SeaBIOS, the memory avalaible 
> change and the size of banked window changes too. So I hoped there was a way to have enough memory to run my application...

The options don't impact SeaBIOS' memory usage, so this reafirms my
guess that what you are seeing is a secondary effect.  As a guess,
this could be stack usage - some old program call the bios with very
small stacks and if the stack overflows silent corruption can occur.
The fact that you're seeing different memory layouts with random
changes to config options could be due to different options causing
the compiler to layout the stack slightly differently and thus causing
slightly different corruption - just a guess.

The other thing I noticed with your C-DOS image is that it doesn't
know about the EBDA and uses that memory for its own purposes.  Thus,
the seabios runtime usage of the ebda along with c-dos could cause
conflicts.  I don't think this is the cause of your current issue
though as some moving the ebda in some quick tests I ran doesn't seem
to impact the output of "stop".

> Otherwise, I will have to stuck with Qemu 0.11 and PC-BIOS.
> 
> Do you know if there is any DOS utility to get the memory map ?

I'm not familiar with DOS utilities.  You could ask on the freedos
mailing list.

-Kevin

[-- Attachment #2: Type: text/html, Size: 6816 bytes --]

Re: [Qemu-devel] Re : Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Kevin O'Connor]

On Wed, May 09, 2012 at 08:57:53AM -0700, Alain Ribière wrote:
> Tanks for your answer.
> 
> 
> I have to apologize, I made a mistake. I rechecked my application and used git bisect.
> So I found that this is not a SeaBIOS issue (actually it's more a C-DOS issues...). My problem comes from this Qemu commit :
> 
> 49669fc551e0ccd2310a9584a9b7343a0bcae477 is the first bad commit
> commit 49669fc551e0ccd2310a9584a9b7343a0bcae477
[...]
> 
> Also I found the theorical memory map of the C-DOS in the documentation of my legacy application :
> start         end
> 
> 000000    9FFFF    Program area
> 
> 0A0000    0BFFFF    Video memory
> 
> 0C00000    0C7FFF    BIOS graphic card
> 0C8000    0CBFFF    BIOS cache controller
> 0CC000    0CFFFF    BIOS removable disk
> 0D0000    0D3FFF    BIOS network adapter
> 0D4000    0EFFFF    CDOS system
> 0F0000    0FFFFF    ROM BIOS
> 
> I wonder what is the CDOS system at 0D4000... It seems a bit strange.

Very odd.  This is still controlled by seabios.  Indeed, if I apply
this (clearly incorrect) patch to seabios:

diff --git a/src/shadow.c b/src/shadow.c
index c0c8cc2..73c0670 100644
--- a/src/shadow.c
+++ b/src/shadow.c
@@ -83,7 +83,7 @@ make_bios_readonly_intel(u16 bdf, u32 pam0)
     for (i=0; i<6; i++) {
         u32 mem = BUILD_ROM_START + i * 32*1024;
         u32 pam = pam0 + 1 + i;
-        if (RomEnd <= mem + 16*1024) {
+        if (0 && RomEnd <= mem + 16*1024) {
             if (RomEnd > mem)
                 pci_config_writeb(bdf, pam, 0x31);
             break;

Then the C-DOS image you posted seems happier.  Basically, if C-DOS is
allowed to write to the e-segment then it seems to do weird things.

> 
> I do not understand why this commit affects the C-DOS memory allocation.
> But you're probably right, C-DOS is using memory addresses he shouldn't for it's own purpose causing conflicts...
> 
> If you have any idea.
> 
> 
> Thanks, and sorry again for the wrong issue (I will remove the SeaBIOS mailing list in the future mails)...
> 

This is still in the SeaBIOS realm.  (Though I'm thinking it's really
a C-DOS bug.)

-Kevin

Re: [Qemu-devel] Re : Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Kevin O'Connor]

On Wed, May 09, 2012 at 08:44:45PM -0400, Kevin O'Connor wrote:
> On Wed, May 09, 2012 at 08:57:53AM -0700, Alain Ribière wrote:
> > I wonder what is the CDOS system at 0D4000... It seems a bit strange.
> 
> Very odd.  This is still controlled by seabios.  Indeed, if I apply
> this (clearly incorrect) patch to seabios:
> 
> diff --git a/src/shadow.c b/src/shadow.c
> index c0c8cc2..73c0670 100644
> --- a/src/shadow.c
> +++ b/src/shadow.c
[...]

Just for kicks, I tried a simple hack to try and reserve more memory
in the c/d/e-segments, and it seems to work for the image you posted.
Run:

python -c 'import sys; sys.stdout.write("\x55\xaa\xc8\xcb\x6e" + "\x00"*(100*1024 - 5))' > dummy.rom

and then add "-option-rom dummy.rom" to your qemu command line.  (The
idea is to build a 100K dummy option rom that seabios will load and
mark as read-only memory.)

-Kevin

[Qemu-devel] Re : Re : Re : [SeaBIOS] : Memory problem with Qemu/SeaBIOS

[Alain Ribière]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

Both solutions work for my application !
Thanks a lot. I think I will use one of them, even if I know it's not really "clean".

Thanks again,

Alain




________________________________

On Wed, May 09, 2012 at 08:44:45PM -0400, Kevin O'Connor wrote:
> On Wed, May 09, 2012 at 08:57:53AM -0700, Alain Ribière wrote:
> > I wonder what is the CDOS system at 0D4000... It seems a bit strange.
> 
> Very odd.  This is still controlled by seabios.  Indeed, if I apply
> this (clearly incorrect) patch to seabios:
> 
> diff --git a/src/shadow.c b/src/shadow.c
> index c0c8cc2..73c0670 100644
> --- a/src/shadow.c
> +++ b/src/shadow.c
[...]

Just for kicks, I tried a simple hack to try and reserve more memory
in the c/d/e-segments, and it seems to work for the image you posted.
Run:

python -c 'import sys; sys.stdout.write("\x55\xaa\xc8\xcb\x6e" + "\x00"*(100*1024 - 5))' > dummy.rom

and then add "-option-rom dummy.rom" to your qemu command line.  (The
idea is to build a 100K dummy option rom that seabios will load and
mark as read-only memory.)

-Kevin

[-- Attachment #2: Type: text/html, Size: 1643 bytes --]