* [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts @ 2012-07-18 16:37 Stefan Weil 2012-07-20 3:53 ` Alexey Kardashevskiy 0 siblings, 1 reply; 8+ messages in thread From: Stefan Weil @ 2012-07-18 16:37 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: Stefan Weil, qemu-devel 32-bit applications on Windows normally only get virtual memory in the lower 2 GiB address space. Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB of contiguous virtual memory in that address space. Therefore running system emulations with 1 GiB or more RAM will abort with a failure. The linker flag --large-address-aware allows addresses in the upper 2 GiB. With this flag, it is possible to run emulated machines with up to 2047 MiB RAM. Signed-off-by: Stefan Weil <sw@weilnetz.de> --- I tested the executables with large address awareness on a 64 bit Windows (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs when 1024 or more MiB are requested). Maybe the support for large addresses is broken in my Wine version. Please report any different test results. Regards, Stefan Weil configure | 4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/configure b/configure index df5c99a..b681f9a 100755 --- a/configure +++ b/configure @@ -2937,9 +2937,9 @@ else POD2MAN="pod2man" fi -# Use ASLR, no-SEH and DEP if available +# Use ASLR, large addresses, no-SEH and DEP if available if test "$mingw32" = "yes" ; then - for flag in --dynamicbase --no-seh --nxcompat; do + for flag in --dynamicbase --large-address-aware --no-seh --nxcompat; do if $ld --help 2>/dev/null | grep ".$flag" >/dev/null 2>/dev/null ; then LDFLAGS="-Wl,$flag $LDFLAGS" fi -- 1.7.0.4 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-18 16:37 [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts Stefan Weil @ 2012-07-20 3:53 ` Alexey Kardashevskiy 2012-07-20 5:23 ` Stefan Weil 0 siblings, 1 reply; 8+ messages in thread From: Alexey Kardashevskiy @ 2012-07-20 3:53 UTC (permalink / raw) To: Stefan Weil; +Cc: qemu-devel On 19/07/12 02:37, Stefan Weil wrote: > 32-bit applications on Windows normally only get virtual memory in > the lower 2 GiB address space. > > Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB > of contiguous virtual memory in that address space. Therefore running > system emulations with 1 GiB or more RAM will abort with a failure. > > The linker flag --large-address-aware allows addresses in the upper 2 GiB. > With this flag, it is possible to run emulated machines with up to > 2047 MiB RAM. > > Signed-off-by: Stefan Weil <sw@weilnetz.de> > --- > > I tested the executables with large address awareness on a 64 bit Windows > (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs > when 1024 or more MiB are requested). > > Maybe the support for large addresses is broken in my Wine version. > Please report any different test results. I tried with native WindowsXP, the patch did not help. QEMU was compiled with the new parameter: Install prefix c:/Program Files/QEMU BIOS directory c:/Program Files/QEMU binary directory c:/Program Files/QEMU library directory c:/Program Files/QEMU/lib include directory c:/Program Files/QEMU/include config directory c:/Program Files/QEMU Source path /home/aik/testken/qemu C compiler i686-pc-mingw32-gcc Host C compiler gcc CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers -Wold-style-declaration -Wold-style-definition -Wtype-limits LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g make make install install python python smbd /usr/sbin/smbd host CPU i386 host big endian no target list ppc64-softmmu [...] qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe ***qemu_memalign 200 (512) size 800 (2048) Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) ***qemu_vmalloc 40000000 (1073741824) Failed to allocate memory: 8 This application has requested the Runtime to terminate it in an unusual way. Please contact the application's support team for more information. The lines with *** are mine printf("%s %x(%u)", __func__, etc). It still works with "-m 1023". > Regards, > > Stefan Weil > > > configure | 4 ++-- > 1 files changed, 2 insertions(+), 2 deletions(-) > > diff --git a/configure b/configure > index df5c99a..b681f9a 100755 > --- a/configure > +++ b/configure > @@ -2937,9 +2937,9 @@ else > POD2MAN="pod2man" > fi > > -# Use ASLR, no-SEH and DEP if available > +# Use ASLR, large addresses, no-SEH and DEP if available > if test "$mingw32" = "yes" ; then > - for flag in --dynamicbase --no-seh --nxcompat; do > + for flag in --dynamicbase --large-address-aware --no-seh --nxcompat; do > if $ld --help 2>/dev/null | grep ".$flag" >/dev/null 2>/dev/null ; then > LDFLAGS="-Wl,$flag $LDFLAGS" > fi > -- Alexey ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 3:53 ` Alexey Kardashevskiy @ 2012-07-20 5:23 ` Stefan Weil 2012-07-20 5:37 ` Alexey Kardashevskiy 0 siblings, 1 reply; 8+ messages in thread From: Stefan Weil @ 2012-07-20 5:23 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: qemu-devel Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: > On 19/07/12 02:37, Stefan Weil wrote: >> 32-bit applications on Windows normally only get virtual memory in >> the lower 2 GiB address space. >> >> Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB >> of contiguous virtual memory in that address space. Therefore running >> system emulations with 1 GiB or more RAM will abort with a failure. >> >> The linker flag --large-address-aware allows addresses in the upper 2 GiB. >> With this flag, it is possible to run emulated machines with up to >> 2047 MiB RAM. >> >> Signed-off-by: Stefan Weil <sw@weilnetz.de> >> --- >> >> I tested the executables with large address awareness on a 64 bit Windows >> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs >> when 1024 or more MiB are requested). >> >> Maybe the support for large addresses is broken in my Wine version. >> Please report any different test results. > > I tried with native WindowsXP, the patch did not help. > > QEMU was compiled with the new parameter: > > Install prefix c:/Program Files/QEMU > BIOS directory c:/Program Files/QEMU > binary directory c:/Program Files/QEMU > library directory c:/Program Files/QEMU/lib > include directory c:/Program Files/QEMU/include > config directory c:/Program Files/QEMU > Source path /home/aik/testken/qemu > C compiler i686-pc-mingw32-gcc > Host C compiler gcc > CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g > QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN > -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE > -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings > -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all > -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs > -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers > -Wold-style-declaration -Wold-style-definition -Wtype-limits > LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware > -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g > make make > install install > python python > smbd /usr/sbin/smbd > host CPU i386 > host big endian no > target list ppc64-softmmu > [...] > > > > qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L > qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel > ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 > -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe > ***qemu_memalign 200 (512) size 800 (2048) > Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) > ***qemu_vmalloc 40000000 (1073741824) > Failed to allocate memory: 8 > > This application has requested the Runtime to terminate it in an unusual way. > Please contact the application's support team for more information. > > > The lines with *** are mine printf("%s %x(%u)", __func__, etc). > > It still works with "-m 1023". Do you have enough RAM? Even with 64 bit Windows 7, I am limited to -m 7000 with 64 bit QEMU on a 8 GiB host. 32 bit QEMU runs there with up to 2047 MiB. On 32 bit Windows XP, I'd expect that you need 2 GiB (or better 3 GiB) of physical RAM in your host, and close all other programs before you try QEMU. Maybe I can run a test with Windows XP during the weekend. By the way: the patch must be modified to work for 64 bit QEMU. MinGW-w64's ld shows the new parameter in its help, but does not support it for 64 bit executables. Regards, Stefan ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 5:23 ` Stefan Weil @ 2012-07-20 5:37 ` Alexey Kardashevskiy 2012-07-20 6:05 ` Alexey Kardashevskiy 0 siblings, 1 reply; 8+ messages in thread From: Alexey Kardashevskiy @ 2012-07-20 5:37 UTC (permalink / raw) To: Stefan Weil; +Cc: qemu-devel On 20/07/12 15:23, Stefan Weil wrote: > Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: >> On 19/07/12 02:37, Stefan Weil wrote: >>> 32-bit applications on Windows normally only get virtual memory in >>> the lower 2 GiB address space. >>> >>> Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB >>> of contiguous virtual memory in that address space. Therefore running >>> system emulations with 1 GiB or more RAM will abort with a failure. >>> >>> The linker flag --large-address-aware allows addresses in the upper 2 GiB. >>> With this flag, it is possible to run emulated machines with up to >>> 2047 MiB RAM. >>> >>> Signed-off-by: Stefan Weil <sw@weilnetz.de> >>> --- >>> >>> I tested the executables with large address awareness on a 64 bit Windows >>> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs >>> when 1024 or more MiB are requested). >>> >>> Maybe the support for large addresses is broken in my Wine version. >>> Please report any different test results. >> >> I tried with native WindowsXP, the patch did not help. >> >> QEMU was compiled with the new parameter: >> >> Install prefix c:/Program Files/QEMU >> BIOS directory c:/Program Files/QEMU >> binary directory c:/Program Files/QEMU >> library directory c:/Program Files/QEMU/lib >> include directory c:/Program Files/QEMU/include >> config directory c:/Program Files/QEMU >> Source path /home/aik/testken/qemu >> C compiler i686-pc-mingw32-gcc >> Host C compiler gcc >> CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g >> QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN >> -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE >> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >> -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all >> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >> -Wold-style-declaration -Wold-style-definition -Wtype-limits >> LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware >> -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g >> make make >> install install >> python python >> smbd /usr/sbin/smbd >> host CPU i386 >> host big endian no >> target list ppc64-softmmu >> [...] >> >> >> >> qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L >> qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel >> ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 >> -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe >> ***qemu_memalign 200 (512) size 800 (2048) >> Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) >> ***qemu_vmalloc 40000000 (1073741824) >> Failed to allocate memory: 8 >> >> This application has requested the Runtime to terminate it in an unusual way. >> Please contact the application's support team for more information. >> >> >> The lines with *** are mine printf("%s %x(%u)", __func__, etc). >> >> It still works with "-m 1023". > > Do you have enough RAM? Even with 64 bit Windows 7, I am > limited to -m 7000 with 64 bit QEMU on a 8 GiB host. > 32 bit QEMU runs there with up to 2047 MiB. I put another VirtualAlloc(64MB) call in qemu_vmalloc(1023MB) to check if it fails but it did not so it is possible to allocate 1087MB but in 2 pieces. > On 32 bit Windows XP, I'd expect that you need 2 GiB (or better > 3 GiB) of physical RAM in your host, and close all other > programs before you try QEMU. How do other programs matter? It is virtual 2GB per process, no? Ah, anyway, it has 2.994.340KB of physical RAM, swapping is enabled, the task manager shows at least "2000000" in "Available", one cygwin shell is running. And QEMU can always alloc 1023MB (in one chunk) and never 1024MB (also in one chunk). For me it looks like XP limitation but I could not find any mention of it in MSDN. > Maybe I can run a test with Windows XP during the weekend. > > By the way: the patch must be modified to work for 64 bit QEMU. > MinGW-w64's ld shows the new parameter in its help, but does > not support it for 64 bit executables. Not sure if I understood :) The compiler I use is "i686-pc-mingw32-gcc -m32", target is ppc64-softmmu. -- Alexey ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 5:37 ` Alexey Kardashevskiy @ 2012-07-20 6:05 ` Alexey Kardashevskiy 2012-07-20 6:38 ` Alexey Kardashevskiy 0 siblings, 1 reply; 8+ messages in thread From: Alexey Kardashevskiy @ 2012-07-20 6:05 UTC (permalink / raw) To: Stefan Weil; +Cc: qemu-devel On 20/07/12 15:37, Alexey Kardashevskiy wrote: > On 20/07/12 15:23, Stefan Weil wrote: >> Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: >>> On 19/07/12 02:37, Stefan Weil wrote: >>>> 32-bit applications on Windows normally only get virtual memory in >>>> the lower 2 GiB address space. >>>> >>>> Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB >>>> of contiguous virtual memory in that address space. Therefore running >>>> system emulations with 1 GiB or more RAM will abort with a failure. >>>> >>>> The linker flag --large-address-aware allows addresses in the upper 2 GiB. >>>> With this flag, it is possible to run emulated machines with up to >>>> 2047 MiB RAM. >>>> >>>> Signed-off-by: Stefan Weil <sw@weilnetz.de> >>>> --- >>>> >>>> I tested the executables with large address awareness on a 64 bit Windows >>>> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs >>>> when 1024 or more MiB are requested). >>>> >>>> Maybe the support for large addresses is broken in my Wine version. >>>> Please report any different test results. >>> >>> I tried with native WindowsXP, the patch did not help. >>> >>> QEMU was compiled with the new parameter: >>> >>> Install prefix c:/Program Files/QEMU >>> BIOS directory c:/Program Files/QEMU >>> binary directory c:/Program Files/QEMU >>> library directory c:/Program Files/QEMU/lib >>> include directory c:/Program Files/QEMU/include >>> config directory c:/Program Files/QEMU >>> Source path /home/aik/testken/qemu >>> C compiler i686-pc-mingw32-gcc >>> Host C compiler gcc >>> CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g >>> QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN >>> -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE >>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >>> -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all >>> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >>> -Wold-style-declaration -Wold-style-definition -Wtype-limits >>> LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware >>> -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g >>> make make >>> install install >>> python python >>> smbd /usr/sbin/smbd >>> host CPU i386 >>> host big endian no >>> target list ppc64-softmmu >>> [...] >>> >>> >>> >>> qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L >>> qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel >>> ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 >>> -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe >>> ***qemu_memalign 200 (512) size 800 (2048) >>> Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) >>> ***qemu_vmalloc 40000000 (1073741824) >>> Failed to allocate memory: 8 >>> >>> This application has requested the Runtime to terminate it in an unusual way. >>> Please contact the application's support team for more information. >>> >>> >>> The lines with *** are mine printf("%s %x(%u)", __func__, etc). >>> >>> It still works with "-m 1023". >> >> Do you have enough RAM? Even with 64 bit Windows 7, I am >> limited to -m 7000 with 64 bit QEMU on a 8 GiB host. >> 32 bit QEMU runs there with up to 2047 MiB. > > > I put another VirtualAlloc(64MB) call in qemu_vmalloc(1023MB) to check if > it fails but it did not so it is possible to allocate 1087MB but in 2 pieces. > > >> On 32 bit Windows XP, I'd expect that you need 2 GiB (or better >> 3 GiB) of physical RAM in your host, and close all other >> programs before you try QEMU. > > How do other programs matter? It is virtual 2GB per process, no? > Ah, anyway, it has 2.994.340KB of physical RAM, swapping is enabled, the > task manager shows at least "2000000" in "Available", one cygwin shell is > running. > And QEMU can always alloc 1023MB (in one chunk) and never 1024MB (also in > one chunk). For me it looks like XP limitation but I could not find any > mention of it in MSDN. > > >> Maybe I can run a test with Windows XP during the weekend. >> >> By the way: the patch must be modified to work for 64 bit QEMU. >> MinGW-w64's ld shows the new parameter in its help, but does >> not support it for 64 bit executables. > > > Not sure if I understood :) > The compiler I use is "i686-pc-mingw32-gcc -m32", target is ppc64-softmmu. > > Did a simple test inside QEMU, ran it twice: run#1: first alloc 64MB, then 1023MB - failed. va(0x4000000) -> ptr=1d0d0000, last error = 0 va(0x3FF00000) -> ptr=00000000, last error = 8 run#2: first alloc 1023MB, then 64MB - succeeded. va(0x3FF00000) -> ptr=1d0d0000, last error = 0 va(0x4000000) -> ptr=5d130000, last error = 0 As we can see, there is 340000h gap between 2 chunks. Have no idea what that means :) May be something is wrong with my machine, dunno. -- Alexey ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 6:05 ` Alexey Kardashevskiy @ 2012-07-20 6:38 ` Alexey Kardashevskiy 2012-07-20 17:19 ` Stefan Weil 0 siblings, 1 reply; 8+ messages in thread From: Alexey Kardashevskiy @ 2012-07-20 6:38 UTC (permalink / raw) To: Stefan Weil; +Cc: qemu-devel On 20/07/12 16:05, Alexey Kardashevskiy wrote: > On 20/07/12 15:37, Alexey Kardashevskiy wrote: >> On 20/07/12 15:23, Stefan Weil wrote: >>> Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: >>>> On 19/07/12 02:37, Stefan Weil wrote: >>>>> 32-bit applications on Windows normally only get virtual memory in >>>>> the lower 2 GiB address space. >>>>> >>>>> Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB >>>>> of contiguous virtual memory in that address space. Therefore running >>>>> system emulations with 1 GiB or more RAM will abort with a failure. >>>>> >>>>> The linker flag --large-address-aware allows addresses in the upper 2 GiB. >>>>> With this flag, it is possible to run emulated machines with up to >>>>> 2047 MiB RAM. >>>>> >>>>> Signed-off-by: Stefan Weil <sw@weilnetz.de> >>>>> --- >>>>> >>>>> I tested the executables with large address awareness on a 64 bit Windows >>>>> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs >>>>> when 1024 or more MiB are requested). >>>>> >>>>> Maybe the support for large addresses is broken in my Wine version. >>>>> Please report any different test results. >>>> >>>> I tried with native WindowsXP, the patch did not help. >>>> >>>> QEMU was compiled with the new parameter: >>>> >>>> Install prefix c:/Program Files/QEMU >>>> BIOS directory c:/Program Files/QEMU >>>> binary directory c:/Program Files/QEMU >>>> library directory c:/Program Files/QEMU/lib >>>> include directory c:/Program Files/QEMU/include >>>> config directory c:/Program Files/QEMU >>>> Source path /home/aik/testken/qemu >>>> C compiler i686-pc-mingw32-gcc >>>> Host C compiler gcc >>>> CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g >>>> QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN >>>> -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE >>>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >>>> -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all >>>> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >>>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >>>> -Wold-style-declaration -Wold-style-definition -Wtype-limits >>>> LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware >>>> -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g >>>> make make >>>> install install >>>> python python >>>> smbd /usr/sbin/smbd >>>> host CPU i386 >>>> host big endian no >>>> target list ppc64-softmmu >>>> [...] >>>> >>>> >>>> >>>> qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L >>>> qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel >>>> ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 >>>> -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe >>>> ***qemu_memalign 200 (512) size 800 (2048) >>>> Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) >>>> ***qemu_vmalloc 40000000 (1073741824) >>>> Failed to allocate memory: 8 >>>> >>>> This application has requested the Runtime to terminate it in an unusual way. >>>> Please contact the application's support team for more information. >>>> >>>> >>>> The lines with *** are mine printf("%s %x(%u)", __func__, etc). >>>> >>>> It still works with "-m 1023". >>> >>> Do you have enough RAM? Even with 64 bit Windows 7, I am >>> limited to -m 7000 with 64 bit QEMU on a 8 GiB host. >>> 32 bit QEMU runs there with up to 2047 MiB. >> >> >> I put another VirtualAlloc(64MB) call in qemu_vmalloc(1023MB) to check if >> it fails but it did not so it is possible to allocate 1087MB but in 2 pieces. >> >> >>> On 32 bit Windows XP, I'd expect that you need 2 GiB (or better >>> 3 GiB) of physical RAM in your host, and close all other >>> programs before you try QEMU. >> >> How do other programs matter? It is virtual 2GB per process, no? >> Ah, anyway, it has 2.994.340KB of physical RAM, swapping is enabled, the >> task manager shows at least "2000000" in "Available", one cygwin shell is >> running. >> And QEMU can always alloc 1023MB (in one chunk) and never 1024MB (also in >> one chunk). For me it looks like XP limitation but I could not find any >> mention of it in MSDN. >> >> >>> Maybe I can run a test with Windows XP during the weekend. >>> >>> By the way: the patch must be modified to work for 64 bit QEMU. >>> MinGW-w64's ld shows the new parameter in its help, but does >>> not support it for 64 bit executables. >> >> >> Not sure if I understood :) >> The compiler I use is "i686-pc-mingw32-gcc -m32", target is ppc64-softmmu. >> >> > > Did a simple test inside QEMU, ran it twice: > > run#1: first alloc 64MB, then 1023MB - failed. > va(0x4000000) -> ptr=1d0d0000, last error = 0 > va(0x3FF00000) -> ptr=00000000, last error = 8 > > run#2: first alloc 1023MB, then 64MB - succeeded. > va(0x3FF00000) -> ptr=1d0d0000, last error = 0 > va(0x4000000) -> ptr=5d130000, last error = 0 > > As we can see, there is 340000h gap between 2 chunks. > > Have no idea what that means :) May be something is wrong with my machine, > dunno. Looked further, c:\windows\system32\sysctl32.dll is mapped there so there is just no big chunk of RAM. But still there is some RAM, it is just fragmented. -- Alexey ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 6:38 ` Alexey Kardashevskiy @ 2012-07-20 17:19 ` Stefan Weil 2012-07-23 18:32 ` Blue Swirl 0 siblings, 1 reply; 8+ messages in thread From: Stefan Weil @ 2012-07-20 17:19 UTC (permalink / raw) To: Alexey Kardashevskiy; +Cc: qemu-devel Am 20.07.2012 08:38, schrieb Alexey Kardashevskiy: > On 20/07/12 16:05, Alexey Kardashevskiy wrote: >> On 20/07/12 15:37, Alexey Kardashevskiy wrote: >>> On 20/07/12 15:23, Stefan Weil wrote: >>>> Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: >>>>> On 19/07/12 02:37, Stefan Weil wrote: >>>>>> 32-bit applications on Windows normally only get virtual memory in >>>>>> the lower 2 GiB address space. >>>>>> >>>>>> Because of memory fragmentation, VirtualAlloc() usually won't get 1 GiB >>>>>> of contiguous virtual memory in that address space. Therefore running >>>>>> system emulations with 1 GiB or more RAM will abort with a failure. >>>>>> >>>>>> The linker flag --large-address-aware allows addresses in the upper 2 GiB. >>>>>> With this flag, it is possible to run emulated machines with up to >>>>>> 2047 MiB RAM. >>>>>> >>>>>> Signed-off-by: Stefan Weil<sw@weilnetz.de> >>>>>> --- >>>>>> >>>>>> I tested the executables with large address awareness on a 64 bit Windows >>>>>> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but hangs >>>>>> when 1024 or more MiB are requested). >>>>>> >>>>>> Maybe the support for large addresses is broken in my Wine version. >>>>>> Please report any different test results. >>>>> >>>>> I tried with native WindowsXP, the patch did not help. >>>>> >>>>> QEMU was compiled with the new parameter: >>>>> >>>>> Install prefix c:/Program Files/QEMU >>>>> BIOS directory c:/Program Files/QEMU >>>>> binary directory c:/Program Files/QEMU >>>>> library directory c:/Program Files/QEMU/lib >>>>> include directory c:/Program Files/QEMU/include >>>>> config directory c:/Program Files/QEMU >>>>> Source path /home/aik/testken/qemu >>>>> C compiler i686-pc-mingw32-gcc >>>>> Host C compiler gcc >>>>> CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g >>>>> QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 -DWIN32_LEAN_AND_MEAN >>>>> -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE >>>>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >>>>> -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all >>>>> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >>>>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >>>>> -Wold-style-declaration -Wold-style-definition -Wtype-limits >>>>> LDFLAGS -Wl,--nxcompat -Wl,--no-seh -Wl,--large-address-aware >>>>> -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g >>>>> make make >>>>> install install >>>>> python python >>>>> smbd /usr/sbin/smbd >>>>> host CPU i386 >>>>> host big endian no >>>>> target list ppc64-softmmu >>>>> [...] >>>>> >>>>> >>>>> >>>>> qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L >>>>> qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel >>>>> ./guest.vmlinux.n -net nic,model=virtio -net user,hostfwd=tcp::5555-:22 >>>>> -drive file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe >>>>> ***qemu_memalign 200 (512) size 800 (2048) >>>>> Warning: Disabling some instructions which are not emulated by TCG (0x0, 0x6) >>>>> ***qemu_vmalloc 40000000 (1073741824) >>>>> Failed to allocate memory: 8 >>>>> >>>>> This application has requested the Runtime to terminate it in an unusual way. >>>>> Please contact the application's support team for more information. >>>>> >>>>> >>>>> The lines with *** are mine printf("%s %x(%u)", __func__, etc). >>>>> >>>>> It still works with "-m 1023". >>>> >>>> Do you have enough RAM? Even with 64 bit Windows 7, I am >>>> limited to -m 7000 with 64 bit QEMU on a 8 GiB host. >>>> 32 bit QEMU runs there with up to 2047 MiB. >>> >>> >>> I put another VirtualAlloc(64MB) call in qemu_vmalloc(1023MB) to check if >>> it fails but it did not so it is possible to allocate 1087MB but in 2 pieces. >>> >>> >>>> On 32 bit Windows XP, I'd expect that you need 2 GiB (or better >>>> 3 GiB) of physical RAM in your host, and close all other >>>> programs before you try QEMU. >>> >>> How do other programs matter? It is virtual 2GB per process, no? >>> Ah, anyway, it has 2.994.340KB of physical RAM, swapping is enabled, the >>> task manager shows at least "2000000" in "Available", one cygwin shell is >>> running. >>> And QEMU can always alloc 1023MB (in one chunk) and never 1024MB (also in >>> one chunk). For me it looks like XP limitation but I could not find any >>> mention of it in MSDN. >>> >>> >>>> Maybe I can run a test with Windows XP during the weekend. >>>> >>>> By the way: the patch must be modified to work for 64 bit QEMU. >>>> MinGW-w64's ld shows the new parameter in its help, but does >>>> not support it for 64 bit executables. >>> >>> >>> Not sure if I understood :) >>> The compiler I use is "i686-pc-mingw32-gcc -m32", target is ppc64-softmmu. >>> >>> >> >> Did a simple test inside QEMU, ran it twice: >> >> run#1: first alloc 64MB, then 1023MB - failed. >> va(0x4000000) -> ptr=1d0d0000, last error = 0 >> va(0x3FF00000) -> ptr=00000000, last error = 8 >> >> run#2: first alloc 1023MB, then 64MB - succeeded. >> va(0x3FF00000) -> ptr=1d0d0000, last error = 0 >> va(0x4000000) -> ptr=5d130000, last error = 0 >> >> As we can see, there is 340000h gap between 2 chunks. >> >> Have no idea what that means :) May be something is wrong with my machine, >> dunno. > > Looked further, c:\windows\system32\sysctl32.dll is mapped there so there > is just no big chunk of RAM. But still there is some RAM, it is just > fragmented. Yes, that's the problem: the QEMU executable and all the DLLs occupy RAM in the range from 0 to 2 GiB, so this area is very fragmented and it's impossible to get a large contiguous memory area (which VirtualAlloc requires!). On 64 bit Windows, addresses from 2 GiB to 4 GiB can be used with --large-address-aware, so there is no such problem. With 64 bit executables, even more RAM above 4 GiB is available. If you have enough RAM, you can configure a 32 bit Windows to provide 3 GiB instead of 2 GiB to user applications. This non-standard configuration also allows larger sizes for VirtualAlloc. See http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=vs.85%29.aspx for more information. See http://qemu.weilnetz.de/w32/virtualalloc-test/ for a simple test program. Regards, Stefan Weil ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts 2012-07-20 17:19 ` Stefan Weil @ 2012-07-23 18:32 ` Blue Swirl 0 siblings, 0 replies; 8+ messages in thread From: Blue Swirl @ 2012-07-23 18:32 UTC (permalink / raw) To: Stefan Weil; +Cc: Alexey Kardashevskiy, qemu-devel On Fri, Jul 20, 2012 at 5:19 PM, Stefan Weil <sw@weilnetz.de> wrote: > Am 20.07.2012 08:38, schrieb Alexey Kardashevskiy: > >> On 20/07/12 16:05, Alexey Kardashevskiy wrote: >>> >>> On 20/07/12 15:37, Alexey Kardashevskiy wrote: >>>> >>>> On 20/07/12 15:23, Stefan Weil wrote: >>>>> >>>>> Am 20.07.2012 05:53, schrieb Alexey Kardashevskiy: >>>>>> >>>>>> On 19/07/12 02:37, Stefan Weil wrote: >>>>>>> >>>>>>> 32-bit applications on Windows normally only get virtual memory in >>>>>>> the lower 2 GiB address space. >>>>>>> >>>>>>> Because of memory fragmentation, VirtualAlloc() usually won't get 1 >>>>>>> GiB >>>>>>> of contiguous virtual memory in that address space. Therefore running >>>>>>> system emulations with 1 GiB or more RAM will abort with a failure. >>>>>>> >>>>>>> The linker flag --large-address-aware allows addresses in the upper 2 >>>>>>> GiB. >>>>>>> With this flag, it is possible to run emulated machines with up to >>>>>>> 2047 MiB RAM. >>>>>>> >>>>>>> Signed-off-by: Stefan Weil<sw@weilnetz.de> >>>>>>> --- >>>>>>> >>>>>>> I tested the executables with large address awareness on a 64 bit >>>>>>> Windows >>>>>>> (works) and with Wine on Debian 32 bit Linux (no longer aborts, but >>>>>>> hangs >>>>>>> when 1024 or more MiB are requested). >>>>>>> >>>>>>> Maybe the support for large addresses is broken in my Wine version. >>>>>>> Please report any different test results. >>>>>> >>>>>> >>>>>> I tried with native WindowsXP, the patch did not help. >>>>>> >>>>>> QEMU was compiled with the new parameter: >>>>>> >>>>>> Install prefix c:/Program Files/QEMU >>>>>> BIOS directory c:/Program Files/QEMU >>>>>> binary directory c:/Program Files/QEMU >>>>>> library directory c:/Program Files/QEMU/lib >>>>>> include directory c:/Program Files/QEMU/include >>>>>> config directory c:/Program Files/QEMU >>>>>> Source path /home/aik/testken/qemu >>>>>> C compiler i686-pc-mingw32-gcc >>>>>> Host C compiler gcc >>>>>> CFLAGS -O2 -D_FORTIFY_SOURCE=2 -g >>>>>> QEMU_CFLAGS -m32 -D__USE_MINGW_ANSI_STDIO=1 >>>>>> -DWIN32_LEAN_AND_MEAN >>>>>> -DWINVER=0x501 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 >>>>>> -D_LARGEFILE_SOURCE >>>>>> -Wstrict-prototypes -Wredundant-decls -Wall -Wundef -Wwrite-strings >>>>>> -Wmissing-prototypes -fno-strict-aliasing -fstack-protector-all >>>>>> -Wendif-labels -Wmissing-include-dirs -Wempty-body -Wnested-externs >>>>>> -Wformat-security -Wformat-y2k -Winit-self -Wignored-qualifiers >>>>>> -Wold-style-declaration -Wold-style-definition -Wtype-limits >>>>>> LDFLAGS -Wl,--nxcompat -Wl,--no-seh >>>>>> -Wl,--large-address-aware >>>>>> -Wl,--dynamicbase -Wl,--warn-common -m32 -static -g >>>>>> make make >>>>>> install install >>>>>> python python >>>>>> smbd /usr/sbin/smbd >>>>>> host CPU i386 >>>>>> host big endian no >>>>>> target list ppc64-softmmu >>>>>> [...] >>>>>> >>>>>> >>>>>> >>>>>> qemu/ppc64-softmmu/qemu-system-ppc64 -M pseries -m 1024 -nographic -L >>>>>> qemu/pc-bios/ -net none -append console=hvc0 -initrd ./1.cpio -kernel >>>>>> ./guest.vmlinux.n -net nic,model=virtio -net >>>>>> user,hostfwd=tcp::5555-:22 >>>>>> -drive >>>>>> file=test-virtio-blk.img,if=virtio,index=0,media=disk,cache=unsafe >>>>>> ***qemu_memalign 200 (512) size 800 (2048) >>>>>> Warning: Disabling some instructions which are not emulated by TCG >>>>>> (0x0, 0x6) >>>>>> ***qemu_vmalloc 40000000 (1073741824) >>>>>> Failed to allocate memory: 8 >>>>>> >>>>>> This application has requested the Runtime to terminate it in an >>>>>> unusual way. >>>>>> Please contact the application's support team for more information. >>>>>> >>>>>> >>>>>> The lines with *** are mine printf("%s %x(%u)", __func__, etc). >>>>>> >>>>>> It still works with "-m 1023". >>>>> >>>>> >>>>> Do you have enough RAM? Even with 64 bit Windows 7, I am >>>>> limited to -m 7000 with 64 bit QEMU on a 8 GiB host. >>>>> 32 bit QEMU runs there with up to 2047 MiB. >>>> >>>> >>>> >>>> I put another VirtualAlloc(64MB) call in qemu_vmalloc(1023MB) to check >>>> if >>>> it fails but it did not so it is possible to allocate 1087MB but in 2 >>>> pieces. >>>> >>>> >>>>> On 32 bit Windows XP, I'd expect that you need 2 GiB (or better >>>>> 3 GiB) of physical RAM in your host, and close all other >>>>> programs before you try QEMU. >>>> >>>> >>>> How do other programs matter? It is virtual 2GB per process, no? >>>> Ah, anyway, it has 2.994.340KB of physical RAM, swapping is enabled, the >>>> task manager shows at least "2000000" in "Available", one cygwin shell >>>> is >>>> running. >>>> And QEMU can always alloc 1023MB (in one chunk) and never 1024MB (also >>>> in >>>> one chunk). For me it looks like XP limitation but I could not find any >>>> mention of it in MSDN. >>>> >>>> >>>>> Maybe I can run a test with Windows XP during the weekend. >>>>> >>>>> By the way: the patch must be modified to work for 64 bit QEMU. >>>>> MinGW-w64's ld shows the new parameter in its help, but does >>>>> not support it for 64 bit executables. >>>> >>>> >>>> >>>> Not sure if I understood :) >>>> The compiler I use is "i686-pc-mingw32-gcc -m32", target is >>>> ppc64-softmmu. >>>> >>>> >>> >>> Did a simple test inside QEMU, ran it twice: >>> >>> run#1: first alloc 64MB, then 1023MB - failed. >>> va(0x4000000) -> ptr=1d0d0000, last error = 0 >>> va(0x3FF00000) -> ptr=00000000, last error = 8 >>> >>> run#2: first alloc 1023MB, then 64MB - succeeded. >>> va(0x3FF00000) -> ptr=1d0d0000, last error = 0 >>> va(0x4000000) -> ptr=5d130000, last error = 0 >>> >>> As we can see, there is 340000h gap between 2 chunks. >>> >>> Have no idea what that means :) May be something is wrong with my >>> machine, >>> dunno. >> >> >> Looked further, c:\windows\system32\sysctl32.dll is mapped there so there >> is just no big chunk of RAM. But still there is some RAM, it is just >> fragmented. > > > Yes, that's the problem: the QEMU executable and all the DLLs > occupy RAM in the range from 0 to 2 GiB, so this area is very > fragmented and it's impossible to get a large contiguous memory > area (which VirtualAlloc requires!). I think the real fix is to implement guest RAM using DIMMs slots so that either one contiguous block can be used for all DIMMs (for huge page table benefits on Linux) or several host memory blocks can be used, one for each DIMM (for Windows). > > On 64 bit Windows, addresses from 2 GiB to 4 GiB can be used > with --large-address-aware, so there is no such problem. > With 64 bit executables, even more RAM above 4 GiB is available. > > If you have enough RAM, you can configure a 32 bit Windows to > provide 3 GiB instead of 2 GiB to user applications. This > non-standard configuration also allows larger sizes for VirtualAlloc. > > See > http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=vs.85%29.aspx > > for more information. > > See http://qemu.weilnetz.de/w32/virtualalloc-test/ for a simple test > program. > > Regards, > > Stefan Weil > > ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2012-07-23 18:33 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-07-18 16:37 [Qemu-devel] [PATCH] configure: Support system emulation with large memory on w32 hosts Stefan Weil 2012-07-20 3:53 ` Alexey Kardashevskiy 2012-07-20 5:23 ` Stefan Weil 2012-07-20 5:37 ` Alexey Kardashevskiy 2012-07-20 6:05 ` Alexey Kardashevskiy 2012-07-20 6:38 ` Alexey Kardashevskiy 2012-07-20 17:19 ` Stefan Weil 2012-07-23 18:32 ` Blue Swirl
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).