[Qemu-devel] Downloaded files are corrupt

[Qemu-devel] Downloaded files are corrupt

[Aaron McDonald]

I'm having a problem downloading files. I'm using qemu 0.6.1 to run Feather 
Linux from within Alt Linux on a 800 MHtz machine with 256MB of RAM.

qemu -hda harddisk -cdrom feather-0.6.1.iso -boot d -m 256 -enable-audio 
-user-net -localtime

Everything starts up ok but if I download something from the internet, I 
don't seem to get the last byte and so the file is corrupt. I have the same 
issue with a Damn Small Linux iso.

I can download the file just fine from my host OS (Alt Linux) so I'm ruling 
out a network issue. Also, I'm having the same problem when using Windows 
2000 as the host OS on a 2.6 GHtz machine with 1GB of RAM so I'm thinking 
that it's not a hardware issue either.

I posted this issue to the bulletin board a couple of days ago. Another user 
responded and said that he'd had the same problem w/ qemu >0.6.0 running 
debian/knoppix under Win XP (host) with "-user-net".

Thanks,
Aaron

Re: [Qemu-devel] Downloaded files are corrupt

[Magnus Damm]

Have you tried downloading files from the integrated TFTP server
included with the -user-net implementation? Does that work out ok?

/ magnus

On Thu, 2004-11-18 at 22:34, Aaron McDonald wrote:
> I'm having a problem downloading files. I'm using qemu 0.6.1 to run Feather 
> Linux from within Alt Linux on a 800 MHtz machine with 256MB of RAM.
> 
> qemu -hda harddisk -cdrom feather-0.6.1.iso -boot d -m 256 -enable-audio 
> -user-net -localtime
> 
> Everything starts up ok but if I download something from the internet, I 
> don't seem to get the last byte and so the file is corrupt. I have the same 
> issue with a Damn Small Linux iso.
> 
> I can download the file just fine from my host OS (Alt Linux) so I'm ruling 
> out a network issue. Also, I'm having the same problem when using Windows 
> 2000 as the host OS on a 2.6 GHtz machine with 1GB of RAM so I'm thinking 
> that it's not a hardware issue either.
> 
> I posted this issue to the bulletin board a couple of days ago. Another user 
> responded and said that he'd had the same problem w/ qemu >0.6.0 running 
> debian/knoppix under Win XP (host) with "-user-net".
> 
> Thanks,
> Aaron
> 
> 
> 
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

[Qemu-devel] Re: Downloaded files are corrupt

[Ronald]

Le Thu, 18 Nov 2004 22:45:38 +0100, Magnus Damm a écrit :

> Have you tried downloading files from the integrated TFTP server included
> with the -user-net implementation? Does that work out ok?
> 
> / magnus

Quick review for (linux host and guest)

tftp is ok, -redir is ok too (ssh && scp host to
guest), streaming works (mplayer -ao alsa -vo null
http://64.236.34.97:80/stream/1001), I can browse the web, but I
get the same corruption downloading files from http (tried with links and
curl, curl print 'curl: (18) transfer closed with 1 bytes remaining to
read' for http://fabrice.bellard.free.fr/qemu/qemu-0.6.1.tar.gz, tar tzf list until hw/cuda.c gzip complain about data--crc and
data-lenght). 

> 
> On Thu, 2004-11-18 at 22:34, Aaron McDonald wrote:
>> I'm having a problem downloading files. I'm using qemu 0.6.1 to run
>> Feather Linux from within Alt Linux on a 800 MHtz machine with 256MB of
>> RAM.
>> 
>> qemu -hda harddisk -cdrom feather-0.6.1.iso -boot d -m 256 -enable-audio
>> -user-net -localtime
>> 
>> Everything starts up ok but if I download something from the internet, I
>> don't seem to get the last byte and so the file is corrupt. I have the
>> same issue with a Damn Small Linux iso.
>> 
>> I can download the file just fine from my host OS (Alt Linux) so I'm
>> ruling out a network issue. Also, I'm having the same problem when using
>> Windows 2000 as the host OS on a 2.6 GHtz machine with 1GB of RAM so I'm
>> thinking that it's not a hardware issue either.
>> 
>> I posted this issue to the bulletin board a couple of days ago. Another
>> user responded and said that he'd had the same problem w/ qemu >0.6.0
>> running debian/knoppix under Win XP (host) with "-user-net".
>> 
>> Thanks,
>> Aaron
>> 
>> 
>> 
>> 
>> _______________________________________________ Qemu-devel mailing list
>> Qemu-devel@nongnu.org
>> http://lists.nongnu.org/mailman/listinfo/qemu-devel

[Qemu-devel] Re: Downloaded files are corrupt

[Ronald]

Le Thu, 18 Nov 2004 23:28:17 +0100, Ronald a écrit :

> Le Thu, 18 Nov 2004 22:45:38 +0100, Magnus Damm a écrit :
> 
>> Have you tried downloading files from the integrated TFTP server
>> included with the -user-net implementation? Does that work out ok?
>> 
>> / magnus
> 
> Quick review for (linux host and guest)
> 
> tftp is ok, -redir is ok too (ssh && scp host to guest), streaming works
> (mplayer -ao alsa -vo null http://64.236.34.97:80/stream/1001), I can
> browse the web, but I get the same corruption downloading files from http
> (tried with links and curl, curl print 'curl: (18) transfer closed with 1
> bytes remaining to read' for
> http://fabrice.bellard.free.fr/qemu/qemu-0.6.1.tar.gz, tar tzf list until
> hw/cuda.c gzip complain about data--crc and data-lenght).
> 
> 

After reboot some (partial) results for windows host too(same guest image
as previously)

C:\Program Files\Qemu>qemu -L .\bios -m 128 -localtime -user-net
-enable-audio -redir tcp:2222::22 -tftp /temp -hda c:\qemu.dsk -boot c

tftp is ok, ssh too but there is some delay between password enter and
getting the shell (~1 minute).
I don't have internet access under windows, so I can't try file
downloading, sorry.
Guest is runnig 2.6 if that matter.

>> On Thu, 2004-11-18 at 22:34, Aaron McDonald wrote:
>>> I'm having a problem downloading files. I'm using qemu 0.6.1 to run
>>> Feather Linux from within Alt Linux on a 800 MHtz machine with 256MB of
>>> RAM.
>>> 
>>> qemu -hda harddisk -cdrom feather-0.6.1.iso -boot d -m 256
>>> -enable-audio -user-net -localtime
>>> 
>>> Everything starts up ok but if I download something from the internet,
>>> I don't seem to get the last byte and so the file is corrupt. I have
>>> the same issue with a Damn Small Linux iso.
>>> 
>>> I can download the file just fine from my host OS (Alt Linux) so I'm
>>> ruling out a network issue. Also, I'm having the same problem when
>>> using Windows 2000 as the host OS on a 2.6 GHtz machine with 1GB of RAM
>>> so I'm thinking that it's not a hardware issue either.
>>> 
>>> I posted this issue to the bulletin board a couple of days ago. Another
>>> user responded and said that he'd had the same problem w/ qemu >0.6.0
>>> running debian/knoppix under Win XP (host) with "-user-net".
>>> 
>>> Thanks,
>>> Aaron
>>> 
>>> 
>>> 
>>> 
>>> _______________________________________________ Qemu-devel mailing list
>>> Qemu-devel@nongnu.org
>>> http://lists.nongnu.org/mailman/listinfo/qemu-devel

Re: [Qemu-devel] Downloaded files are corrupt

[Julian Seward]

Yes, I also noticed corruption on downloaded files using Red Hat 7.3
(guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.

Also got network corruption on X connections from X clients on
the guest connected to an X server elsewhere: X would complain about
protocol violations (I can't remember the details).  Sometimes an
X client would only run for a couple of minutes before dying as
a consequence of this.

J

On Thursday 18 November 2004 21:34, Aaron McDonald wrote:
> I'm having a problem downloading files. I'm using qemu 0.6.1 to run Feather
> Linux from within Alt Linux on a 800 MHtz machine with 256MB of RAM.
>
> qemu -hda harddisk -cdrom feather-0.6.1.iso -boot d -m 256 -enable-audio
> -user-net -localtime
>
> Everything starts up ok but if I download something from the internet, I
> don't seem to get the last byte and so the file is corrupt. I have the same
> issue with a Damn Small Linux iso.
>
> I can download the file just fine from my host OS (Alt Linux) so I'm ruling
> out a network issue. Also, I'm having the same problem when using Windows
> 2000 as the host OS on a 2.6 GHtz machine with 1GB of RAM so I'm thinking
> that it's not a hardware issue either.
>
> I posted this issue to the bulletin board a couple of days ago. Another
> user responded and said that he'd had the same problem w/ qemu >0.6.0
> running debian/knoppix under Win XP (host) with "-user-net".
>
> Thanks,
> Aaron
>
>
>
>
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel

[Qemu-devel] valgrind functionality in qemu?

[Marc E. Fiuczynski]

Just curious whether anyone has thought about putting things like the memory profiling, thread debugging, etc. features of valgrind (http://valgrind.kde.org/) into qemu?!  It sure would be AWESOME to use qemu to hunt down memory bugs in kernels and/or perform cache profiling to optimize layout of data structures. 

Marc

Re: [Qemu-devel] valgrind functionality in qemu?

[Johannes Schindelin]

Hi,

On Sat, 20 Nov 2004, Marc E. Fiuczynski wrote:

> Just curious whether anyone has thought about putting things like the
> memory profiling, thread debugging, etc. features of valgrind
> (http://valgrind.kde.org/) into qemu?!

Yes, I for one have.

The upside of it would be the flexibility you have: you can let QEmu do
virtually everything, and you can even make very complex "watch points"
or "invariant conditions".

The downside is execution speed. Valgrind does not have to deal with a
complete translation, but only the translation of memory accesses. Thus,
valgrind should be much faster all the time. Also note that the memory
usage would be much higher with QEmu, which gets worse especially with
growing code size.

It would also be difficult to find the reference to the source code,
because there is no support for working with the debug information in the
code in QEmu that I know of. If you want to debug the Linux kernel, you
need to hack the "-kernel" loading code for that.

I don't want to discourage you, but be warned: while it sure would be
useful, and also feasible, you will have to put in a lot of work and
thought into that project.

Ciao,
Dscho

Re: [Qemu-devel] valgrind functionality in qemu?

[Sylvain Petreolle]

[-- Attachment #1: Type: text/plain, Size: 667 bytes --]

Talking about qemu and valgrind,
qemu immediately exits if you run it under valgrind.
I tried it with latest qemu CVS and valgrind 2.2.0.

Actual cmdline was :
valgrind --tool=memcheck --leak-check=yes qemu -boot d -cdrom /home/syl/qemu_images/tccboot.iso
(I attached the result)

> On Sat, 20 Nov 2004, Marc E. Fiuczynski wrote:
> 
> > Just curious whether anyone has thought about putting things like the
> > memory profiling, thread debugging, etc. features of valgrind
> > (http://valgrind.kde.org/) into qemu?!
> 

=====
Sylvain Petreolle (spetreolle_at_users_dot_sourceforge_dot_net)
humans are like computers,
yesterday the BIOS was all
- today its just a word

[-- Attachment #2: qemuresult --]
[-- Type: application/octet-stream, Size: 2218 bytes --]

==3247== Memcheck, a memory error detector for x86-linux.
==3247== Copyright (C) 2002-2004, and GNU GPL'd, by Julian Seward et al.
==3247== Using valgrind-2.2.0, a program supervision framework for x86-linux.
==3247== Copyright (C) 2000-2004, and GNU GPL'd, by Julian Seward et al.
==3247== For more details, rerun with: -v
==3247== 
==3247== Warning: set address range perms: large range 138739712, a 0, v 1
==3247== Syscall param write(buf) contains uninitialised or unaddressable byte(s)
==3247==    at 0x1BAB6723: __write_nocancel (in /lib/tls/libc-2.3.3.so)
==3247==    by 0x1BC2E12F: (within /usr/X11R6/lib/libX11.so.6.2)
==3247==    by 0x1BC2EECE: _X11TransWrite (in /usr/X11R6/lib/libX11.so.6.2)
==3247==    by 0x1BC0E721: (within /usr/X11R6/lib/libX11.so.6.2)
==3247==  Address 0x1BCCC30E is 14 bytes inside a block of size 16384 alloc'd
==3247==    at 0x1B9053FD: calloc (vg_replace_malloc.c:176)
==3247==    by 0x1BC0055D: XOpenDisplay (in /usr/X11R6/lib/libX11.so.6.2)
==3247==    by 0x1B994913: (within /usr/lib/libSDL-1.2.so.0.7.0)
==3247==    by 0x1B989E03: SDL_VideoInit (in /usr/lib/libSDL-1.2.so.0.7.0)
==3247== 
==3247== ERROR SUMMARY: 3 errors from 1 contexts (suppressed: 39 from 1)
==3247== malloc/free: in use at exit: 139106106 bytes in 354 blocks.
==3247== malloc/free: 1407 allocs, 1053 frees, 139234616 bytes allocated.
==3247== For counts of detected errors, rerun with: -v
==3247== searching for pointers to 354 not-freed blocks.
==3247== checked 19790712 bytes.
==3247== 
==3247== 
==3247== 8 bytes in 1 blocks are definitely lost in loss record 1 of 29
==3247==    at 0x1B9054EA: realloc (vg_replace_malloc.c:197)
==3247==    by 0x1BC50FF2: (within /usr/X11R6/lib/libX11.so.6.2)
==3247==    by 0x1BC5260E: (within /usr/X11R6/lib/libX11.so.6.2)
==3247==    by 0x1BC5341B: (within /usr/X11R6/lib/libX11.so.6.2)
==3247== 
==3247== LEAK SUMMARY:
==3247==    definitely lost: 8 bytes in 1 blocks.
==3247==    possibly lost:   0 bytes in 0 blocks.
==3247==    still reachable: 139105898 bytes in 352 blocks.
==3247==         suppressed: 200 bytes in 1 blocks.
==3247== Reachable blocks (those to which a pointer was found) are not shown.
==3247== To see them, rerun with: --show-reachable=yes

[Qemu-devel] Re: valgrind functionality in qemu?

[Ben Pfaff]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> It would also be difficult to find the reference to the source code,
> because there is no support for working with the debug information in the
> code in QEmu that I know of. If you want to debug the Linux kernel, you
> need to hack the "-kernel" loading code for that.

It's not *that* hard, it just takes writing some code.  We did
this in a simulation paper: "Understanding Data Lifetime via
Whole System Simulation" available at
<URL:http://www.stanford.edu/~blp/papers>, although our
implementation used Bochs instead of qemu because qemu wasn't
quite ready at the time.
-- 
Ben Pfaff 
email: blp@cs.stanford.edu
web: http://benpfaff.org

Re: [Qemu-devel] Re: valgrind functionality in qemu?

[Johannes Schindelin]

Hi,

On Mon, 22 Nov 2004, Ben Pfaff wrote:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:
>
> > It would also be difficult to find the reference to the source code,
> > because there is no support for working with the debug information in the
> > code in QEmu that I know of. If you want to debug the Linux kernel, you
> > need to hack the "-kernel" loading code for that.
>
> It's not *that* hard, it just takes writing some code.  We did
> this in a simulation paper: "Understanding Data Lifetime via
> Whole System Simulation" available at
> <URL:http://www.stanford.edu/~blp/papers>, although our
> implementation used Bochs instead of qemu because qemu wasn't
> quite ready at the time.

Okay. Unfortunately, you didn't publish the source code for TaintBochs.
So how did you tackle following problems:

- when deciding what to taint, you want to be as specific as possible. How
did you tell bochs what was tainted, and what not?

- when you tested inside bochs, you didn't have control over loading of
programs. How did bochs know where the code came from?

- even more importantly, when you analyzed where tainting data was
propagated or freed, how did you find out which *source code* was
responsible for that?

I would do tha "just write some code" part, but I still look for
elegant solutions to those problems.

Ciao,
Dscho

[Qemu-devel] Re: valgrind functionality in qemu?

[Ben Pfaff]

Johannes Schindelin <Johannes.Schindelin@gmx.de> writes:

> Unfortunately, you didn't publish the source code for TaintBochs.

It's not very pretty.  We're working on a nicer version that we
might release later.

> So how did you tackle following problems:
>
> - when deciding what to taint, you want to be as specific as possible. How
> did you tell bochs what was tainted, and what not?

This isn't really relevant for the purpose of doing valgrind-like
functionality, for what it's worth.  However, we used various
methods.  For example, for tainting passwords input from the
keyboard we added a toggle switch to the Bochs interface that
determined whether keypressed were considered tainted, and for
tainting network input we did a simple string search in network
packets (which is not very smart but worked okay for the simple
case we wanted to try).

> - when you tested inside bochs, you didn't have control over loading of
> programs. How did bochs know where the code came from?

We used Mission Critical's "crash" utility as a kernel debugger.
We added a module to it that allowed us to generate a core dump
of any process in the guest.  When you combine a core dump with
the program binary and the program source, gdb and addr2line can
tell you everything you want to know.

> - even more importantly, when you analyzed where tainting data was
> propagated or freed, how did you find out which *source code* was
> responsible for that?

gdb and addr2line as above.

I think all this is in the paper actually.  We included a fair
number of details of our implementation there.

> I would do tha "just write some code" part, but I still look for
> elegant solutions to those problems.

-- 
"The road to hell is paved with convenient shortcuts."
--Peter da Silva

Re: [Qemu-devel] Downloaded files are corrupt

[Martin Jansa]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Julian Seward wrote:
| Yes, I also noticed corruption on downloaded files using Red Hat 7.3
| (guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.

me too ;-(
newest ~x86 gentoo (host), GNU Hurd K7 (guest)

after apt-get upgrade, I get list of errors, every package is 1 Byte smaller

qemu-0.6.1 -user-net
- --
Martin Jansa          UIN: 136542059        JID: JaMa@jabber.mk.cvut.cz
~        Kazdy ma fotografickou pamet. Jenom nekdo nema film.
~                -- Steven Wright one Liners
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBoMZsN1Ujt2V2gBwRAn6eAKCgY20SfmmgHZ1acTptA4DNx6SRfACgtlno
qV0gZBPZEOMh7HmMi161+ps=
=r2RB
-----END PGP SIGNATURE-----

Re: [Qemu-devel] Downloaded files are corrupt

[Julian Seward]

On Sunday 21 November 2004 16:46, Martin Jansa wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Julian Seward wrote:
> | Yes, I also noticed corruption on downloaded files using Red Hat 7.3
> | (guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.
>
> me too ;-(
> newest ~x86 gentoo (host), GNU Hurd K7 (guest)
>
> after apt-get upgrade, I get list of errors, every package is 1 Byte
> smaller

Perhaps there is an off-by-1 error somewhere in the networking
simulation.

It's a shame, because this is a show-stopping bug for me.  There
is real useful stuff I want to use Qemu for, but reliable
networking is a prerequisite.

J

[Qemu-devel] Re: Downloaded files are corrupt

[Ronald]

Le Sun, 21 Nov 2004 18:30:45 +0000, Julian Seward a écrit :

> On Sunday 21 November 2004 16:46, Martin Jansa wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>>
>> Julian Seward wrote:
>> | Yes, I also noticed corruption on downloaded files using Red Hat 7.3
>> | (guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.
>>
>> me too ;-(
>> newest ~x86 gentoo (host), GNU Hurd K7 (guest)
>>
>> after apt-get upgrade, I get list of errors, every package is 1 Byte
>> smaller
> 
> Perhaps there is an off-by-1 error somewhere in the networking simulation.
> 
> It's a shame, because this is a show-stopping bug for me.  There is real
> useful stuff I want to use Qemu for, but reliable networking is a
> prerequisite.
> 
> J

You are still able to setup tun network instead of user-net under linux
host.  

Re: [Qemu-devel] Downloaded files are corrupt

[Fabrice Bellard]

Martin Jansa wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Julian Seward wrote:
> | Yes, I also noticed corruption on downloaded files using Red Hat 7.3
> | (guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.
> 
> me too ;-(
> newest ~x86 gentoo (host), GNU Hurd K7 (guest)
> 
> after apt-get upgrade, I get list of errors, every package is 1 Byte 
> smaller

I just commited a fix. Tell me if it is OK now.

Fabrice.

Re: [Qemu-devel] Downloaded files are corrupt

[Martin Jansa]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Fabrice Bellard wrote:
| Martin Jansa wrote:
|
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> Julian Seward wrote:
|> | Yes, I also noticed corruption on downloaded files using Red Hat 7.3
|> | (guest), SuSE 9.1 (host), running qemu-0.6.0, with user-net.
|>
|> me too ;-(
|> newest ~x86 gentoo (host), GNU Hurd K7 (guest)
|>
|> after apt-get upgrade, I get list of errors, every package is 1 Byte
|> smaller
|
|
| I just commited a fix. Tell me if it is OK now.
|
| Fabrice.

That's it, now I have downloaded 35MB in 66 files and all seems OK

Thanks a lot Fabrice

- --
Martin Jansa          UIN: 136542059        JID: JaMa@jabber.mk.cvut.cz
~        Kazdy ma fotografickou pamet. Jenom nekdo nema film.
~                -- Steven Wright one Liners
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBoPtKN1Ujt2V2gBwRAjAvAJ9hc/4bnw5wzTxzdSHjxcUVwRkTfgCgmHLH
dXUdd5YdZjsMtmChtRTmCgE=
=sgz7
-----END PGP SIGNATURE-----

Re: [Qemu-devel] valgrind functionality in qemu?

[Blue Swirl]

Hi,

For your information: Sparc system emulator can use the ELF debugging 
information in Proll and/or Linux/*BSD kernel images. It's very to see the 
prom and kernel function names in qemu.log.

Even cooler is single stepping through interrupt handler code with full 
source using GDB. :-)

_________________________________________________________________
FREE pop-up blocking with the new MSN Toolbar - get it now! 
http://toolbar.msn.click-url.com/go/onm00200415ave/direct/01/