[Qemu-devel] harddrives and QEMU

[Qemu-devel] harddrives and QEMU

[Brett (Mare) Henley]

Hello all,
   Been a long time user and appreciator of QEMU and would like to thank 
everyone who put so much hard work into it.
   I found a difficulty that I'm not sure where to start so this could be
a lengthy email.
   The short of it goes like this:
    I hook up a 12G IDE drive through a usb-ide adaptor so I can boot 
off it using qemu with Windows as the Host OS. On the drive is FreeBSD. 
I can boot off this drive with my machine just fine and I realized it 
would be a boon if I could use my usb adaptor to build or update new 
boot drives etc.
    WaxDragon was savvy enough to tell me to access a drive like this in 
windows I need to use -hda \\.\PhysicalDriveX (where X is a drive number 
  that the storage manager thinks should be it usually 1)
    To my joy FreeBSD actually started to boot. I was amazed and then it 
got to sizing up my drive (Mind you the Bios reported a drive size of -1 
Mbytes) It hung for a long time. Started chewing up memory and every 10 
seconds or so the drive would access and rattle. Finally a report of the 
drive size came that said something like 248043832 Megabytes. And my 
windows machine got slower and slower it took 5 minutes to get the qemu 
console up and another 5 before my quit command took effect.
   That's the short synopsis.
   The Longer one goes into greater detail. WaxDragon's first suggestion 
was to use -hdachs to define the drive for the bios. I had to read up 
well to do this but entered 65383,16,63 as my definition and still ended 
up with exactly the same behavior.
   Still determined to make something of this. I tried a 4G HD and by 
using -hdachs 8912,15,63 (from reading it on the drive) I got a 
respectable though not entirely accurate bios size and the machine in 
qemu booted puppy linux and let me install it on the hard drive. 
Ultimately using this method I had a pack I could boot off of using a 
line like:
  qemu -L . -m 192 -hda \\.\PhysicalDrive1 -localtime -enable-audio

   I didn't need the -hdachs and it booted spiffy as can be. Trying
to get it to boot off a real machine was a good bit more problematic as 
the drive would be lettered /dev/sda1 etc so grub had a fit and needed 
to be told otherwise. Still when all the kinks were worked around it 
actually booted and worked. But not nearly as easy as it could have.
   Thinking perhaps it's a question of an 8G limit I tried a 6G drive 
via usb and had the same trouble sizing and using the drive I did with 
the 12G drive.
   I also understand there's a problem with Linux QEMU accessing large 
drives via device node.
   Is there any work being done to fix or create options to make using 
physical drives as easy as using image files?
   Is there any work being done to let physical drives be considered
another type of drive. Such as a usb drive instead of a ide drive. This 
would help making os installation much simpler on hard drives from the 
host OS.

Thanks.

Re: [Qemu-devel] harddrives and QEMU

[Jim C. Brown]

On Wed, Oct 05, 2005 at 01:22:33PM -0500, Brett (Mare) Henley wrote:
>   The Longer one goes into greater detail. WaxDragon's first suggestion 
> was to use -hdachs to define the drive for the bios. I had to read up 
> well to do this but entered 65383,16,63 as my definition and still ended 
> up with exactly the same behavior.

-hdachs doesn't make a difference, unless the OS reads the size of the
hard disk from the bios.

If it gets its size from the MBR or partition table, it won't help at all.

There is probably a way to force freebsd into using a certain size or c/h/s
value for the hard disk. I recommend trying this to see if its just the bad
disk size thats causing problems.

>   I also understand there's a problem with Linux QEMU accessing large 
> drives via device node.

No. That works fine.

>   Is there any work being done to fix or create options to make using 
> physical drives as easy as using image files?

They are as easy to use. It just happens that the syntax for Windows raw
access is uglier than /dev/hda.

>   Is there any work being done to let physical drives be considered
> another type of drive. Such as a usb drive instead of a ide drive. This 
> would help making os installation much simpler on hard drives from the 
> host OS.

'Fraid not. (At least nothing that I know of, perhaps someone else will step up
and announce usb-on-qemu support soon.)

Of course, even if qemu had support for usb that wouldn't necessarily solve
your problem.

> 
> Thanks.
> 

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] harddrives and QEMU

[Brett Henley]

Jim C. Brown wrote:

>On Wed, Oct 05, 2005 at 01:22:33PM -0500, Brett (Mare) Henley wrote:
>  
>
>>  The Longer one goes into greater detail. WaxDragon's first suggestion 
>>was to use -hdachs to define the drive for the bios. I had to read up 
>>well to do this but entered 65383,16,63 as my definition and still ended 
>>up with exactly the same behavior.
>>    
>>
>
>-hdachs doesn't make a difference, unless the OS reads the size of the
>hard disk from the bios.
>
>If it gets its size from the MBR or partition table, it won't help at all.
>
>There is probably a way to force freebsd into using a certain size or c/h/s
>value for the hard disk. I recommend trying this to see if its just the bad
>disk size thats causing problems.
>
>  
>
Alright, but here's the rub. If a drive can be booted by a machine. Why 
can't it boot from Qemu if it's accessing the raw disk via the windows 
interface? This needs no messing with bios or disksize to boot of a 
regular machine.

Re: [Qemu-devel] harddrives and QEMU

[Karl Magdsick]

> >
> Alright, but here's the rub. If a drive can be booted by a machine. Why
> can't it boot from Qemu if it's accessing the raw disk via the windows
> interface? This needs no messing with bios or disksize to boot of a
> regular machine.
>

I hope someone else will chime in, but my guess is that the problem
lies in that an MS Windows "drive" is really a partition, not the entire
drive. Under Windows you're specifying the equivalent of
the Linux /dev/hda1 , /dev/hda2, /dev/hdb1, /dev/hdb5 and not
actually /dev/hda or /dev/hdb.

In other words, to QEMU it looks like the drive got chopped off at some
point.  All of the data before a certain point and after another point on
the HD is simply missing (and the indexing is incorrect if a non-zero number
of bytes have been chopped from the beginning of the disk).

I'm not sure if MS Windows includes the MBR in the first drive on the disk.

In order to pass the "D drive" to qemu, and actually give QEMU access
to the entire raw HD, the "D drive" partition would have to fill the entire
HD, and MS Windows would have to make the MBR available as part
of the first (only, in this case) partition on the HD.

Presumably, you'd like QEMU to make up a fake partition table/MBR
to present to the guest OS so that the guest OS sees a self-consistent
disk.



-Karl

Re: [Qemu-devel] harddrives and QEMU

[Jim C. Brown]

On Wed, Oct 05, 2005 at 06:15:09PM -0500, Brett Henley wrote:
> Alright, but here's the rub. If a drive can be booted by a machine. Why 
> can't it boot from Qemu if it's accessing the raw disk via the windows 
> interface? This needs no messing with bios or disksize to boot of a 
> regular machine.
> 

Good question. It shouldn't. Clearly something is going wonky.

Incidently, can you boot the puppy linux drive on the bare hardware? Or does
that only work in qemu?

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.

Re: [Qemu-devel] harddrives and QEMU

[Karl Magdsick]

I wrote:
> In order to pass the "D drive" to qemu, and actually give QEMU access
> to the entire raw HD, the "D drive" partition would have to fill the entire
> HD, and MS Windows would have to make the MBR available as part
> of the first (only, in this case) partition on the HD.

Here I'm of course presuming that partitions aren't overlapping.

-Karl

Re: [Qemu-devel] harddrives and QEMU

[Thomas Steffen]

[-- Attachment #1: Type: text/plain, Size: 936 bytes --]

On 10/6/05, Karl Magdsick <kmagnum@gmail.com> wrote:
>
> I wrote:
> > In order to pass the "D drive" to qemu, and actually give QEMU access
> > to the entire raw HD, the "D drive" partition would have to fill the
> entire
> > HD, and MS Windows would have to make the MBR available as part
> > of the first (only, in this case) partition on the HD.
>
> Here I'm of course presuming that partitions aren't overlapping.


That is an interesting idea. On every other system I know, there is one
partition that includes the whole disc. So if the partitioning is the
problem here, it should be possible to create one partition that spans the
whole disc (overlapping all other partitions), even with the classical IBM
PC partition scheme. fdisk in expert mode should be able to do that.

Of course the next question is how happy windows is accessing this
partition. Should be an interesting experiment at least.

Thomas

[-- Attachment #2: Type: text/html, Size: 1259 bytes --]

Re: [Qemu-devel] harddrives and QEMU

[Brett (Mare) Henley]

To Clerify since there are a lot of variables here I'll define the test 
bed I'm working with.

I have 3 external ide drives.

12G, 6G and 4G. not a one has a dos partition on them.
they were partitioned with fdisk and then had the os's installed on them.

the 12G was a dd copy of the 6 gig with some growing of the filesystem
to handle the greater room.

Both the 6G and the 12G are FreeBSD and both booth flawlessly on my
3G HP laptop. Via a usb-ide adaptor. (Bytecc's BT-200)

the 4G I built with puppy linux to test and installed it using
the USB adaptor. It booted just fine but would not in QEMU.

I then stripped the 4G drive and installed puppy linux via qemu
using -hda \\.\PhysicalDrive1 -hdachs 8912,15,63 (I had to use this 
otherwise the drive did not size up to be installable)

Puppy Linux installed as was directly bootable using -hda 
\\.\PhysicalDrive1 without the -hdachs definition.

Trying to boot this drive striaght from the usb adaptor on my HP
laptop failed because the drive was expecting hda1 for the filesystem
and in Linux that ends up being sda1. So I had to tweak grub into
using sda1 to be the root and boot.

That worked. But this is why I suggested being able to emulate other 
drive interfaces.

When I'm testing. I'm running winxp sp2. QEMU loads the booter and
the kernel and tries to come up but when it gets to sizing the harddrive
it chews up memory and slows the platform down to a crawl. eventually
giving a drive size that would make Google proud. like 200Petabytes..
It clunks on and trying to shut down qemu takes a long time to get it
to switch to the console and accept the quit command.

Let me know what you think. :)

The trouble making hard drive maniac.

Me


Thomas Steffen wrote:
> On 10/6/05, *Karl Magdsick* <kmagnum@gmail.com 
> <mailto:kmagnum@gmail.com>> wrote:
> 
>     I wrote:
>      > In order to pass the "D drive" to qemu, and actually give QEMU access
>      > to the entire raw HD, the "D drive" partition would have to fill
>     the entire
>      > HD, and MS Windows would have to make the MBR available as part
>      > of the first (only, in this case) partition on the HD.
> 
>     Here I'm of course presuming that partitions aren't overlapping.
> 
> 
> That is an interesting idea. On every other system I know, there is one 
> partition that includes the whole disc. So if the partitioning is the 
> problem here, it should be possible to create one partition that spans 
> the whole disc (overlapping all other partitions), even with the 
> classical IBM PC partition scheme. fdisk in expert mode should be able 
> to do that.
> 
> Of course the next question is how happy windows is accessing this 
> partition. Should be an interesting experiment at least.
> 
> Thomas
> 
> !DSPAM:4345463a357575300763055!
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Qemu-devel mailing list
> Qemu-devel@nongnu.org
> http://lists.nongnu.org/mailman/listinfo/qemu-devel
> 
> 
> !DSPAM:4345463a357575300763055!

Re: [Qemu-devel] harddrives and QEMU

[Jim C. Brown]

On Thu, Oct 06, 2005 at 08:01:31AM -0400, Karl Magdsick wrote:
> 
> I hope someone else will chime in, but my guess is that the problem
> lies in that an MS Windows "drive" is really a partition, not the entire
> drive. Under Windows you're specifying the equivalent of
> the Linux /dev/hda1 , /dev/hda2, /dev/hdb1, /dev/hdb5 and not
> actually /dev/hda or /dev/hdb.

Ah. If that's true, that would perfectly explain the errors.

> In order to pass the "D drive" to qemu, and actually give QEMU access
> to the entire raw HD, the "D drive" partition would have to fill the entire
> HD, and MS Windows would have to make the MBR available as part
> of the first (only, in this case) partition on the HD.
> 
> I'm not sure if MS Windows includes the MBR in the first drive on the disk.
> 

If it doesn't then there's no way to boot from the first drive. Although
I was under the impression that Windows did in fact provide support for that,
the scheme may not work for USB drives.

> Presumably, you'd like QEMU to make up a fake partition table/MBR
> to present to the guest OS so that the guest OS sees a self-consistent
> disk.
> 
> -Karl
> 

That will become a must if Windows provides no way to access the full hard
disk drive.

-- 
Infinite complexity begets infinite beauty.
Infinite precision begets infinite perfection.