63 sectors

63 sectors

[Avi Kivity]

Qemu sets the sectors-per-track setting of virtual disks to 63.  This 
seems to be in accordance with the specs; drivers/ide/ide-disk.c says:

>         /*
>          * The ATA spec tells large drives to return
>          * C/H/S = 16383/16/63 independent of their size.
>          * Some drives can be jumpered to use 15 heads instead of 16.
>          * Some drives can be jumpered to use 4092 cyls instead of 16383.
>          */
>         if ((id->cyls == 16383
>              || (id->cyls == 4092 && id->cur_cyls == 16383)) &&
>             id->sectors == 63 &&
>             (id->heads == 15 || id->heads == 16) &&
>             (id->lba_capacity >= 16383*63*id->heads))
>                 return 1;

That setting has some unfortunate side effects.  Partitioning tools will 
locate the first partition at the second cylinder, which is at the 63rd 
sector.  This means that if the guest uses a 4K block filesystem on the 
first partition (an incredibly common occurance), then every single 
access will not be 4K aligned with respect to the virtual block device.  
This will cause fragmentation and read/modify/write cycles with:

- qcow2 (which uses aligned 4K blocks)
- any host filesystem which uses 4K blocks
- any host disk which uses 4K blocks (not yet common)

I can think of a few workarounds, all bad:
- add a partitioning tool (or option to qemu-img) to format the disk, 
placing the first partition on the fourth cylinder, aligning it.  tell 
the users not to wipe the disks out but instead install to one of the 
existing parititions
- add a tool to optimize an existing disk by extending it and moving the 
partitions around so they are aligned.  may break boot loaders.
- make qcow4 use 512 byte sectors.  will increase overhead and doesn't 
solve problems on the host filesystem and disk.
- have qcow51 detect misaligned accesses and adjust itself somehow.  
doesn't help raw and other formats.  likely very difficult.

Does anybody know if scsi will have the same problems?  Can anyone 
suggest other workarounds?

-- 
error compiling committee.c: too many arguments to function

Re: 63 sectors

[David Mair]

Avi Kivity wrote:
> Qemu sets the sectors-per-track setting of virtual disks to 63.  This 
> seems to be in accordance with the specs; drivers/ide/ide-disk.c says:
> 
>>         /*
>>          * The ATA spec tells large drives to return
>>          * C/H/S = 16383/16/63 independent of their size.
>>          * Some drives can be jumpered to use 15 heads instead of 16.
>>          * Some drives can be jumpered to use 4092 cyls instead of 16383.
>>          */
>>         if ((id->cyls == 16383
>>              || (id->cyls == 4092 && id->cur_cyls == 16383)) &&
>>             id->sectors == 63 &&
>>             (id->heads == 15 || id->heads == 16) &&
>>             (id->lba_capacity >= 16383*63*id->heads))
>>                 return 1;
> 
> That setting has some unfortunate side effects.  Partitioning tools will 
> locate the first partition at the second cylinder, which is at the 63rd 
> sector.  This means that if the guest uses a 4K block filesystem on the 
> first partition (an incredibly common occurance), then every single 
> access will not be 4K aligned with respect to the virtual block device.  
> This will cause fragmentation and read/modify/write cycles with:

Oh, the tales I could tell of big disk hardware vendors that cache their storage 
platforms on 4k aligned blocks and have horrible performance for reads across a 
block boundary and only provide drivers for Windows to format LUNs with 4k 
alignment.

AFAIK, this is actually a relic limitation of the IBM PC-BIOS hard disk geometry 
functions and the MS-DOS partitioning scheme. The BIOS provides only five bits 
for sector number in a track and zero is invalid. And, sectors have to claim to 
be aligned on a cylinder boundary except for the first partition which is 
permitted to begin on the second head of the first cylinder (so sectors/track 
reserved sectors and the mis-alignment).

> - qcow2 (which uses aligned 4K blocks)
> - any host filesystem which uses 4K blocks
> - any host disk which uses 4K blocks (not yet common)
> 
> I can think of a few workarounds, all bad:
> - add a partitioning tool (or option to qemu-img) to format the disk, 
> placing the first partition on the fourth cylinder, aligning it.  tell 
> the users not to wipe the disks out but instead install to one of the 
> existing parititions
> - add a tool to optimize an existing disk by extending it and moving the 
> partitions around so they are aligned.  may break boot loaders.
> - make qcow4 use 512 byte sectors.  will increase overhead and doesn't 
> solve problems on the host filesystem and disk.
> - have qcow51 detect misaligned accesses and adjust itself somehow.  
> doesn't help raw and other formats.  likely very difficult.

A solution used on another platform I'm familiar with is for the platform's 
storage drivers to report disk geometry as having have 32 sectors per track. The 
native partitioning tool will then always create partitions with 32 sector 
alignment. It's possible a BIOS change could be made to achieve that.

> Does anybody know if scsi will have the same problems?  Can anyone 
> suggest other workarounds?

If you are using a guest platform like linux and don't need multiple file 
systems on the same [virtual] disk then just format the whole disk rather than 
partition it:

# mkfs -t ext3 /dev/sdb

(instead of /dev/sdb1, 2, 3, etc)

Re: 63 sectors

[Avi Kivity]

David Mair wrote:
> A solution used on another platform I'm familiar with is for the 
> platform's storage drivers to report disk geometry as having have 32 
> sectors per track. The native partitioning tool will then always 
> create partitions with 32 sector alignment. It's possible a BIOS 
> change could be made to achieve that.
>

I tried 60 sectors per track, but Windows saw the disk as an 8GB disk 
and refused to format.  Linux had similar problems.

>> Does anybody know if scsi will have the same problems?  Can anyone 
>> suggest other workarounds?
>
> If you are using a guest platform like linux and don't need multiple 
> file systems on the same [virtual] disk then just format the whole 
> disk rather than partition it:
>
> # mkfs -t ext3 /dev/sdb
>
> (instead of /dev/sdb1, 2, 3, etc)

That won't work with the installers (at least not simply), and doesn't 
allow for a swap partition.


-- 
error compiling committee.c: too many arguments to function

Re: 63 sectors

[Gerd Hoffmann]

> I can think of a few workarounds, all bad:
> - add a partitioning tool (or option to qemu-img) to format the disk,
> placing the first partition on the fourth cylinder, aligning it.  tell
> the users not to wipe the disks out but instead install to one of the
> existing parititions

Well, that one slightly modified could work out quite well, at least for
linux.

As far I know the linux kernel uses the geometry provided by the storage
adapter only in case the disk is blank.  If there is a partition table
present on the disk, the linux kernel will calculate the geometry based
on that.

Background for this is that scsi disks have no disk geometry, they just
have a bunch of sectors.  The scsi hostadapter bioses have to pull out
some geometry out of thin air to satisfy ms-dos and real os boot
loaders, and there was no standard on how to do that.  Thus moving disks
from one scsi adapter to another may result in hba-reported and on-disk
geometry being different, and the only sane way to deal with that is
using the on-disk geometry.  With LBA this is much less an issue these
days though.

So qemu-img could create a partition table, containing one partition,
hinting 32 sectors/cylinder.  Linux should keep that geometry then, even
when distro installers are deleting the partition and creating their own
scheme.

Dunno how what *BSD or Windows guests will deal with that though.

cheers,
  Gerd

Re: 63 sectors

[Gerd Hoffmann]

Gerd Hoffmann wrote:
> So qemu-img could create a partition table, containing one partition,
> hinting 32 sectors/cylinder.  Linux should keep that geometry then, even
> when distro installers are deleting the partition and creating their own
> scheme.

Scratch that, tested it, didn't work :-(

cheers,
  Gerd

Re: 63 sectors

[H. Peter Anvin]

Avi Kivity wrote:
> 
> That setting has some unfortunate side effects.  Partitioning tools will 
> locate the first partition at the second cylinder, which is at the 63rd 
> sector.  This means that if the guest uses a 4K block filesystem on the 
> first partition (an incredibly common occurance), then every single 
> access will not be 4K aligned with respect to the virtual block device.  
> This will cause fragmentation and read/modify/write cycles with:
> 
> - qcow2 (which uses aligned 4K blocks)
> - any host filesystem which uses 4K blocks
> - any host disk which uses 4K blocks (not yet common)
> 
> I can think of a few workarounds, all bad:
> - add a partitioning tool (or option to qemu-img) to format the disk, 
> placing the first partition on the fourth cylinder, aligning it.  tell 
> the users not to wipe the disks out but instead install to one of the 
> existing parititions
> - add a tool to optimize an existing disk by extending it and moving the 
> partitions around so they are aligned.  may break boot loaders.
> - make qcow4 use 512 byte sectors.  will increase overhead and doesn't 
> solve problems on the host filesystem and disk.
> - have qcow51 detect misaligned accesses and adjust itself somehow.  
> doesn't help raw and other formats.  likely very difficult.
> 
> Does anybody know if scsi will have the same problems?  Can anyone 
> suggest other workarounds?
> 

This is not just limited to KVM.  The right thing is for partitioning 
tools to realize that this isn't 1981 and actually align partitions to 
block boundaries.  Windows Vista aligns partitions to megabyte 
boundaries; I personally think that is somewhat excessive, but aligning 
partitions to 128K (or possibly 64K) would be better.

This idiotic default damages virtualizers, flash drives, hardware RAID, 
and pretty soon spinning media as well.

	-hpa

RE: 63 sectors

[Liu Yu-B13201]

> -----Original Message-----
> From: kvm-owner@vger.kernel.org 
> [mailto:kvm-owner@vger.kernel.org] On Behalf Of Avi Kivity
> Sent: Wednesday, September 03, 2008 4:12 PM
> To: KVM list
> Cc: H. Peter Anvin
> Subject: 63 sectors
> 
> Qemu sets the sectors-per-track setting of virtual disks to 63.  This 
> seems to be in accordance with the specs; drivers/ide/ide-disk.c says:
> 
> >         /*
> >          * The ATA spec tells large drives to return
> >          * C/H/S = 16383/16/63 independent of their size.
> >          * Some drives can be jumpered to use 15 heads 
> instead of 16.
> >          * Some drives can be jumpered to use 4092 cyls 
> instead of 16383.
> >          */
> >         if ((id->cyls == 16383
> >              || (id->cyls == 4092 && id->cur_cyls == 16383)) &&
> >             id->sectors == 63 &&
> >             (id->heads == 15 || id->heads == 16) &&
> >             (id->lba_capacity >= 16383*63*id->heads))
> >                 return 1;
> 
> That setting has some unfortunate side effects.  Partitioning 
> tools will 
> locate the first partition at the second cylinder, which is 
> at the 63rd 
> sector.  This means that if the guest uses a 4K block 
> filesystem on the 
> first partition (an incredibly common occurance), then every single 
> access will not be 4K aligned with respect to the virtual 
> block device.  
> This will cause fragmentation and read/modify/write cycles with:
> 
> - qcow2 (which uses aligned 4K blocks)
> - any host filesystem which uses 4K blocks
> - any host disk which uses 4K blocks (not yet common)
> 
> I can think of a few workarounds, all bad:
> - add a partitioning tool (or option to qemu-img) to format the disk, 
> placing the first partition on the fourth cylinder, aligning 
> it.  tell 
> the users not to wipe the disks out but instead install to one of the 
> existing parititions
> - add a tool to optimize an existing disk by extending it and 
> moving the 
> partitions around so they are aligned.  may break boot loaders.
> - make qcow4 use 512 byte sectors.  will increase overhead 
> and doesn't 
> solve problems on the host filesystem and disk.
> - have qcow51 detect misaligned accesses and adjust itself somehow.  
> doesn't help raw and other formats.  likely very difficult.
> 
> Does anybody know if scsi will have the same problems?  Can anyone 
> suggest other workarounds?
> 

How about adding 1 sector offset when access disk image.
So that it looks like the 63rd sector, but it's indeed the 64th.

Re: 63 sectors

[H. Peter Anvin]

Liu Yu-B13201 wrote:
> How about adding 1 sector offset when access disk image.
> So that it looks like the 63rd sector, but it's indeed the 64th.

The problem with that is you make it suck on OSes which actually 
partition sanely.

	-hpa

RE: 63 sectors

[Liu Yu-B13201]

> -----Original Message-----
> From: H. Peter Anvin [mailto:hpa@zytor.com] 
> Sent: Thursday, September 04, 2008 11:33 AM
> To: Liu Yu-B13201
> Cc: Avi Kivity; KVM list
> Subject: Re: 63 sectors
> 
> Liu Yu-B13201 wrote:
> > How about adding 1 sector offset when access disk image.
> > So that it looks like the 63rd sector, but it's indeed the 64th.
> 
> The problem with that is you make it suck on OSes which actually 
> partition sanely.
> 

You mean guest OSes?
Guest OSes never know this offset, adding one sector offset is in qemu and transparent to them.

Re: 63 sectors

[H. Peter Anvin]

Liu Yu-B13201 wrote:
>>
>> Liu Yu-B13201 wrote:
>>> How about adding 1 sector offset when access disk image.
>>> So that it looks like the 63rd sector, but it's indeed the 64th.
>> The problem with that is you make it suck on OSes which actually 
>> partition sanely.
> 
> You mean guest OSes?
> Guest OSes never know this offset, adding one sector offset is in qemu and transparent to them.

I mean guest OSes.

And no, they won't know the difference, but you are suggesting 
optimizing for guest OSes who are legacy/broken *AT THE EXPENSE* of 
guest OSes who do things sanely.

As partition and install utilities get a clue, there are going to be 
less of the former and more of the latter.

	-hpa

RE: 63 sectors

[Itamar Heim]

This can be left to the userspace to pass as an option, to optimize for relevant guests, since the one running the userspace probably knows which OS he will run on it, so he can pass "-sector64" as an optimization. 

-----Original Message-----
From: kvm-owner@vger.kernel.org [mailto:kvm-owner@vger.kernel.org] On Behalf Of H. Peter Anvin
Sent: Thursday, September 04, 2008 6:57 AM
To: Liu Yu-B13201
Cc: Avi Kivity; KVM list
Subject: Re: 63 sectors

Liu Yu-B13201 wrote:
>>
>> Liu Yu-B13201 wrote:
>>> How about adding 1 sector offset when access disk image.
>>> So that it looks like the 63rd sector, but it's indeed the 64th.
>> The problem with that is you make it suck on OSes which actually 
>> partition sanely.
> 
> You mean guest OSes?
> Guest OSes never know this offset, adding one sector offset is in qemu and transparent to them.

I mean guest OSes.

And no, they won't know the difference, but you are suggesting 
optimizing for guest OSes who are legacy/broken *AT THE EXPENSE* of 
guest OSes who do things sanely.

As partition and install utilities get a clue, there are going to be 
less of the former and more of the latter.

	-hpa
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: 63 sectors

[Avi Kivity]

Itamar Heim wrote:
> This can be left to the userspace to pass as an option, to optimize for relevant guests, since the one running the userspace probably knows which OS he will run on it, so he can pass "-sector64" as an optimization. 
>   

Or rather -drive ...,offset=512


-- 
error compiling committee.c: too many arguments to function

Re: 63 sectors

[Ian Kirk]

Avi Kivity wrote:

> - add a partitioning tool (or option to qemu-img) to format the disk, placing
> the first partition on the fourth cylinder, aligning it.  tell the users not
> to wipe the disks out but instead install to one of the existing parititions

I thought that updating qemu-img to have quasi-fdisk functionality was a
good idea myself. Is there a respected libfdisk.so thing that could be
used to abstract most of the work ?

Re: 63 sectors

[Laurent Vivier]

Le jeudi 04 septembre 2008 à 09:58 +0100, Ian Kirk a écrit :
> Avi Kivity wrote:
> 
> > - add a partitioning tool (or option to qemu-img) to format the disk, placing
> > the first partition on the fourth cylinder, aligning it.  tell the users not
> > to wipe the disks out but instead install to one of the existing parititions
> 
> I thought that updating qemu-img to have quasi-fdisk functionality was a
> good idea myself. Is there a respected libfdisk.so thing that could be
> used to abstract most of the work ?

libparted ?

Laurent
-- 
----------------- Laurent.Vivier@bull.net  ------------------
  "La perfection est atteinte non quand il ne reste rien à
ajouter mais quand il ne reste rien à enlever." Saint Exupéry

Re: 63 sectors

[H. Peter Anvin]

Ian Kirk wrote:
> Avi Kivity wrote:
> 
>> - add a partitioning tool (or option to qemu-img) to format the disk, placing
>> the first partition on the fourth cylinder, aligning it.  tell the users not
>> to wipe the disks out but instead install to one of the existing parititions
> 
> I thought that updating qemu-img to have quasi-fdisk functionality was a
> good idea myself. Is there a respected libfdisk.so thing that could be
> used to abstract most of the work ?

libparted, although I wouldn't call it "respectable".

Writing a partition table to an otherwise empty disk is actually dirt 
simple.

	-hpa

Re: 63 sectors

[H. Peter Anvin]

H. Peter Anvin wrote:
> 
> libparted, although I wouldn't call it "respectable".
> 

That was meant as a sarcastic remark.  Sorry for the confusion.

	-hpa

Re: 63 sectors

[Charles Duffy]

Ian Kirk wrote:
> I thought that updating qemu-img to have quasi-fdisk functionality was a
> good idea myself. Is there a respected libfdisk.so thing that could be
> used to abstract most of the work ?

Exporting textual output and piping that to sfdisk is one approach.