Re: [Qemu-devel] [BUG] qemu-1.1.2 [FIXED-BY] qcow2: Fix avail_sectors in cluster allocation code

[Philipp Hahn <hahn@univention.de>]

[-- Attachment #1.1: Type: text/plain, Size: 1336 bytes --]

Hello Kevin,

On Wednesday 12 December 2012 18:29:48 Philipp Hahn wrote:
> I just re-run my "git bisect run ~/bisect.sh"  case, but it again arrived
> at that patch. I just queued another run for tonight so make sure the test
> is reliable:

The run from last night again arrived at the refecenced patch.

> > Ideally we would find a sequence of qemu-io commands to reliably
> > reproduce this. First thing worth trying would be running the current
> > qemu-iotests suite on the old versions. If we don't find it this way, I
> > guess we need to catch it with code review. I'm not sure if I can get to
> > it this week, and starting next week I'll be on vacation, so any help
> > with finding a reproducer would be appreciated.

I took a closer look at what gets corrupted; I've attached my notes.
Please notice that the partitions are not alignd properly.

If you would like to look at the full qcow2_alloc_clusters_offset trace, I can 
provide you with a link to the trace file.

BYtE
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        be open.                       fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/

[-- Attachment #1.2: 29355_qemu-corruption2.txt --]
[-- Type: text/plain, Size: 5539 bytes --]

# FILE=/var/cache/apt/archives/liblqr-1-0_0.4.1-1.3.201104131631_amd64.deb
# debugfs /dev/vg_ucs/rootfs -R "stat $FILE"
debugfs 1.41.12 (17-May-2010)
Inode: 1385717   Type: regular    Mode:  0644   Flags: 0x0
Generation: 2854262870    Version: 0x00000000
User:     0   Group:     0   Size: 38884
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 80
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x50979ae9 -- Mon Nov  5 11:54:33 2012
atime: 0x50979b0d -- Mon Nov  5 11:55:09 2012
mtime: 0x4de1dbce -- Sun May 29 07:38:22 2011
Size of extra inode fields: 4
BLOCKS:
(0-9):5728401-5728410
TOTAL: 10

# BSIZE=4096
# BOFFSET=0
# dd bs=$BSIZE count=1 if=$FILE skip=$BOFFSET 2>/dev/null | md5sum
065b19ba6e9153dcc88003ea06076f9f  -

# BLOCK=5728401
# dd bs=$BSIZE count=1 if=/dev/vg_ucs/rootfs skip=$BLOCK 2>/dev/null | md5sum
065b19ba6e9153dcc88003ea06076f9f  -

# dmsetup table
vg_ucs-rootfs: 0 97910784 linear 254:3 384
# LV_SOFFSET=384
# dd bs=512c count=8 if=/dev/vda3 skip=$((BLOCK*8+LV_SOFFSET)) 2>/dev/null | md5sum
065b19ba6e9153dcc88003ea06076f9f  -

# fdisk -l -u /dev/vda
/dev/vda3         4739175   104856254    50058540   8e  Linux LVM
# PART_SOFFSET=4739175
# dd bs=512c count=8 if=/dev/vda skip=$((BLOCK*8+LV_SOFFSET+PART_SOFFSET)) 2>/dev/null | md5sum
065b19ba6e9153dcc88003ea06076f9f  -

# debugfs /dev/vg_ucs/rootfs -R "icheck $(seq 5728387 5728403 | tr '\n' ' ')"
debugfs 1.41.12 (17-May-2010)
Block   Inode number
5728387 1385715  
5728388 1385715  
5728389 1385715  
5728390 1385715  
5728391 1385715  
5728392 1385715  
5728393 1385715  
5728394 1385715  
5728395 1385715  
5728396 1385715  
5728397 1385716  
5728398 1385716  
5728399 1385716  
5728400 1385716  
5728401 1385717  
5728402 1385717  
5728403 1385717  

# debugfs /dev/vg_ucs/rootfs -R "ncheck 1385715 1385716 1385717"
debugfs 1.41.12 (17-May-2010)
Inode   Pathname 
1385715 /var/cache/apt/archives/libhtml-template-perl_2.9-2.7.201104290220_all.deb
1385717 /var/cache/apt/archives/liblqr-1-0_0.4.1-1.3.201104131631_amd64.deb
1385716 /var/cache/apt/archives/libio-socket-inet6-perl_2.65-1.1.3.201104291113_all.deb

# md5sum /var/cache/apt/archives/libhtml-template-perl_2.9-2.7.201104290220_all.deb /var/cache/apt/archives/liblqr-1-0_0.4.1-1.3.201104131631_amd64.deb /var/cache/apt/archives/libio-socket-inet6-perl_2.65-1.1.3.201104291113_all.deb
123f4c338bd875825d5d762e8bb48b2a  /var/cache/apt/archives/libhtml-template-perl_2.9-2.7.201104290220_all.deb
eadbf53d7313df2560f4741fbe982008  /var/cache/apt/archives/liblqr-1-0_0.4.1-1.3.201104131631_amd64.deb
d0c7bf2d62c125409e00e459ac94277a  /var/cache/apt/archives/libio-socket-inet6-perl_2.65-1.1.3.201104291113_all.deb

# apt-cache show libhtml-template-perl liblqr-1-0 libio-socket-inet6-perl | egrep 'MD5sum|Package'
Package: libhtml-template-perl
MD5sum: 123f4c338bd875825d5d762e8bb48b2a
Package: liblqr-1-0
MD5sum: 94ccb97b38bedef97072fdcc7bce1872
Package: libio-socket-inet6-perl
MD5sum: 8f04f2da2a7d2eefb9e77bf87a0b8972



# IMAGE=/var/lib/libvirt/images/stefan_UCS-3.0-2-13.3-Kolab-Slave.qcow2
# BLOCK=5728401 BSIZE=4096 BOFFSET=0 LV_SOFFSET=384 PART_SOFFSET=4739175
# OFFSET=$((BLOCK*BSIZE + LV_SOFFSET*512 + PART_SOFFSET*512))
# echo $((OFFSET >> 16 >> 13)) $(((OFFSET >> 16) & ((1 << 13) - 1))) $(((OFFSET >> 0) & ((1 << 16) - 1)))
48 1836 56832
# qemu-io -c "read -v $OFFSET $BSIZE" "$IMAGE" | sed -ne 's/^\([0-9a-f]\+:\) /\1/p' | xxd -r -seek -$OFFSET | md5sum
065b19ba6e9153dcc88003ea06076f9f  -
# qcow2.py -r -s "$IMAGE" --read $OFFSET $BSIZE | xxd -r | md5sum
l1=0x30 l2=0x72c c=0xde00
065b19ba6e9153dcc88003ea06076f9f  -

# qcow2.py -r -s "$IMAGE" | egrep "L1\[$((0x30))\]|L2\[$((0x72c))\]|cluster_bits| size|l1_size"
0000000000000000: + cluster_bits=16 (64 KiB)
0000000000000000: + size=50 GiB
0000000000000000: + l1_size=150
0000000014780000: + L1[48]=0x800000016cad0000
000000016cad0000: +  L2[1836]=0x8000000177a60000
# dd bs=1 count=$BSIZE if=$IMAGE skip=$((0x177a60000+0xde00)) 2>/dev/null | md5sum
065b19ba6e9153dcc88003ea06076f9f  -

# printf '0x%0x\n' $OFFSET
0x6072cde00
# ./scripts/simpletrace.py trace-events /trace-17705 | grep -n -A3 offset=0x6072c
2132:qcow2_alloc_clusters_offset    10.707 co=0x1938190 offset=0x6072c4e00 n_start=0x27 n_end=0x11f
2133-qcow2_alloc_clusters_offset    19.610 co=0x15c5030 offset=0x6072e4e00 n_start=0x27 n_end=0x417
2134-qcow2_alloc_clusters_offset 22891.351 co=0x1938950 offset=0x607250000 n_start=0x0  n_end=0x39f
2135-qcow2_alloc_clusters_offset    20.294 co=0x1961550 offset=0x607200000 n_start=0x0  n_end=0x1e7
3236:qcow2_alloc_clusters_offset 64432.668 co=0x1938950 offset=0x6072c0000 n_start=0x0  n_end=0x1f
2137-qcow2_alloc_clusters_offset  1557.462 co=0x1961340 offset=0x606fd0000 n_start=0x0  n_end=0x57
2138-qcow2_alloc_clusters_offset 17697.902 co=0x15c5030 offset=0x6072f0000 n_start=0x0  n_end=0x397
2139-qcow2_alloc_clusters_offset 97534.098 co=0x15c5030 offset=0x601625e00 n_start=0x2f n_end=0x14df
--
2827:qcow2_alloc_clusters_offset    16.653 co=0x195ca40 offset=0x6072c3e00 n_start=0x1f n_end=0x27
2827:qcow2_alloc_clusters_offset    16.653 co=0x195ca40 offset=0x6072c3e00 n_start=0x1f n_end=0x27
2828-qcow2_alloc_clusters_offset    31.017 co=0x195c5d0 offset=0x606c80000 n_start=0x0  n_end=0x7
2829-qcow2_alloc_clusters_offset   757.123 co=0x195c5d0 offset=0x6072e3e00 n_start=0x1f n_end=0x27
2830-qcow2_alloc_clusters_offset     9.656 co=0x195ca40 offset=0x607497e00 n_start=0x3f n_end=0x47

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 197 bytes --]