[Qemu-devel] GSoC - Improved VHD(Virtual Hard Disk) image format compatibility and usability

[Qemu-devel] GSoC - Improved VHD(Virtual Hard Disk) image format compatibility and usability

[Lyu Mitnick]

[-- Attachment #1: Type: text/plain, Size: 7716 bytes --]

Hello all,

This is my only proposal for the Google Summer of Code. Since the student
application deadline is very near, I have already submitted it to melange.
Comments and suggestions are really appreciated.

--

*Contact Information*



Email : mitnick.lyu@gmail.com

Phone Number : + 886-963233988



*Biography*



I am a 22 years old computer science student in National Taiwan University.
With the age of 20, I have finished my undergraduate project “VNC (Virtual
Network Computing) on EFI (Extensible Firmware Interface)”. We can remote
control the other computer without entering OS. As far as I know, there is
no one do the project I did. So I have met many problems and there is no
relevant information on web. I use background knowledge about BIOS and
System Software to solve these issues by myself. I learned not only EFI
skill but also the ability to find the solution of problems during the
project.



*Why QEMU*



I use QEMU to emulate various CPU in my embedded system class. I appreciate
to QEMU and I want to contribute to it. On the other hand, I like techniques
about system level software and I am so obsessed with the principle and
implementation of virtualization and QEMU.



*Why This Project*



There isn’t fully support of VHD (Virtual Hard Disk) format in QEMU. I list
the new features that are not yet supported ([1], [2], [3]) in QEMU in the
following:



[1] *Add “Fixed hard disk image” support into VHD format*

[2] *Add “Differencing hard disk image” support into VHD format*



There are three image types of VHD (Virtual Hard Disk) format:

·       Fixed hard disk image

·       Dynamic hard disk image

·       Differencing hard disk image

However there is only “Dynamic hard disk image” support in QEMU.



[3] *Add “Splitting hard disk images” support into VHD format*



Some file systems, such as the FAT32 file system, have a 4-GB limit on file
size. If the hard disk image expands more than 4 GB, hypervisor will split
the hard disk image into another file, which QEMU doesn’t support yet.



[4] *Add “asynchronous I/O” support into VHD format using coroutines*



All newer block drivers support asynchronous reads and writes, only some old
drivers (ie. block dirver for VHD) don’t. Adding asynchronous I/O to the
block driver for VHD would improve its usability.



There are two implementation methods of asynchronous I/O, callbacks and
coroutines. Coroutines is used to replace callbacks to avoid:

·       Splitting up code into separate functions across each blocking point

·       Lot of code end up packing/unpacking variables that need to passed
to callbacks

·       Hard to follow the code

I would try to make use of the ongoing coroutine effort in this project.



I have an experience of porting binutils so I am familiar with object
file/executable formats. I found there is a little relevance between object
file/executable formats and virtual hard disk file format. So I choose this
project as my first contribution to QEMU.



*Schedule*



4/9   – 4/25 Get familiar with Git, build system of QEMU, QEMU coding style
and

                   QEMU coding guideline

4/26 – 5/9   Add “Fixed hard disk image” support into VHD format

5/10 – 6/11 Add “Differencing hard disk image” support into VHD format

6/12 – 7/11 Add “Splitting hard disk images” support into VHD format

7/12 – 7/15 Prepare for midterm evaluation

7/16 – 8/15 Add “asynchronous I/O” support into VHD format using coroutine

8/16 – 8/22 Write document, testing

8/23 – 8/26 Prepare for final evaluation



*Approach*



The followings are the detailed description of the approach I will take. For
each subproject, I list the basic steps to solve the problem and the
modification points what I learned.



[1] *Add “Fixed hard disk image” support into VHD format*



“Fixed hard disk image” is “raw file with a footer” in QEMU languages.



Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

o   block/vpc.c implements block driver for VHD format

·       Take a look of block/bochs.c

o   block/bochs.c implements block driver for the “various” disk images used
by Bochs



Modification points:

·       block_int.h:BLOCK_FLAG_FIXED

o   #define BLOCK_FLAG_FIXED    8

·       block_int.h:BLOCK_OPT_FIXED

o   #define BLOCK_OPT_FIXED    “fixed”, used as parameter options of
vpc_create()

·       block/vpc.c:vpc_open()

o   No dyndisk_header

·       block/vpc.c:get_sector_offset()

o   Different formula (No modification in block/vpc.c:vpc_read())

·       block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_FIXED

o   No dyndisk_header

o   No BAT (Block Allocation Table)

·       block/vpc.c:vpc_close()

o   No pagetable

·       block/vpc.c:vpc_create_options[]

o   Add element
{
    .name = BLOCK_OPT_FIXED,
    .type = OPT_FLAG,
    .help = “VHD fixed hard disk image”
}



[2] *Add “Differencing hard disk image” support into VHD format*



“Differencing hard disk image” is “VHD with backing file” in QEMU languages.



Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

·       Take a look of block/bochs.c

·       Take a look of block/qcow2.c

o   block/qcow2.c implements block driver for the QCOW version 2 format,
which have backing file supported.



Modification points:

·       block/vpc.c:vpc_open()

o   Convert Parent Locator Entries to native path and store at backing_file

·       block/vpc.c:vpc_read()

o   Read the sectors marked dirty from the differencing hard disk and the
sectors marked clean from the parent hard disk

o   Open the parent hard disk file on demand (Optional optimization)

·       block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_BACKING_FMT

o   Set Disk Type, Unique ID, Parent Unique ID, Parent Time Stamp, Parent
Unicode Name, Parent Locator Entries

·       block/vpc.c:vpc_close()

o   Close all opened files

·       block/vpc.c:vpc_create_options[]

o   Add element
{
    .name = BLOCK_OPT_BACKING_FILE,
    .type = OPT_STRING,
    .help = "File name of a parent hard disk file"
}



[3] *Add “Splitting hard disk images” support into VHD format*



Basic steps to solve the problem:

·       Study the specification of VHD

·       Trace the code of block/vpc.c

o   This feature is not tied to the host platform or file system but image
format



Modification points:

·       block/fat32.c

o   Design a block driver for fat32, which handles underlying 4-GB limit on
file size and abstract a sequential continuous file.

·       block/vpc.c:vpc_create()

o   Assign fat32 block driver to bs->drv if opened file is the first file of
split files and initialize the fat32 block driver.

o   Assign fat32 block driver to bs->drv if file is opened under FAT32 file
system and initialize the fat32 block driver.



[4] *Add “asynchronous I/O” support into VHD format using coroutine*



Basic steps to solve the problem:

·       Trace the code of block/vpc.c

·       Take a look of block/qcow2.c (master tree)

·       Take a look of block/qcow2.c (Kevin’s branch)



Modification points:

·       block/vpc.c:vpc_aio_cancel()

·       block/vpc.c:vpc_aio_pool

·       block/vpc.c:vpc_aio_setup()

·       block/vpc.c:vpc_aio_readv()

·       block/vpc.c:vpc_aio_writev()

·       block/vpc.c:vpc_aio_flush()

 --



Thanks a lot

Mitnick

[-- Attachment #2: Type: text/html, Size: 49631 bytes --]