From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from [140.186.70.92] (port=51532 helo=eggs.gnu.org) by lists.gnu.org with esmtp (Exim 4.43) id 1Q7xjN-0002Ze-HX for qemu-devel@nongnu.org; Thu, 07 Apr 2011 18:31:18 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1Q7xjI-0007jB-JJ for qemu-devel@nongnu.org; Thu, 07 Apr 2011 18:31:13 -0400 Received: from mail-yx0-f173.google.com ([209.85.213.173]:40879) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1Q7xjI-0007ij-7H for qemu-devel@nongnu.org; Thu, 07 Apr 2011 18:31:08 -0400 Received: by yxk8 with SMTP id 8so1486662yxk.4 for ; Thu, 07 Apr 2011 15:31:07 -0700 (PDT) MIME-Version: 1.0 Date: Fri, 8 Apr 2011 06:31:07 +0800 Message-ID: From: Lyu Mitnick Content-Type: multipart/alternative; boundary=00163628518e5af61f04a05baad7 Subject: [Qemu-devel] GSoC - Improved VHD(Virtual Hard Disk) image format compatibility and usability List-Id: qemu-devel.nongnu.org List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , To: qemu-devel@nongnu.org --00163628518e5af61f04a05baad7 Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable Hello all, This is my only proposal for the Google Summer of Code. Since the student application deadline is very near, I have already submitted it to melange. Comments and suggestions are really appreciated. -- *Contact Information* Email : mitnick.lyu@gmail.com Phone Number : + 886-963233988 *Biography* I am a 22 years old computer science student in National Taiwan University. With the age of 20, I have finished my undergraduate project =93VNC (Virtua= l Network Computing) on EFI (Extensible Firmware Interface)=94. We can remote control the other computer without entering OS. As far as I know, there is no one do the project I did. So I have met many problems and there is no relevant information on web. I use background knowledge about BIOS and System Software to solve these issues by myself. I learned not only EFI skill but also the ability to find the solution of problems during the project. *Why QEMU* I use QEMU to emulate various CPU in my embedded system class. I appreciate to QEMU and I want to contribute to it. On the other hand, I like technique= s about system level software and I am so obsessed with the principle and implementation of virtualization and QEMU. *Why This Project* There isn=92t fully support of VHD (Virtual Hard Disk) format in QEMU. I li= st the new features that are not yet supported ([1], [2], [3]) in QEMU in the following: [1] *Add =93Fixed hard disk image=94 support into VHD format* [2] *Add =93Differencing hard disk image=94 support into VHD format* There are three image types of VHD (Virtual Hard Disk) format: =B7 Fixed hard disk image =B7 Dynamic hard disk image =B7 Differencing hard disk image However there is only =93Dynamic hard disk image=94 support in QEMU. [3] *Add =93Splitting hard disk images=94 support into VHD format* Some file systems, such as the FAT32 file system, have a 4-GB limit on file size. If the hard disk image expands more than 4 GB, hypervisor will split the hard disk image into another file, which QEMU doesn=92t support yet. [4] *Add =93asynchronous I/O=94 support into VHD format using coroutines* All newer block drivers support asynchronous reads and writes, only some ol= d drivers (ie. block dirver for VHD) don=92t. Adding asynchronous I/O to the block driver for VHD would improve its usability. There are two implementation methods of asynchronous I/O, callbacks and coroutines. Coroutines is used to replace callbacks to avoid: =B7 Splitting up code into separate functions across each blocking po= int =B7 Lot of code end up packing/unpacking variables that need to passe= d to callbacks =B7 Hard to follow the code I would try to make use of the ongoing coroutine effort in this project. I have an experience of porting binutils so I am familiar with object file/executable formats. I found there is a little relevance between object file/executable formats and virtual hard disk file format. So I choose this project as my first contribution to QEMU. *Schedule* 4/9 =96 4/25 Get familiar with Git, build system of QEMU, QEMU coding sty= le and QEMU coding guideline 4/26 =96 5/9 Add =93Fixed hard disk image=94 support into VHD format 5/10 =96 6/11 Add =93Differencing hard disk image=94 support into VHD forma= t 6/12 =96 7/11 Add =93Splitting hard disk images=94 support into VHD format 7/12 =96 7/15 Prepare for midterm evaluation 7/16 =96 8/15 Add =93asynchronous I/O=94 support into VHD format using coro= utine 8/16 =96 8/22 Write document, testing 8/23 =96 8/26 Prepare for final evaluation *Approach* The followings are the detailed description of the approach I will take. Fo= r each subproject, I list the basic steps to solve the problem and the modification points what I learned. [1] *Add =93Fixed hard disk image=94 support into VHD format* =93Fixed hard disk image=94 is =93raw file with a footer=94 in QEMU languag= es. Basic steps to solve the problem: =B7 Study the specification of VHD =B7 Trace the code of block/vpc.c o block/vpc.c implements block driver for VHD format =B7 Take a look of block/bochs.c o block/bochs.c implements block driver for the =93various=94 disk images= used by Bochs Modification points: =B7 block_int.h:BLOCK_FLAG_FIXED o #define BLOCK_FLAG_FIXED 8 =B7 block_int.h:BLOCK_OPT_FIXED o #define BLOCK_OPT_FIXED =93fixed=94, used as parameter options of vpc_create() =B7 block/vpc.c:vpc_open() o No dyndisk_header =B7 block/vpc.c:get_sector_offset() o Different formula (No modification in block/vpc.c:vpc_read()) =B7 block/vpc.c:vpc_create() o Deal with option BLOCK_OPT_FIXED o No dyndisk_header o No BAT (Block Allocation Table) =B7 block/vpc.c:vpc_close() o No pagetable =B7 block/vpc.c:vpc_create_options[] o Add element { .name =3D BLOCK_OPT_FIXED, .type =3D OPT_FLAG, .help =3D =93VHD fixed hard disk image=94 } [2] *Add =93Differencing hard disk image=94 support into VHD format* =93Differencing hard disk image=94 is =93VHD with backing file=94 in QEMU l= anguages. Basic steps to solve the problem: =B7 Study the specification of VHD =B7 Trace the code of block/vpc.c =B7 Take a look of block/bochs.c =B7 Take a look of block/qcow2.c o block/qcow2.c implements block driver for the QCOW version 2 format, which have backing file supported. Modification points: =B7 block/vpc.c:vpc_open() o Convert Parent Locator Entries to native path and store at backing_file =B7 block/vpc.c:vpc_read() o Read the sectors marked dirty from the differencing hard disk and the sectors marked clean from the parent hard disk o Open the parent hard disk file on demand (Optional optimization) =B7 block/vpc.c:vpc_create() o Deal with option BLOCK_OPT_BACKING_FMT o Set Disk Type, Unique ID, Parent Unique ID, Parent Time Stamp, Parent Unicode Name, Parent Locator Entries =B7 block/vpc.c:vpc_close() o Close all opened files =B7 block/vpc.c:vpc_create_options[] o Add element { .name =3D BLOCK_OPT_BACKING_FILE, .type =3D OPT_STRING, .help =3D "File name of a parent hard disk file" } [3] *Add =93Splitting hard disk images=94 support into VHD format* Basic steps to solve the problem: =B7 Study the specification of VHD =B7 Trace the code of block/vpc.c o This feature is not tied to the host platform or file system but image format Modification points: =B7 block/fat32.c o Design a block driver for fat32, which handles underlying 4-GB limit on file size and abstract a sequential continuous file. =B7 block/vpc.c:vpc_create() o Assign fat32 block driver to bs->drv if opened file is the first file o= f split files and initialize the fat32 block driver. o Assign fat32 block driver to bs->drv if file is opened under FAT32 file system and initialize the fat32 block driver. [4] *Add =93asynchronous I/O=94 support into VHD format using coroutine* Basic steps to solve the problem: =B7 Trace the code of block/vpc.c =B7 Take a look of block/qcow2.c (master tree) =B7 Take a look of block/qcow2.c (Kevin=92s branch) Modification points: =B7 block/vpc.c:vpc_aio_cancel() =B7 block/vpc.c:vpc_aio_pool =B7 block/vpc.c:vpc_aio_setup() =B7 block/vpc.c:vpc_aio_readv() =B7 block/vpc.c:vpc_aio_writev() =B7 block/vpc.c:vpc_aio_flush() -- Thanks a lot Mitnick --00163628518e5af61f04a05baad7 Content-Type: text/html; charset=Big5 Content-Transfer-Encoding: quoted-printable Hello all,

This is my only proposal for the Google Summe= r of Code. Since the student 
application deadline is v= ery near, I have already submitted it to melange. 
Comments = and suggestions are really appreciated.

--

Cont= act Information

 

Email : mitnick.lyu@gmail.com

Phone Number : + 886-963233988

 

Biog= raphy

 

I am a 22 years old computer science student = in National Taiwan University. With the age of 20, I have finished my undergraduate pro= ject “VNC (Virtual Network Computing) on EFI (Extensible Firmware Interfac= e)”. We can remote control the other computer without entering OS. As far as I know= , there is no one do the project I did. So I have met many problems and there is no relevant information on web. I use background knowledge about BIOS and Syst= em Software to solve these issues by myself. I learned not only EFI skill but = also the ability to find the solution of problems during the project.

 

Why QEMU

 

I use QEMU to emulate various CPU in my embed= ded system class. I appreciate to QEMU and I want to contribute to it. On the o= ther hand, I like techniques about system level software and I am so obsessed wi= th the principle and implementation of virtualization and QEMU.

 

Why This Project

 

There isn’t fully support of VHD (Virtu= al Hard Disk) format in QEMU. I list the new features that are not yet supported ([= 1], [2], [3]) in QEMU in the following:

 

[1] Add “Fixed hard disk image&rdquo= ; support into VHD format

[2] Add “Differencing hard disk imag= e” support into VHD format

 

There are three image types of VHD (Virtual H= ard Disk) format:

=A1P     =   Fixed hard disk image

=A1P     =   Dynamic hard disk image

=A1P     =   Differencing hard disk image

However there is only “Dynamic hard dis= k image” support in QEMU.

 

[3] Add “Splitting hard disk images&= rdquo; support into VHD format

 

Some file systems, such as the FAT32 file sys= tem, have a 4-GB limit on file size. If the hard disk image expands more than 4 = GB, hypervisor will split the hard disk image into another file, which QEMU doe= sn’t support yet.

 

[4] Add “asynchronous I/O” sup= port into VHD format using coroutines

 

All newer block drivers support asynchronous reads and writes, only some old drivers (ie. block dirver for VHD) don&rsqu= o;t. Adding asynchronous I/O to the block driver for VHD would improve its usability.

 

There are two implementation methods of asynchronous I/O, callbacks and coroutines. Coroutines is used to replace callbacks to avoid:

=A1P     =   Splitting up code into separate functions across each blocking point

=A1P     =   Lot of code end up packing/unpacking variables that need to passed to callbacks=

=A1P     =   Hard to follow the code

I would try to make use of the ongoing corout= ine effort in this project.

 

I have an experience of porting binutils so I= am familiar with object file/executable formats. I found there is a little relevance between object file/executable formats and virtual hard disk file format. S= o I choose this project as my first contribution to QEMU.

 

Sche= dule

 

4/9 &n= bsp; – 4/25 Get familiar with Git, build system of QEMU, QEMU codin= g style and       <= /span>

           &nbs= p;       QEMU coding guideline

4/26 – 5/9   Add “Fixed hard disk image” support int= o VHD format

5/10 – 6/11 Add “Differencing har= d disk image” support into VHD format

6/12 – 7/11 Add “Splitting hard d= isk images” support into VHD format

7/12 – 7/15 Prepare for midterm evaluat= ion

7/16 – 8/15 Add “asynchronous I/O= ” support into VHD format using coroutine

8/16 – 8/22 Write document, testing

8/23 – 8/26 Prepare for final evaluatio= n

 

Appr= oach

 

The followings are the detailed description o= f the approach I will take. For each subproject, I list the basic steps to so= lve the problem and the modification points what I learned.

 

[1] Add “Fixed hard disk image&rdquo= ; support into VHD format

 

“Fixed hard disk image” is &ldquo= ;raw file with a footer” in QEMU languages.

 

Basic steps to solve the problem:

=A1P     =   Study the specification of VHD

=A1P &n= bsp;     Trace the code of block/vpc.c

o   block/vpc.c implements block driver for VHD format

=A1P &n= bsp;     Take a look of block/bochs.c

o   block/bochs.c implements block driver for the “various” disk images used by Bochs

 

Modification points:

=A1P     =   block_int.h:BLOCK_FLAG_FIXED

o   #define BLOCK_FLAG_FIXED    8

=A1P     =   block_int.h:BLOCK_OPT_FIXED

o   #define BLOCK_OPT_FIXED    “fixed”, used as parameter options of vpc_create()

=A1P     =   block/vpc.c:vpc_open()

o   No dyndisk_header

=A1P     =   block/vpc.c:get_sector_offset()

o   Different formula (No modification in block/vpc.c:vpc_read())

=A1P     =   block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_FIXED

o   No dyndisk_header

o   No BAT (Block Allocation Table)

=A1P     =   block/vpc.c:vpc_close()

o   No pagetable

=A1P     =   block/vpc.c:vpc_create_options[]<= /p>

o   Add element
{
    .name =3D BLOCK_OPT_FIXED,
    .type =3D OPT_F= LAG,
    .help =3D &ldqu= o;VHD fixed hard disk image”
}

 

[2] Add “Differencing hard disk imag= e” support into VHD format

 

“Differencing hard disk image” is= “VHD with backing file” in QEMU languages.

 

Basic steps to solve the problem:

=A1P     =   Study the specification of VHD

=A1P     =   Trace the code of block/vpc.c

=A1P     =   Take a look of block/bochs.c

=A1P     =   Take a look of block/qcow2.c

o   block/qcow2.c implements block driver for the QCOW version 2 format, which have backing f= ile supported.

 

Modification points:

=A1P     =   block/vpc.c:vpc_open()

o   Convert Parent Locator Entries to native path and store at backing_file

=A1P     =   block/vpc.c:vpc_read()

o   Read the sectors marked dirty from the differencing hard disk and the sectors ma= rked clean from the parent hard disk

o   Open the parent hard disk file on demand (Optional optimization)

=A1P     =   block/vpc.c:vpc_create()

o   Deal with option BLOCK_OPT_BACKING_FMT

o   Set Disk Type, Unique ID, Parent Unique ID, Parent Time Stamp, Parent Unicode Name, Parent Locator Entries

=A1P     =   block/vpc.c:vpc_close()

o   Close all opened files

=A1P     =   block/vpc.c:vpc_create_options[]<= /p>

o   Add element
{        
    .name =3D BLOCK_OPT_BACKING_FILE,        
    .type =3D OPT_STRING,        
    .help =3D "= ;File name of a parent hard disk file"
}

 

[3] Add “Splitting hard disk images&= rdquo; support into VHD format

 

Basic steps to solve the problem:

=A1P     =   Study the specification of VHD

=A1P     =   Trace the code of block/vpc.c

o   This feature is not tied to the host platform or file system but image format

 

Modification points:

=A1P     =   block/fat32.c

o   Design a block driver for fat32, which handles underlying 4-GB limit on file size = and abstract a sequential continuous file.

=A1P     =   block/vpc.c:vpc_create()

o   Assign fat32 block driver to bs->drv if opened file is the first file of split files and initialize the fat32 block driver.

o   Assign fat32 block driver to bs->drv if file is opened under FAT32 file system = and initialize the fat32 block driver.

 

[4] Add “asynchronous I/O” sup= port into VHD format using coroutine

 

Basic steps to solve the problem:

=A1P     =   Trace the code of block/vpc.c

=A1P     =   Take a look of block/qcow2.c (master tree)

=A1P     =   Take a look of block/qcow2.c (Kevin’s branch)

 

Modification points:

=A1P     =   block/vpc.c:vpc_aio_cancel()

=A1P     =   block/vpc.c:vpc_aio_pool

=A1P     =   block/vpc.c:vpc_aio_setup()

=A1P     =   block/vpc.c:vpc_aio_readv()

=A1P     =   block/vpc.c:vpc_aio_writev()

=A1P     =   block/vpc.c:vpc_aio_flush()

 --

 

Thanks a lot

<= div>
Mitnick
--00163628518e5af61f04a05baad7--