Re: Xen Memory De-duplication

[Shriram Rajagopalan <rshriram@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 4706 bytes --]

Not sure about the DMA part, but I suggest you also take a look at satori
project code (memshr modules) in xen.
http://www.usenix.org/events/usenix09/tech/slides/milos.pdf

On Sun, Oct 10, 2010 at 5:34 AM, Pasi Kärkkäinen <pasik@iki.fi> wrote:

> On Sun, Oct 10, 2010 at 10:54:58AM +0530, Aditya Gadre wrote:
> >    This kind of implementation will require the disk blocks from
> different
> >    DomUs to be mapped to same physical disk block.
> >    For example,
> >    1) Shared read only filesystem
> >    2) Union based filesystem
> >    3) Virtual machine images deployed on a host filesystem which has
> >    deduplication enabled
> >
>
> I guess Xen blktap qcow* images should do? And maybe blktap2 VHD?
>
> -- Pasi
>
> >    What kind of  arrangement of filesystem is used in production
> environments
> >    for DomUs which host large number of VMs as in cloud enviorment?
> >
> >    On Sun, Oct 10, 2010 at 5:10 AM, Dan Magenheimer
> >    <[1]dan.magenheimer@oracle.com> wrote:
> >
> >      I*m not an expert on it but I believe this sounds very similar to
> the
> >      page sharing implementation that already exists in Xen 4.0.  The
> >      implementation in Xen only works on HVM guests and only on machines
> that
> >      have EPT though.  The patches (which were accepted into Xen) were
> posted
> >      here:
> >
> >
> >
> >      [2]
> http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
> >
> >
> >
> >      From: Aditya Gadre [mailto:[3]adivb2003@gmail.com]
> >      Sent: Saturday, October 09, 2010 11:56 AM
> >      To: [4]Xen-devel@lists.xensource.com
> >      Subject: [Xen-devel] Xen Memory De-duplication
> >
> >
> >
> >      Aim is to implement Xen Memory Deduplication with minimum overhead.
> >
> >      Our approach to de-duplication is as follows
> >
> >      In most cases, Domain-U uses a small set of well-known operating
> systems
> >      such as Linux, FreeBSD and Microsoft Windows. In such environment
> many
> >      domains share read-only filesystems that contain operating system
> and
> >      frequently usedprogram files and libraries.Each domain has their own
> >      writable filesystems for storing data and temporary files. In this
> >      configuration, multiple pages scattered in different domains mostly
> >      happen to contain same disk block. So, in our approach to perform
> >      deduplication we intend to add a data structure in dom 0 which store
> >      disk block number and the machine frame number(MFN) when a read
> request
> >      for the read only code(and data) is made. Now when another domain U
> >      places the request for the block of code and Dom 0 recieves a
> request
> >      for I/O (DMA), it will first check into the data structure for the
> entry
> >      for the block. If it finds the block it will return the MFN of the
> >      already read page and map it to the requesting domain's PFN
> resulting in
> >      zero I/O processing time of blocks which are already read. This in
> turn
> >      results in de-duplication of the read only pages accessed by
> multiple
> >      domains without any overhead of hashing the page.
> >
> >      Test case scenario:
> >
> >      Consider a Dom0 linux kernel using a filesystem with deduplication
> >      enabled. Then we install a DomU kernel with the virtual disk as a
> image
> >      file on the disk(.img). Then we make multiple copies of the image to
> >      deploy multiple DomUs running same kernel. Now, as deduplication is
> >      enabled in the file system initially all the blocks of the domains
> will
> >      be pointing to the same disk blocks. Now when the kernel's are
> booted,
> >      they all will consume memory only once for the programs(code
> segment)
> >      loaded in the memory. Now as these OSs start to write to their own
> >      virtual filesystems the blocks of the image will be COW'ed by the
> >      filesystem resulting in different block number.
> >      Is such a approach implemented?  We intend to implement this as a
> >      project. What are the suspected challanges?
> >
> >      Regards,
> >      Aditya Gadre
> >
> > References
> >
> >    Visible links
> >    1. mailto:dan.magenheimer@oracle.com
> >    2.
> http://lists.xensource.com/archives/html/xen-devel/2009-12/msg00797.html
> >    3. mailto:adivb2003@gmail.com
> >    4. mailto:Xen-devel@lists.xensource.com
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>



-- 
perception is but an offspring of its own self

[-- Attachment #1.2: Type: text/html, Size: 6231 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel