Re: Memory de-duplication (Persistent pages)

Re: Memory de-duplication (Persistent pages)

[Kaustubh Kabra]

Sir,

We are pretty clear with the internal working of tmem.
Now as "ephemeral" pages are already de-duplicated in current implementation .

1>>So, we propose to do sharing of "persistent" pages also .
2>>We also propose n/w bipass manager for "tmem" as its not existing currently.

Will it be really useful ?
If yes, it will be implemented within stipulated time frame.

Thank you.

RE: Memory de-duplication (Persistent pages)

[Dan Magenheimer]

In my opinion, both could be useful projects.

> -----Original Message-----
> From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
> Sent: Saturday, September 04, 2010 1:02 PM
> To: Xen-devel
> Cc: Dan Magenheimer
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> Sir,
> 
> We are pretty clear with the internal working of tmem.
> Now as "ephemeral" pages are already de-duplicated in current
> implementation .
> 
> 1>>So, we propose to do sharing of "persistent" pages also .
> 2>>We also propose n/w bipass manager for "tmem" as its not existing
> currently.
> 
> Will it be really useful ?
> If yes, it will be implemented within stipulated time frame.
> 
> Thank you.

Re: Memory de-duplication (Persistent pages)

[Kaustubh Kabra]

[-- Attachment #1.1: Type: text/plain, Size: 1663 bytes --]

To solve memory deduplication in Xen using Gpu :

Current problem :
If Memory deduplication is done on Cpu ,a lot of memory is used for data
structures and lot of computation time is wasted in finding duplicate page
and becomes waste, if duplicate page is not found.

Proposed GPU solution: Let Gpu perform seraching for a shared page using
multiple threads in parallel,improving searching operation and save data
structures on GPU device memory ,which is possible. Communication between
Cpu and Gpu can be done using inter processor synchronization which is also
possible. Cpu process will run as a daemon that will invoke gpu process when
ever deduplication is scheduled / if performance is good enough then for
every page written in memory.

Correct me , if anything is wrong.

On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer <dan.magenheimer@oracle.com
> wrote:

> In my opinion, both could be useful projects.
>
> > -----Original Message-----
> > From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
> > Sent: Saturday, September 04, 2010 1:02 PM
> > To: Xen-devel
> > Cc: Dan Magenheimer
> > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> >
> > Sir,
> >
> > We are pretty clear with the internal working of tmem.
> > Now as "ephemeral" pages are already de-duplicated in current
> > implementation .
> >
> > 1>>So, we propose to do sharing of "persistent" pages also .
> > 2>>We also propose n/w bipass manager for "tmem" as its not existing
> > currently.
> >
> > Will it be really useful ?
> > If yes, it will be implemented within stipulated time frame.
> >
> > Thank you.
>



-- 
*Kaustubh Kabra
http://www.kaustubhwise.000a.biz*

[-- Attachment #1.2: Type: text/html, Size: 2317 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

RE: Memory de-duplication (Persistent pages)

[Dan Magenheimer]

[-- Attachment #1.1: Type: text/plain, Size: 2520 bytes --]

This is an interesting idea, but  (1) there is no "standard " GPU so I don't think it will be possible to write Xen code generically for this; (2) most of the overhead for searching will be in moving data from RAM to cache... the CPU overhead is probably a small fraction so a GPU may not help much (unless it has a much wider bus to RAM); (3) in tmem's current deduplication implementation, the search is done synchronously on every put_page... I don't know if this will work well asynchronously due to locking, existing data structures, etc.


Dan

 

From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com] 
Sent: Wednesday, September 15, 2010 1:09 PM
To: Xen-devel
Cc: Dan Magenheimer
Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)

 

To solve memory deduplication in Xen using Gpu :

Current problem :
If Memory deduplication is done on Cpu ,a lot of memory is used for data structures and lot of computation time is wasted in finding duplicate page and becomes waste, if duplicate page is not found.

Proposed GPU solution: Let Gpu perform seraching for a shared page using multiple threads in parallel,improving searching operation and save data structures on GPU device memory ,which is possible. Communication between Cpu and Gpu can be done using inter processor synchronization which is also possible. Cpu process will run as a daemon that will invoke gpu process when ever deduplication is scheduled / if performance is good enough then for every page written in memory.

Correct me , if anything is wrong.

On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer <HYPERLINK "mailto:dan.magenheimer@oracle.com"dan.magenheimer@oracle.com> wrote:

In my opinion, both could be useful projects.


> -----Original Message-----
> From: Kaustubh Kabra [mailto:HYPERLINK "mailto:kaustubhwise@gmail.com"kaustubhwise@gmail.com]
> Sent: Saturday, September 04, 2010 1:02 PM
> To: Xen-devel
> Cc: Dan Magenheimer
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
>

> Sir,
>
> We are pretty clear with the internal working of tmem.
> Now as "ephemeral" pages are already de-duplicated in current
> implementation .
>
> 1>>So, we propose to do sharing of "persistent" pages also .
> 2>>We also propose n/w bipass manager for "tmem" as its not existing
> currently.
>
> Will it be really useful ?
> If yes, it will be implemented within stipulated time frame.
>
> Thank you.




-- 
Kaustubh Kabra
http://www.kaustubhwise.000a.biz



[-- Attachment #1.2: Type: text/html, Size: 8595 bytes --]

[-- Attachment #2: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Re: Memory de-duplication (Persistent pages)

[Pasi Kärkkäinen]

On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer wrote:
>    This is an interesting idea, but  (1) there is no "standard " GPU so I
>    don't think it will be possible to write Xen code generically for this;
>

How about things like OpenCL ? 

-- Pasi

>    (2) most of the overhead for searching will be in moving data from RAM to
>    cache... the CPU overhead is probably a small fraction so a GPU may not
>    help much (unless it has a much wider bus to RAM); (3) in tmem's current
>    deduplication implementation, the search is done synchronously on every
>    put_page... I don't know if this will work well asynchronously due to
>    locking, existing data structures, etc.
> 
>    Dan
> 
> 
> 
>    From: Kaustubh Kabra [mailto:kaustubhwise@gmail.com]
>    Sent: Wednesday, September 15, 2010 1:09 PM
>    To: Xen-devel
>    Cc: Dan Magenheimer
>    Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> 
> 
>    To solve memory deduplication in Xen using Gpu :
> 
>    Current problem :
>    If Memory deduplication is done on Cpu ,a lot of memory is used for data
>    structures and lot of computation time is wasted in finding duplicate page
>    and becomes waste, if duplicate page is not found.
> 
>    Proposed GPU solution: Let Gpu perform seraching for a shared page using
>    multiple threads in parallel,improving searching operation and save data
>    structures on GPU device memory ,which is possible. Communication between
>    Cpu and Gpu can be done using inter processor synchronization which is
>    also possible. Cpu process will run as a daemon that will invoke gpu
>    process when ever deduplication is scheduled / if performance is good
>    enough then for every page written in memory.
> 
>    Correct me , if anything is wrong.
> 
>    On Mon, Sep 6, 2010 at 11:05 PM, Dan Magenheimer
>    <[1]dan.magenheimer@oracle.com> wrote:
> 
>    In my opinion, both could be useful projects.
> 
>    > -----Original Message-----
>    > From: Kaustubh Kabra [mailto:[2]kaustubhwise@gmail.com]
>    > Sent: Saturday, September 04, 2010 1:02 PM
>    > To: Xen-devel
>    > Cc: Dan Magenheimer
>    > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
>    >
> 
>    > Sir,
>    >
>    > We are pretty clear with the internal working of tmem.
>    > Now as "ephemeral" pages are already de-duplicated in current
>    > implementation .
>    >
>    > 1>>So, we propose to do sharing of "persistent" pages also .
>    > 2>>We also propose n/w bipass manager for "tmem" as its not existing
>    > currently.
>    >
>    > Will it be really useful ?
>    > If yes, it will be implemented within stipulated time frame.
>    >
>    > Thank you.
> 
>    --
>    Kaustubh Kabra
>    [3]http://www.kaustubhwise.000a.biz
> 
> References
> 
>    Visible links
>    1. mailto:dan.magenheimer@oracle.com
>    2. mailto:kaustubhwise@gmail.com
>    3. http://www.kaustubhwise.000a.biz/

> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel

RE: Memory de-duplication (Persistent pages)

[Dan Magenheimer]

> From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> Sent: Thursday, September 16, 2010 12:24 PM
> To: Dan Magenheimer
> Cc: Kaustubh Kabra; Xen-devel
> Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> 
> On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer wrote:
> >    This is an interesting idea, but  (1) there is no "standard " GPU
> so I
> >    don't think it will be possible to write Xen code generically for
> this;
> >
> 
> How about things like OpenCL ?

While I know nothing about OpenCL other than a quick read
in wikipedia, I suspect that putting an OpenCL abstraction
layer in the hypervisor is not going to be popular with
Xen developers. :-)

Re: Memory de-duplication (Persistent pages)

[Pasi Kärkkäinen]

On Thu, Sep 16, 2010 at 12:34:17PM -0700, Dan Magenheimer wrote:
> > From: Pasi Kärkkäinen [mailto:pasik@iki.fi]
> > Sent: Thursday, September 16, 2010 12:24 PM
> > To: Dan Magenheimer
> > Cc: Kaustubh Kabra; Xen-devel
> > Subject: Re: [Xen-devel] Memory de-duplication (Persistent pages)
> > 
> > On Thu, Sep 16, 2010 at 09:22:29AM -0700, Dan Magenheimer wrote:
> > >    This is an interesting idea, but  (1) there is no "standard " GPU
> > so I
> > >    don't think it will be possible to write Xen code generically for
> > this;
> > >
> > 
> > How about things like OpenCL ?
> 
> While I know nothing about OpenCL other than a quick read
> in wikipedia, I suspect that putting an OpenCL abstraction
> layer in the hypervisor is not going to be popular with
> Xen developers. :-)
>

Hehe, yeah, I can believe that :) OpenCL gives GPU abstraction
for non-graphics apps.. that's why I mentioned it.

"Xen hypervisor 5.0 requires a GPU with at least 4 GB of memory 
with optimized OpenCL drivers to support memory de-duplication". 

Maybe not ;)

-- Pasi

Re: Memory de-duplication (Persistent pages)

[Tim Deegan]

At 20:34 +0100 on 16 Sep (1284669257), Dan Magenheimer wrote:
> > How about things like OpenCL ?
> 
> While I know nothing about OpenCL other than a quick read
> in wikipedia, I suspect that putting an OpenCL abstraction
> layer in the hypervisor is not going to be popular with
> Xen developers. :-)

There's nothing to say that the GPU code has to live in the hypervisor.
A tools domain that was already scanning for duplicates in other
domains' memory (or for that matter in block traffic) could use whatever
GPU resources it liked.

Even in the tmem case a domain could be given read-only access to the
tmem pool (er, if IOMMUs support such a thing, which I don't recall off
the top of my head) and supply hints to the hypervisor.  That way the
hypervisor would still be scanning with the CPU but only on pages where
it was likely to succeed.

Of course there's still a question of what the memory traffic does to
performance elsewhere (and if GPU cores are going to share L3 cache with
CPUs that could make a new bottleneck) but that's not to say it wouldn't
be a win.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, XenServer Engineering
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

RE: Memory de-duplication (Persistent pages)

[Dan Magenheimer]

> > While I know nothing about OpenCL other than a quick read
> > in wikipedia, I suspect that putting an OpenCL abstraction
> > layer in the hypervisor is not going to be popular with
> > Xen developers. :-)
> 
> There's nothing to say that the GPU code has to live in the hypervisor.
> A tools domain that was already scanning for duplicates in other
> domains' memory (or for that matter in block traffic) could use
> whatever
> GPU resources it liked.
> 
> Even in the tmem case a domain could be given read-only access to the
> tmem pool (er, if IOMMUs support such a thing, which I don't recall off
> the top of my head) and supply hints to the hypervisor.  That way the
> hypervisor would still be scanning with the CPU but only on pages where
> it was likely to succeed.

While I agree this is not impossible, tmem is designed to
be highly concurrent and the data structures manage potentially
millions of pages with thousands of pages put/got per vcpu
per domain per second.  I think just trying to deal with the
locking from a tools domain would negate any performance gain
from a GPU.  And, in any case, the overhead of tmem deduplication
is unlikely to be large enough (except possibly in contrived
cases) to be worth the work.