looking for ideas on VM related research projects

looking for ideas on VM related research projects

[Kallol Biswas]

Hi,
   I have a strong team of good, energetic and young kernel engineers
and research scholar and I am looking for ideas on linux virtual
memory related research projects for them.

Any input or pointers on future virtual memory projects will be appreciated.

Kallol Biswas
408-718-1864

Re: looking for ideas on VM related research projects

[Avi Kivity]

Kallol Biswas wrote:
> Hi,
>    I have a strong team of good, energetic and young kernel engineers
> and research scholar and I am looking for ideas on linux virtual
> memory related research projects for them.
>
> Any input or pointers on future virtual memory projects will be appreciated.
>
>   

- automatically using and breaking up large pages
- task/vma affinity
- automatically migrating memory when thread/vma affinity cannot be 
satisfied

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: looking for ideas on VM related research projects

[Kallol Biswas]

Hi,
    I am from IO and driver background, and not up to date with
current VM developments in linux.

Are you talking about NUMA architecture? Could you clarify or point me
to document that describes the problem with VMA/task affinity? Where
do affinity issues arise?


Kallol

On Sun, Jun 7, 2009 at 11:14 PM, Avi Kivity<avi@redhat.com> wrote:
> Kallol Biswas wrote:
>>
>> Hi,
>>   I have a strong team of good, energetic and young kernel engineers
>> and research scholar and I am looking for ideas on linux virtual
>> memory related research projects for them.
>>
>> Any input or pointers on future virtual memory projects will be
>> appreciated.
>>
>>
>
> - automatically using and breaking up large pages
> - task/vma affinity
> - automatically migrating memory when thread/vma affinity cannot be
> satisfied
>
> --
> Do not meddle in the internals of kernels, for they are subtle and quick to
> panic.
>
>

Re: looking for ideas on VM related research projects

[Avi Kivity]

Kallol Biswas wrote:
> Hi,
>     I am from IO and driver background, and not up to date with
> current VM developments in linux.
>
> Are you talking about NUMA architecture? 

Yes.

> Could you clarify or point me
> to document that describes the problem with VMA/task affinity?

Linux will try to allocate memory on the same node that the allocating 
thread ran on on the time the page was faulted in.  However, for long 
running processes, the thread will eventually migrate to some other cpu 
with only 1/N chance of actually running on the same node (N being the 
number of nodes).

If we can specify that a thread is related to some vma, the scheduler 
can try to keep that thread running on the vma's node, or else it can 
migrate the vma to a node with available cpu power.

>  Where do affinity issues arise?
>   

Large, long-running processes.  My specific interest is virtualization 
with kvm, but I believe HPC and databases can make use of this as well.  
HPC currently solves the problem by statically allocating memory and cpu 
resources.

-- 
error compiling committee.c: too many arguments to function

Re: looking for ideas on VM related research projects

[Wes Felter]

Kallol Biswas wrote:
> Hi,
>    I have a strong team of good, energetic and young kernel engineers
> and research scholar and I am looking for ideas on linux virtual
> memory related research projects for them.

There are some recent research projects that have not been integrated 
into Linux:

http://www.usenix.org/events/hotos09/tech/full_papers/saxena/saxena_html/
http://www.usenix.org/events/osdi08/tech/full_papers/gupta/gupta_html/index.html

I suspect that a combination of difference engine, compressed caching, 
and flash swapping could really benefit some environments.

Wes Felter - wesley@felter.org