[Qemu-devel] Ideas wiki for GSoC 2010

[Qemu-devel] Ideas wiki for GSoC 2010

[Luiz Capitulino]

 Hi there,

 Our wiki page for the Summer of Code 2010 is doing quite well:

http://wiki.qemu.org/Google_Summer_of_Code_2010

 Now the most important is:

1. Get mentors assigned to projects. Just put your name and email in the
   right field. It's ok and even desirable to have two mentors per project,
   but please remember that mentoring is serious work, more info here:

   http://code.google.com/p/google-summer-of-code/wiki/AdviceforMentors

   http://gsoc-wiki.osuosl.org/index.php/Main_Page

2. Do we have kvm-specific projects? Can they be part of the QEMU project
   or do we need a different mentoring organization for it?

3. Fill in the missing information for the suggested project (description,
   skill level, languages, etc)

 I will complete our application tomorrow or on Friday.

PS: I'm CC'ing everyone who suggested projects there, except one or two
I couldn't find the email address.

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>
> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>     or do we need a different mentoring organization for it?
>    

Something really interesting is kvm-assisted tcg.  I'm afraid it's a bit 
too complicated to GSoC.


-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Paolo Bonzini]

On 03/11/2010 08:55 AM, Avi Kivity wrote:
> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>
>> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>> or do we need a different mentoring organization for it?
>
> Something really interesting is kvm-assisted tcg. I'm afraid it's a bit
> too complicated to GSoC.

I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in 
some time, so it's good to have ideas written down.

Also, the selection of projects will be done by members of the 
community, by grading the student's submissions.  The bar would be 
placed higher for someone who picks a complicated project.

Paolo

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Alexander Graf]

On 11.03.2010, at 10:43, Paolo Bonzini wrote:

> On 03/11/2010 08:55 AM, Avi Kivity wrote:
>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>> 
>>> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>>> or do we need a different mentoring organization for it?
>> 
>> Something really interesting is kvm-assisted tcg. I'm afraid it's a bit
>> too complicated to GSoC.
> 
> I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in some time, so it's good to have ideas written down.
> 
> Also, the selection of projects will be done by members of the community, by grading the student's submissions.  The bar would be placed higher for someone who picks a complicated project.

The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)

== Shared memory transport between guest(s) and host ==

Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.

== Pass through file systems (9p, CIFS) ==

I dislike CIFS now that we use it regularly. It just doesn't work for Linux to Linux communication. But as far as 9P is concerned, do you need help there Anthony? If so, would you take over the mentoring?

== Add more sophisticated encodings to VNC server ==

I could probably help out being a secondary mentor here, but Anthony would be a good fit as primary, no? I guess Kraxel could help out too.

== Write a C QMP library based on QEMU JSON and QMP code ==

Suggested by Anthony, mentored by Anthony? :) Possible other candidates are Luiz and Kraxel I guess? I haven't really tracked QMP that much.

== Add support for guest copy/paste ==

This should probably be folded into the above VNC server improvements. By itself it's just too little of a task.

== Device state visualization ==

Jan, Kraxel? Maybe too small for a task?

== Upstreaming some of the Android emulator bits ==

Jan, Anthony?


If you read the suggestion and just think to yourself "well yes, I think I could do it" - then put your name in the wiki :).

Alex

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/11/2010 01:25 PM, Alexander Graf wrote:
> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>
> == Shared memory transport between guest(s) and host ==
>
> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
>    

Not sure what this is.


-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Alexander Graf]

On 11.03.2010, at 12:54, Avi Kivity wrote:

> On 03/11/2010 01:25 PM, Alexander Graf wrote:
>> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>> 
>> == Shared memory transport between guest(s) and host ==
>> 
>> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
>>   
> 
> Not sure what this is.

Cam's shared memory device.

Alex

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/11/2010 01:56 PM, Alexander Graf wrote:
> On 11.03.2010, at 12:54, Avi Kivity wrote:
>
>    
>> On 03/11/2010 01:25 PM, Alexander Graf wrote:
>>      
>>> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>>>
>>> == Shared memory transport between guest(s) and host ==
>>>
>>> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
>>>
>>>        
>> Not sure what this is.
>>      
> Cam's shared memory device.
>    

That's plain shared memory among guests (though the host could also 
participate).  "transport" evokes something like virtio rings.

I could mentor it, though I prefer something in kvm, and it looks close 
to completion.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Alexander Graf]

On 11.03.2010, at 12:58, Avi Kivity wrote:

> On 03/11/2010 01:56 PM, Alexander Graf wrote:
>> On 11.03.2010, at 12:54, Avi Kivity wrote:
>> 
>>   
>>> On 03/11/2010 01:25 PM, Alexander Graf wrote:
>>>     
>>>> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>>>> 
>>>> == Shared memory transport between guest(s) and host ==
>>>> 
>>>> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
>>>> 
>>>>       
>>> Not sure what this is.
>>>     
>> Cam's shared memory device.
>>   
> 
> That's plain shared memory among guests (though the host could also participate).  "transport" evokes something like virtio rings.
> 
> I could mentor it, though I prefer something in kvm, and it looks close to completion.

I agree. Take it off the list then :-).

Another idea I'd have would be upstream integration (and cleanup) of the ARM KVM port: https://wiki.ncl.cs.columbia.edu/wiki/index.php/AndroidVirt:MainPage

Alex

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/11/2010 02:03 PM, Alexander Graf wrote:
> Another idea I'd have would be upstream integration (and cleanup) of the ARM KVM port: https://wiki.ncl.cs.columbia.edu/wiki/index.php/AndroidVirt:MainPage
>    

Huh, didn't even know this thing existed.  Definitely something to merge.

-- 
error compiling committee.c: too many arguments to function

[Qemu-devel] Re: [Android-virt] Ideas wiki for GSoC 2010

[Christoffer Dall]

On Thu, Mar 11, 2010 at 1:15 PM, Avi Kivity <avi@redhat.com> wrote:
> On 03/11/2010 02:03 PM, Alexander Graf wrote:
>> Another idea I'd have would be upstream integration (and cleanup) of the ARM KVM port: https://wiki.ncl.cs.columbia.edu/wiki/index.php/AndroidVirt:MainPage
>>
>
> Huh, didn't even know this thing existed.  Definitely something to merge.

I have been wanted to announce this work for some time, but always
thought my patches were too unpolished. However, now is as good a time
as any. In addition to the link provided by Alex, the sources can be
viewed on http://git.ncl.cs.columbia.edu/git and I'm hoping to have
something mature enough for upstream some time around the summer.

I'm unsure how this relates to GSoC, but let me me know how I
could/should be involved.

>
> --
> error compiling committee.c: too many arguments to function
>
> _______________________________________________
> Android-virt mailing list
> Android-virt@lists.cs.columbia.edu
> https://lists.cs.columbia.edu/cucslists/listinfo/android-virt
>

[Qemu-devel] Re: [Android-virt] Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Fri, 12 Mar 2010 11:49:48 +0100
Christoffer Dall <christofferdall@christofferdall.dk> wrote:

> On Thu, Mar 11, 2010 at 1:15 PM, Avi Kivity <avi@redhat.com> wrote:
> > On 03/11/2010 02:03 PM, Alexander Graf wrote:
> >> Another idea I'd have would be upstream integration (and cleanup) of the ARM KVM port: https://wiki.ncl.cs.columbia.edu/wiki/index.php/AndroidVirt:MainPage
> >>
> >
> > Huh, didn't even know this thing existed.  Definitely something to merge.
> 
> I have been wanted to announce this work for some time, but always
> thought my patches were too unpolished. However, now is as good a time
> as any. In addition to the link provided by Alex, the sources can be
> viewed on http://git.ncl.cs.columbia.edu/git and I'm hoping to have
> something mature enough for upstream some time around the summer.

 I'd send RFCs early, just to be sure everything is going in the right
direction. It's okay to have unpolished code in RFCs.

> I'm unsure how this relates to GSoC, but let me me know how I
> could/should be involved.

 I'm not sure either, but doesn't seem a good project if it's only
polishing.

 On the other hand, if there's anything missing that should be
implemented you can be involved by suggesting the project and
becoming the mentor for it.

 Here is our ideas page:

http://wiki.qemu.org/Google_Summer_of_Code_2010

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Cam Macdonell]

On Thu, Mar 11, 2010 at 5:03 AM, Alexander Graf <agraf@suse.de> wrote:
>
> On 11.03.2010, at 12:58, Avi Kivity wrote:
>
>> On 03/11/2010 01:56 PM, Alexander Graf wrote:
>>> On 11.03.2010, at 12:54, Avi Kivity wrote:
>>>
>>>
>>>> On 03/11/2010 01:25 PM, Alexander Graf wrote:
>>>>
>>>>> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>>>>>
>>>>> == Shared memory transport between guest(s) and host ==
>>>>>
>>>>> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.
>>>>>
>>>>>
>>>> Not sure what this is.
>>>>
>>> Cam's shared memory device.
>>>
>>
>> That's plain shared memory among guests (though the host could also participate).  "transport" evokes something like virtio rings.
>>
>> I could mentor it, though I prefer something in kvm, and it looks close to completion.
>
> I agree. Take it off the list then :-).
>

Fair enough.  I'd be willing to take up one of the other suggestions.

Cam

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Paolo Bonzini]

On 03/11/2010 12:25 PM, Alexander Graf wrote:
> == Write a C QMP library based on QEMU JSON and QMP code ==
>
> Suggested by Anthony, mentored by Anthony?:)  Possible other
> candidates are Luiz and Kraxel I guess? I haven't really tracked QMP
> that much.

If you guys are okay with this, I think I could mentor since I followed 
the design of QMP quite closely (and this is the only one that I think I 
could do a decent job with).

BTW, it worked out much better for me in the past when the student and 
mentor were in a similar time zone.

Paolo

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Thu, 11 Mar 2010 13:09:37 +0100
Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 03/11/2010 12:25 PM, Alexander Graf wrote:
> > == Write a C QMP library based on QEMU JSON and QMP code ==
> >
> > Suggested by Anthony, mentored by Anthony?:)  Possible other
> > candidates are Luiz and Kraxel I guess? I haven't really tracked QMP
> > that much.
> 
> If you guys are okay with this, I think I could mentor since I followed 
> the design of QMP quite closely (and this is the only one that I think I 
> could do a decent job with).

 Sure.

> 
> BTW, it worked out much better for me in the past when the student and 
> mentor were in a similar time zone.
> 
> Paolo

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Edgar E. Iglesias]

IMO one of the biggest parts of infrastructure QEMU is missing
for embedded stuff is a common (qdev) way of modeling DMA backchannels.
Or devide to device channels or whatever you want to call it.

Would be nice if someone could bring this to qemu.

Cheers

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Thu, 11 Mar 2010 12:25:24 +0100
Alexander Graf <agraf@suse.de> wrote:

> == Write a C QMP library based on QEMU JSON and QMP code ==
> 
> Suggested by Anthony, mentored by Anthony? :) Possible other candidates are Luiz and Kraxel I guess? I haven't really tracked QMP that much.

 I didn't candidate as a mentor myself because Anthony has a better idea wrt
to the public API.

 But I certainly can help with the implementation. I have more two or three
QMP projects to suggest, btw.

> == Add support for guest copy/paste ==
> 
> This should probably be folded into the above VNC server improvements. By itself it's just too little of a task.
> 
> == Device state visualization ==
> 
> Jan, Kraxel? Maybe too small for a task?

 I think that whether a task is small or not also depends on the student,
of course that we should not come up with a project that can be easily
done in two weeks.

 On the other hand, 'not that difficult' tasks can be an excellent project
for those really new to open source and serious development. You know, when
you're a starter you spend quite a lot of time reading code and trying
things out (and there's nothing wrong with that).

 So, for this kind of project the mentor only should take extra care to
choose a student that is really going to learn a lot in the project.

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Jes Sorensen]

On 03/11/10 12:25, Alexander Graf wrote:
> The list is also still missing a lot of potential mentors for the listed ideas. Let me propose some here :)
>
> == Shared memory transport between guest(s) and host ==
>
> Sounds like Avi would be a good fit. I'm pretty unknowledgeable when it comes to shm.

Actually this one sounds a lot like SGI's XPMEM. Not sure if it would
make sense to try and use that infrastructure for that, but it might
be a good starting point....

Cheers,
Jes

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Thu, 11 Mar 2010 10:43:09 +0100
Paolo Bonzini <pbonzini@redhat.com> wrote:

> On 03/11/2010 08:55 AM, Avi Kivity wrote:
> > On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> >>
> >> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
> >> or do we need a different mentoring organization for it?
> >
> > Something really interesting is kvm-assisted tcg. I'm afraid it's a bit
> > too complicated to GSoC.
> 
> I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in 
> some time, so it's good to have ideas written down.
> 
> Also, the selection of projects will be done by members of the 
> community, by grading the student's submissions.  The bar would be 
> placed higher for someone who picks a complicated project.

 Exactly, we also have a 'skill level' tag, setting it to high should
help and note that we can have from grad students to phd ones.

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Alexander Graf]

On 11.03.2010, at 13:59, Luiz Capitulino wrote:

> On Thu, 11 Mar 2010 10:43:09 +0100
> Paolo Bonzini <pbonzini@redhat.com> wrote:
> 
>> On 03/11/2010 08:55 AM, Avi Kivity wrote:
>>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>>> 
>>>> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>>>> or do we need a different mentoring organization for it?
>>> 
>>> Something really interesting is kvm-assisted tcg. I'm afraid it's a bit
>>> too complicated to GSoC.
>> 
>> I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in 
>> some time, so it's good to have ideas written down.
>> 
>> Also, the selection of projects will be done by members of the 
>> community, by grading the student's submissions.  The bar would be 
>> placed higher for someone who picks a complicated project.
> 
> Exactly, we also have a 'skill level' tag, setting it to high should
> help and note that we can have from grad students to phd ones.

I don't think we should put in a correlation between skill level and degree. I myself only have a Bachelor's degree :-).


Alex

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Thu, 11 Mar 2010 14:00:46 +0100
Alexander Graf <agraf@suse.de> wrote:

> 
> On 11.03.2010, at 13:59, Luiz Capitulino wrote:
> 
> > On Thu, 11 Mar 2010 10:43:09 +0100
> > Paolo Bonzini <pbonzini@redhat.com> wrote:
> > 
> >> On 03/11/2010 08:55 AM, Avi Kivity wrote:
> >>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> >>>> 
> >>>> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
> >>>> or do we need a different mentoring organization for it?
> >>> 
> >>> Something really interesting is kvm-assisted tcg. I'm afraid it's a bit
> >>> too complicated to GSoC.
> >> 
> >> I suppose the GSoC ideas wiki page will migrate to a QEMU ideas page in 
> >> some time, so it's good to have ideas written down.
> >> 
> >> Also, the selection of projects will be done by members of the 
> >> community, by grading the student's submissions.  The bar would be 
> >> placed higher for someone who picks a complicated project.
> > 
> > Exactly, we also have a 'skill level' tag, setting it to high should
> > help and note that we can have from grad students to phd ones.
> 
> I don't think we should put in a correlation between skill level and degree. I myself only have a Bachelor's degree :-).

 Absolutely, my bad :)

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Jamie Lokier]

Avi Kivity wrote:
> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> >
> >2. Do we have kvm-specific projects? Can they be part of the QEMU project
> >    or do we need a different mentoring organization for it?
> >   
> 
> Something really interesting is kvm-assisted tcg.  I'm afraid it's a bit 
> too complicated to GSoC.

Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?

-- Jamie

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Paolo Bonzini]

On 03/11/2010 11:59 PM, Jamie Lokier wrote:
> Avi Kivity wrote:
>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>>
>>> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>>>     or do we need a different mentoring organization for it?
>>>
>>
>> Something really interesting is kvm-assisted tcg.  I'm afraid it's a bit
>> too complicated to GSoC.
>
> Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?

Or running TCG-compiled code as a guest so that for example KVM takes 
care of the softmmu (with hardware-assisted paging possibly, yoohoo). 
But I have no idea if that is what Avi meant. :-)

Paolo

Re: [Qemu-devel] Re: Ideas wiki for GSoC 2010

[Gleb Natapov]

On Fri, Mar 12, 2010 at 10:48:51AM +0100, Paolo Bonzini wrote:
> On 03/11/2010 11:59 PM, Jamie Lokier wrote:
> >Avi Kivity wrote:
> >>On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> >>>
> >>>2. Do we have kvm-specific projects? Can they be part of the QEMU project
> >>>    or do we need a different mentoring organization for it?
> >>>
> >>
> >>Something really interesting is kvm-assisted tcg.  I'm afraid it's a bit
> >>too complicated to GSoC.
> >
> >Is this simpler: kvm-assisted user-mode emulation (no TCG involved)?
> 
> Or running TCG-compiled code as a guest so that for example KVM
> takes care of the softmmu (with hardware-assisted paging possibly,
> yoohoo). But I have no idea if that is what Avi meant. :-)
> 
I think it is.

--
			Gleb.

[Qemu-devel] Re: Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>     or do we need a different mentoring organization for it?
>    

Complete big real mode emulation.  I'll add this.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Lucas Meneghel Rodrigues]

On Wed, 2010-03-10 at 18:30 -0300, Luiz Capitulino wrote:
> Hi there,
> 
>  Our wiki page for the Summer of Code 2010 is doing quite well:
> 
> http://wiki.qemu.org/Google_Summer_of_Code_2010

Just to let you guys know that I'm going to give a talk at the local
university (Unicamp) about kvm autotest, and will spread the word about
the qemu and kvm summer of code applications, will incentivate the
students to apply for qemu and kvm.

The university was the 2nd overall place on number of student proposals
accepted on gsoc for the last couple of years, with an excellent
completion rate, so I believe we could have some good work coming out of
it.

>  Now the most important is:
> 
> 1. Get mentors assigned to projects. Just put your name and email in the
>    right field. It's ok and even desirable to have two mentors per project,
>    but please remember that mentoring is serious work, more info here:
> 
>    http://code.google.com/p/google-summer-of-code/wiki/AdviceforMentors
> 
>    http://gsoc-wiki.osuosl.org/index.php/Main_Page
> 
> 2. Do we have kvm-specific projects? Can they be part of the QEMU project
>    or do we need a different mentoring organization for it?
> 
> 3. Fill in the missing information for the suggested project (description,
>    skill level, languages, etc)
> 
>  I will complete our application tomorrow or on Friday.
> 
> PS: I'm CC'ing everyone who suggested projects there, except one or two
> I couldn't find the email address.
> 
> 
> 

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Adam Litke]

Hi Luiz.  Around the time when I introduced the new Asynchronous monitor
command API we had talked about converting all commands to use this new
API so that we can cut down on duplicate code paths and confusing code.
I would like to propose this as a GSoC project idea.  Do you think it
should stand as its own project or should we merge it into your "Convert
Monitor commands to the QObject API" project?

-- 
Thanks,
Adam

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Fri, 12 Mar 2010 09:11:31 -0600
Adam Litke <agl@us.ibm.com> wrote:

> Hi Luiz.  Around the time when I introduced the new Asynchronous monitor
> command API we had talked about converting all commands to use this new
> API so that we can cut down on duplicate code paths and confusing code.
> I would like to propose this as a GSoC project idea.  Do you think it
> should stand as its own project or should we merge it into your "Convert
> Monitor commands to the QObject API" project?

 I think it's a project by itself, but I wonder if it's too easy/short
for GSoC. An experienced programmer can do the conversion plus testing
in a day or two. There are probably a number of cleanups and adaptions
that can take more, but still seems too short.

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Adam Litke]

On Fri, 2010-03-12 at 12:22 -0300, Luiz Capitulino wrote:
> On Fri, 12 Mar 2010 09:11:31 -0600
> Adam Litke <agl@us.ibm.com> wrote:
> 
> > Hi Luiz.  Around the time when I introduced the new Asynchronous monitor
> > command API we had talked about converting all commands to use this new
> > API so that we can cut down on duplicate code paths and confusing code.
> > I would like to propose this as a GSoC project idea.  Do you think it
> > should stand as its own project or should we merge it into your "Convert
> > Monitor commands to the QObject API" project?
> 
>  I think it's a project by itself, but I wonder if it's too easy/short
> for GSoC. An experienced programmer can do the conversion plus testing
> in a day or two. There are probably a number of cleanups and adaptions
> that can take more, but still seems too short.

So given the relatively small scope of this additional work, maybe it
should be an additional "stretch" goal to be added to your project.
Once the student(s) have gone through the trouble to familiarize
themselves with the monitor code, they would be well-positioned to
complete this extra bit.  How difficult do you imagine it will be to
convert the remaining commands over to QObject?

-- 
Thanks,
Adam

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Fri, 12 Mar 2010 09:38:47 -0600
Adam Litke <agl@us.ibm.com> wrote:

> On Fri, 2010-03-12 at 12:22 -0300, Luiz Capitulino wrote:
> > On Fri, 12 Mar 2010 09:11:31 -0600
> > Adam Litke <agl@us.ibm.com> wrote:
> > 
> > > Hi Luiz.  Around the time when I introduced the new Asynchronous monitor
> > > command API we had talked about converting all commands to use this new
> > > API so that we can cut down on duplicate code paths and confusing code.
> > > I would like to propose this as a GSoC project idea.  Do you think it
> > > should stand as its own project or should we merge it into your "Convert
> > > Monitor commands to the QObject API" project?
> > 
> >  I think it's a project by itself, but I wonder if it's too easy/short
> > for GSoC. An experienced programmer can do the conversion plus testing
> > in a day or two. There are probably a number of cleanups and adaptions
> > that can take more, but still seems too short.
> 
> So given the relatively small scope of this additional work, maybe it
> should be an additional "stretch" goal to be added to your project.

 It could be, our we could try to come up with something additional.

> Once the student(s) have gone through the trouble to familiarize
> themselves with the monitor code, they would be well-positioned to
> complete this extra bit.  How difficult do you imagine it will be to
> convert the remaining commands over to QObject?

 Well, I won't set the goal to convert all of them, because seems
too much work and Anthony has said that we may not want all the
handlers available under QMP. So, this has to be discussed (preferably
before GSoC starts for students).

 Also, I have two other projects that could related to the async
conversion:

- Simplify/Improve the QObject API
- Improve error handling (QError conversion involved)

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Anthony Liguori]

On 03/12/2010 09:56 AM, Luiz Capitulino wrote:
>> Once the student(s) have gone through the trouble to familiarize
>> themselves with the monitor code, they would be well-positioned to
>> complete this extra bit.  How difficult do you imagine it will be to
>> convert the remaining commands over to QObject?
>>      
>   Well, I won't set the goal to convert all of them, because seems
> too much work and Anthony has said that we may not want all the
> handlers available under QMP. So, this has to be discussed (preferably
> before GSoC starts for students).
>    

What I would like to see is a clean break between the human monitor and 
QMP whereas the human monitor is implemented in terms of QMP.

For instance, the x and xp commands are not very useful for QMP.  
However, a generic memory read/write API would be pretty useful.  The 
x/xp commands would be implemented in terms of the memory QMP API.

Likewise, the sum command can be implemented in terms of the above API.

Regards,

Anthony Liguori

>   Also, I have two other projects that could related to the async
> conversion:
>
> - Simplify/Improve the QObject API
> - Improve error handling (QError conversion involved)
>    

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Luiz Capitulino]

On Fri, 12 Mar 2010 10:36:53 -0600
Anthony Liguori <aliguori@linux.vnet.ibm.com> wrote:

> On 03/12/2010 09:56 AM, Luiz Capitulino wrote:
> >> Once the student(s) have gone through the trouble to familiarize
> >> themselves with the monitor code, they would be well-positioned to
> >> complete this extra bit.  How difficult do you imagine it will be to
> >> convert the remaining commands over to QObject?
> >>      
> >   Well, I won't set the goal to convert all of them, because seems
> > too much work and Anthony has said that we may not want all the
> > handlers available under QMP. So, this has to be discussed (preferably
> > before GSoC starts for students).
> >    
> 
> What I would like to see is a clean break between the human monitor and 
> QMP whereas the human monitor is implemented in terms of QMP.
> 
> For instance, the x and xp commands are not very useful for QMP.  
> However, a generic memory read/write API would be pretty useful.  The 
> x/xp commands would be implemented in terms of the memory QMP API.
> 
> Likewise, the sum command can be implemented in terms of the above API.

 Right, makes sense.

 So, we need to convert them all but maybe not make all of them
available under QMP.

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>   Hi there,
>
>   Our wiki page for the Summer of Code 2010 is doing quite well:
>
> http://wiki.qemu.org/Google_Summer_of_Code_2010
>    

I will add another project - iommu emulation.  Could be very useful for 
doing device assignment to nested guests, which could make testing a lot 
easier.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Joerg Roedel]

On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>   Hi there,
>>
>>   Our wiki page for the Summer of Code 2010 is doing quite well:
>>
>> http://wiki.qemu.org/Google_Summer_of_Code_2010
>>    
>
> I will add another project - iommu emulation.  Could be very useful for  
> doing device assignment to nested guests, which could make testing a lot  
> easier.

Good idea. If there is interest I could help to mentor this project.

	Joerg

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/15/2010 02:38 PM, Joerg Roedel wrote:
> On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
>    
>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>      
>>>    Hi there,
>>>
>>>    Our wiki page for the Summer of Code 2010 is doing quite well:
>>>
>>> http://wiki.qemu.org/Google_Summer_of_Code_2010
>>>
>>>        
>> I will add another project - iommu emulation.  Could be very useful for
>> doing device assignment to nested guests, which could make testing a lot
>> easier.
>>      
> Good idea. If there is interest I could help to mentor this project.
>    

Thanks.  I volunteered Anthony, but he may be a little overcommitted.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Anthony Liguori]

On 03/15/2010 07:42 AM, Avi Kivity wrote:
> On 03/15/2010 02:38 PM, Joerg Roedel wrote:
>> On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
>>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>>>    Hi there,
>>>>
>>>>    Our wiki page for the Summer of Code 2010 is doing quite well:
>>>>
>>>> http://wiki.qemu.org/Google_Summer_of_Code_2010
>>>>
>>> I will add another project - iommu emulation.  Could be very useful for
>>> doing device assignment to nested guests, which could make testing a 
>>> lot
>>> easier.
>> Good idea. If there is interest I could help to mentor this project.
>
> Thanks.  I volunteered Anthony, but he may be a little overcommitted.

Joerg, feel free to put your name against too.

Regards,

Anthony Liguori

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Joerg Roedel]

On Mon, Mar 15, 2010 at 08:14:29AM -0500, Anthony Liguori wrote:
> On 03/15/2010 07:42 AM, Avi Kivity wrote:
>> On 03/15/2010 02:38 PM, Joerg Roedel wrote:
>>> On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
>>>> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
>>>>>    Hi there,
>>>>>
>>>>>    Our wiki page for the Summer of Code 2010 is doing quite well:
>>>>>
>>>>> http://wiki.qemu.org/Google_Summer_of_Code_2010
>>>>>
>>>> I will add another project - iommu emulation.  Could be very useful for
>>>> doing device assignment to nested guests, which could make testing 
>>>> a lot
>>>> easier.
>>> Good idea. If there is interest I could help to mentor this project.
>>
>> Thanks.  I volunteered Anthony, but he may be a little overcommitted.
>
> Joerg, feel free to put your name against too.

[x] Done.

Thanks,
	Joerg

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Muli Ben-Yehuda]

On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
> On 03/10/2010 11:30 PM, Luiz Capitulino wrote:

> >  Hi there,
> >
> >  Our wiki page for the Summer of Code 2010 is doing quite well:
> >
> >http://wiki.qemu.org/Google_Summer_of_Code_2010
> 
> I will add another project - iommu emulation.  Could be very useful
> for doing device assignment to nested guests, which could make
> testing a lot easier.

Our experiments show that nested device assignment is pretty much
required for I/O performance in nested scenarios.

Cheers,
Muli

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Joerg Roedel]

On Mon, Mar 15, 2010 at 05:53:13AM -0700, Muli Ben-Yehuda wrote:
> On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:
> > On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> 
> > >  Hi there,
> > >
> > >  Our wiki page for the Summer of Code 2010 is doing quite well:
> > >
> > >http://wiki.qemu.org/Google_Summer_of_Code_2010
> > 
> > I will add another project - iommu emulation.  Could be very useful
> > for doing device assignment to nested guests, which could make
> > testing a lot easier.
> 
> Our experiments show that nested device assignment is pretty much
> required for I/O performance in nested scenarios.

Really? I did a small test with virtio-blk in a nested guest (disk read
with dd, so not a real benchmark) and got a reasonable read-performance
of around 25MB/s from the disk in the l2-guest.

	Joerg

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>
>>> I will add another project - iommu emulation.  Could be very useful
>>> for doing device assignment to nested guests, which could make
>>> testing a lot easier.
>>>        
>> Our experiments show that nested device assignment is pretty much
>> required for I/O performance in nested scenarios.
>>      
> Really? I did a small test with virtio-blk in a nested guest (disk read
> with dd, so not a real benchmark) and got a reasonable read-performance
> of around 25MB/s from the disk in the l2-guest.
>
>    

Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.

I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do 
for other guests.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Anthony Liguori]

On 03/15/2010 08:11 AM, Avi Kivity wrote:
> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>
>>>> I will add another project - iommu emulation.  Could be very useful
>>>> for doing device assignment to nested guests, which could make
>>>> testing a lot easier.
>>> Our experiments show that nested device assignment is pretty much
>>> required for I/O performance in nested scenarios.
>> Really? I did a small test with virtio-blk in a nested guest (disk read
>> with dd, so not a real benchmark) and got a reasonable read-performance
>> of around 25MB/s from the disk in the l2-guest.
>>
>
> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>
> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can 
> do for other guests.

VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to 
be costly.  KVM is a bit unusual in terms of how many times the 
instructions are executed per exit.

Regards,

Anthony Liguori

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/15/2010 03:23 PM, Anthony Liguori wrote:
> On 03/15/2010 08:11 AM, Avi Kivity wrote:
>> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>>
>>>>> I will add another project - iommu emulation.  Could be very useful
>>>>> for doing device assignment to nested guests, which could make
>>>>> testing a lot easier.
>>>> Our experiments show that nested device assignment is pretty much
>>>> required for I/O performance in nested scenarios.
>>> Really? I did a small test with virtio-blk in a nested guest (disk read
>>> with dd, so not a real benchmark) and got a reasonable read-performance
>>> of around 25MB/s from the disk in the l2-guest.
>>>
>>
>> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>>
>> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can 
>> do for other guests.
>
> VMREAD/VMWRITEs are generally optimized by hypervisors as they tend to 
> be costly.  KVM is a bit unusual in terms of how many times the 
> instructions are executed per exit.

Do you know offhand of any unnecessary read/writes?  There's 
update_cr8_intercept(), but on normal exits, I don't see what else we 
can remove.

-- 
error compiling committee.c: too many arguments to function

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Anthony Liguori]

On 03/15/2010 10:06 AM, Avi Kivity wrote:
> On 03/15/2010 03:23 PM, Anthony Liguori wrote:
>> On 03/15/2010 08:11 AM, Avi Kivity wrote:
>>> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>>>
>>>>>> I will add another project - iommu emulation.  Could be very useful
>>>>>> for doing device assignment to nested guests, which could make
>>>>>> testing a lot easier.
>>>>> Our experiments show that nested device assignment is pretty much
>>>>> required for I/O performance in nested scenarios.
>>>> Really? I did a small test with virtio-blk in a nested guest (disk 
>>>> read
>>>> with dd, so not a real benchmark) and got a reasonable 
>>>> read-performance
>>>> of around 25MB/s from the disk in the l2-guest.
>>>>
>>>
>>> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>>>
>>> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we 
>>> can do for other guests.
>>
>> VMREAD/VMWRITEs are generally optimized by hypervisors as they tend 
>> to be costly.  KVM is a bit unusual in terms of how many times the 
>> instructions are executed per exit.
>
> Do you know offhand of any unnecessary read/writes?  There's 
> update_cr8_intercept(), but on normal exits, I don't see what else we 
> can remove.

Yeah, there are a number of examples.

vmcs_clear_bits() and vmcs_set_bits() read a field of the VMCS and then 
immediately writes it.  This is unnecessary as the same information 
could be kept in a shadow variable.  In vmx_fpu_activate, we call 
vmcs_clear_bits() followed immediately by vmcs_set_bits(). which means 
we're reading GUEST_CR0 twice and writing it twice.

vmx_get_rflags() reads from the VMCS and we frequently call get_rflags() 
followed by a set_rflags() to update a bit.  We also don't cache the 
value between calls and there's a few spots in the code that make 
multiple calls.

Regards,

Anthony Liguori

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Avi Kivity]

On 03/16/2010 03:21 AM, Anthony Liguori wrote:
> On 03/15/2010 10:06 AM, Avi Kivity wrote:
>> On 03/15/2010 03:23 PM, Anthony Liguori wrote:
>>> On 03/15/2010 08:11 AM, Avi Kivity wrote:
>>>> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>>>>
>>>>>>> I will add another project - iommu emulation.  Could be very useful
>>>>>>> for doing device assignment to nested guests, which could make
>>>>>>> testing a lot easier.
>>>>>> Our experiments show that nested device assignment is pretty much
>>>>>> required for I/O performance in nested scenarios.
>>>>> Really? I did a small test with virtio-blk in a nested guest (disk 
>>>>> read
>>>>> with dd, so not a real benchmark) and got a reasonable 
>>>>> read-performance
>>>>> of around 25MB/s from the disk in the l2-guest.
>>>>>
>>>>
>>>> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>>>>
>>>> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we 
>>>> can do for other guests.
>>>
>>> VMREAD/VMWRITEs are generally optimized by hypervisors as they tend 
>>> to be costly.  KVM is a bit unusual in terms of how many times the 
>>> instructions are executed per exit.
>>
>> Do you know offhand of any unnecessary read/writes?  There's 
>> update_cr8_intercept(), but on normal exits, I don't see what else we 
>> can remove.
>
> Yeah, there are a number of examples.
>
> vmcs_clear_bits() and vmcs_set_bits() read a field of the VMCS and 
> then immediately writes it.  This is unnecessary as the same 
> information could be kept in a shadow variable.  In vmx_fpu_activate, 
> we call vmcs_clear_bits() followed immediately by vmcs_set_bits(). 
> which means we're reading GUEST_CR0 twice and writing it twice.

This should be much better these days (2.6.34-rc1) as vmx_fpu_activate() 
is called at most once per heavyweight exit (and I have evil plans to 
reduce it even further).  Still, that code should be optimized.

> vmx_get_rflags() reads from the VMCS and we frequently call 
> get_rflags() followed by a set_rflags() to update a bit.  We also 
> don't cache the value between calls and there's a few spots in the 
> code that make multiple calls.

We definitely should cache that (and segment access from the emulator as 
well).  But I'd have thought this to be relatively infrequent.  At least 
with Linux, using x2apic and virtio allows you to eliminate most 
emulator access, if you have npt or ept.

-- 
Do not meddle in the internals of kernels, for they are subtle and quick to panic.

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Joerg Roedel]

On Mon, Mar 15, 2010 at 03:11:42PM +0200, Avi Kivity wrote:
> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>
>>>> I will add another project - iommu emulation.  Could be very useful
>>>> for doing device assignment to nested guests, which could make
>>>> testing a lot easier.
>>>>        
>>> Our experiments show that nested device assignment is pretty much
>>> required for I/O performance in nested scenarios.
>>>      
>> Really? I did a small test with virtio-blk in a nested guest (disk read
>> with dd, so not a real benchmark) and got a reasonable read-performance
>> of around 25MB/s from the disk in the l2-guest.
>>
>>    
>
> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>
> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do  
> for other guests.

Does it matter for the ept-on-ept case? The initial patchset of
nested-vmx implemented it and they reported a performance drop of around
12% between levels which is reasonable. So I expected the loss of
io-performance for l2 also reasonable in this case. My small measurement
was also done using npt-on-npt.

	Joerg

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Anthony Liguori]

On 03/15/2010 08:24 AM, Joerg Roedel wrote:
> On Mon, Mar 15, 2010 at 03:11:42PM +0200, Avi Kivity wrote:
>    
>> On 03/15/2010 03:03 PM, Joerg Roedel wrote:
>>      
>>>        
>>>>> I will add another project - iommu emulation.  Could be very useful
>>>>> for doing device assignment to nested guests, which could make
>>>>> testing a lot easier.
>>>>>
>>>>>            
>>>> Our experiments show that nested device assignment is pretty much
>>>> required for I/O performance in nested scenarios.
>>>>
>>>>          
>>> Really? I did a small test with virtio-blk in a nested guest (disk read
>>> with dd, so not a real benchmark) and got a reasonable read-performance
>>> of around 25MB/s from the disk in the l2-guest.
>>>
>>>
>>>        
>> Your guest wasn't doing a zillion VMREADs and VMWRITEs every exit.
>>
>> I plan to reduce VMREAD/VMWRITE overhead for kvm, but not much we can do
>> for other guests.
>>      
> Does it matter for the ept-on-ept case? The initial patchset of
> nested-vmx implemented it and they reported a performance drop of around
> 12% between levels which is reasonable. So I expected the loss of
> io-performance for l2 also reasonable in this case. My small measurement
> was also done using npt-on-npt.
>    

But that was something like kernbench IIRC which is actually exit light 
once ept is enabled.

Network IO is typically exit heavy and becomes something more of a 
pathological work load (both for nested ept and nested npt).

Regards,

Anthony Liguori

> 	Joerg
>
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>    

Re: [Qemu-devel] Ideas wiki for GSoC 2010

[Muli Ben-Yehuda]

On Mon, Mar 15, 2010 at 02:03:11PM +0100, Joerg Roedel wrote:

> On Mon, Mar 15, 2010 at 05:53:13AM -0700, Muli Ben-Yehuda wrote:

> > On Mon, Mar 15, 2010 at 02:25:41PM +0200, Avi Kivity wrote:

> > > On 03/10/2010 11:30 PM, Luiz Capitulino wrote:
> > 
> > > >  Hi there,
> > > >
> > > >  Our wiki page for the Summer of Code 2010 is doing quite well:
> > > >
> > > >http://wiki.qemu.org/Google_Summer_of_Code_2010
> > > 
> > > I will add another project - iommu emulation.  Could be very
> > > useful for doing device assignment to nested guests, which could
> > > make testing a lot easier.
> > 
> > Our experiments show that nested device assignment is pretty much
> > required for I/O performance in nested scenarios.
> 
> Really? I did a small test with virtio-blk in a nested guest (disk
> read with dd, so not a real benchmark) and got a reasonable
> read-performance of around 25MB/s from the disk in the l2-guest.

Netperf running in L1 with direct access: ~950 Mbps throughput with
25% CPU utilization. Netperf running in L2 with virtio between L2 and
L1 and direct assignment between L1 and L0: roughly the same
throughput, but over 90% CPU utilization! Now extrapolate to 10GbE.

Cheers,
Muli