Re: Enhance perf to support KVM

Avi Kivity
2010-02-26T10:50:34Z

On 02/26/2010 12:35 PM, Ingo Molnar wrote:
>
>>> One additional step needed is to get symbol information from the guest, and to
>>> integrate it into the symbol cache on the host side in ~/.debug. We already
>>> support cross-arch symbols and 'perf archive', so the basic facilities are
>>> there for that. So you can profile on 32-bit PA-RISC and type 'perf report' on
>>> 64-bit x86 and get all the right info.
>>>
>>> For this to work across a guest, a gateway is needed towards the guest.
>>> There's several ways to achieve this. The most practical would be two steps:
>>>
>>> - a user-space facility to access guest images/libraries. (say via ssh, or
>>> just a plain TCP port) This would be useful for general 'remote profiling'
>>> sessions as well, so it's not KVM specific - it would be useful for remote
>>> debugging.
>>>
>>> - The guest /proc/kallsyms (and vmlinux) could be accessed via that channel
>>> as well.
>>>
>>> (Note that this is purely for guest symbol space access - all the profiling
>>> data itself comes via the host kernel.)
>>>
>>> In theory we could build some sort of 'symbol server' facility into the
>>> kernel, which could be enabled in guest kernels too - but i suspect existing,
>>> user-space transports go most of the way already.
>>>
>> There is also vmchannel aka virtio-serial, a guest-to-host communication
>> channel.
>>
> Basically what is needed is plain filesystem access - properly privileged. So
> doing this via a vmchannel would be nice, but for the symbol extraction it
> would be a glorified NFS server in essence.
>

Well, we could run an nfs server over vmchannel, or over a private
network interface.

> Do you have (or plan) any turn-key 'access to all files of the guest' kind of
> guest-transparent facility that could be used for such purposes?

Not really. The guest and host admins are usually different people, who
may, being admins, even actively hate each other. The guest admin would
probably regard it as a security hole. It's probably useful for the
single-host scenario, and of course for developers.

I guess sshfs can fill this role, with one command it gives you secure
access to all guest files, provided you have the proper credentials.

> That would
> have various advantages over a traditional explicit file server approach:
>
> - it would not contaminate the guest port space
>
> - no guest side configuration needed (the various oprofile remote daemons
> always sucked as they needed extra setup)
>
> - it might even be used with a guest that does no networking
>
> - if done fully in the kernel it could be done with a fully 'unaware' guest, etc.
>

Seems sshfs fulfils the first two. For the latter, we could do a
vmchannelfs, but it seems quite a bit of work, and would require fairly
new guest kernels, whereas sshfs would work out of the box on 10 year
old guests and can be easily made to work on Windows.

Somewhat related, see libguestfs/guestfish, though that provides offline
access only.

--
Do not meddle in the internals of kernels, for they are subtle and quick to panic.