[PATCH RFC 0/4]Porting Xentrace to kvm

[PATCH RFC 0/4]Porting Xentrace to kvm

[Liu, Eric E]

Hi,
The following patches port xentrace to kvm which is useful for
performance tuning and debugging.

It is designed to allow debugging traces of kvm to be generated on
Up/Smp machines. Each trace entry is outputted in a trace ring buffer
for per cpu which is mapped to userspace, and the userspace tools can
analyze the data according to some formats definitions.

Since we already have had debugfs_entries and some other kernel debug
mechanism to use, does this kvmtrace make sense? Any comment is
welcomed. 

--Eric (Liu, Feng)

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH RFC 0/4]Porting Xentrace to kvm

[Avi Kivity]

Liu, Eric E wrote:
> Hi,
> The following patches port xentrace to kvm which is useful for
> performance tuning and debugging.
>
> It is designed to allow debugging traces of kvm to be generated on
> Up/Smp machines. Each trace entry is outputted in a trace ring buffer
> for per cpu which is mapped to userspace, and the userspace tools can
> analyze the data according to some formats definitions.
>
> Since we already have had debugfs_entries and some other kernel debug
> mechanism to use, does this kvmtrace make sense? Any comment is
> welcomed. 
>   

This looks very useful.  While kvm_stat provides good data, this is much 
more in depth.

The kernel already contains a method of transferring percpu data to 
userspace; see Documentation/filesystems/relay.txt.  It supports mmap() 
and read().  Please see if it is a good fit.

Please post patches with individual subjects, instead of having the same 
subject for every patch.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH RFC 0/4]Porting Xentrace to kvm

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 1170 bytes --]

Avi Kivity wrote:
> Liu, Eric E wrote:
>> Hi,
>> The following patches port xentrace to kvm which is useful for
>> performance tuning and debugging.
>>
>> It is designed to allow debugging traces of kvm to be generated on
>> Up/Smp machines. Each trace entry is outputted in a trace ring buffer
>> for per cpu which is mapped to userspace, and the userspace tools can
>> analyze the data according to some formats definitions.
>>
>> Since we already have had debugfs_entries and some other kernel debug
>> mechanism to use, does this kvmtrace make sense? Any comment is
>> welcomed. 
>>   
> 
> This looks very useful.  While kvm_stat provides good data, this is much 
> more in depth.
> 
> The kernel already contains a method of transferring percpu data to 
> userspace; see Documentation/filesystems/relay.txt.  It supports mmap() 
> and read().  Please see if it is a good fit.

I would also suggest to use the new kernel standard for instrumentation:
trace_mark().

This would also allow to reuse the trace points with other tracer and
maybe even obsolete a separate transportation channel, e.g. when LTTng
is once :-/ merged.

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 228 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [PATCH RFC 0/4]Porting Xentrace to kvm

[Avi Kivity]

Jan Kiszka wrote:
> Avi Kivity wrote:
>   
>> Liu, Eric E wrote:
>>     
>>> Hi,
>>> The following patches port xentrace to kvm which is useful for
>>> performance tuning and debugging.
>>>
>>> It is designed to allow debugging traces of kvm to be generated on
>>> Up/Smp machines. Each trace entry is outputted in a trace ring buffer
>>> for per cpu which is mapped to userspace, and the userspace tools can
>>> analyze the data according to some formats definitions.
>>>
>>> Since we already have had debugfs_entries and some other kernel debug
>>> mechanism to use, does this kvmtrace make sense? Any comment is
>>> welcomed. 
>>>   
>>>       
>> This looks very useful.  While kvm_stat provides good data, this is much 
>> more in depth.
>>
>> The kernel already contains a method of transferring percpu data to 
>> userspace; see Documentation/filesystems/relay.txt.  It supports mmap() 
>> and read().  Please see if it is a good fit.
>>     
>
> I would also suggest to use the new kernel standard for instrumentation:
> trace_mark().
>
> This would also allow to reuse the trace points with other tracer and
> maybe even obsolete a separate transportation channel, e.g. when LTTng
> is once :-/ merged.
>   

Can one have markers automatically recorded?  Or do you need to connect 
the marker with a logging function?

If the latter, it's too difficult to use.

-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

Re: [PATCH RFC 0/4]Porting Xentrace to kvm

[Jan Kiszka]

[-- Attachment #1.1: Type: text/plain, Size: 2027 bytes --]

Avi Kivity wrote:
> Jan Kiszka wrote:
>> Avi Kivity wrote:
>>  
>>> Liu, Eric E wrote:
>>>    
>>>> Hi,
>>>> The following patches port xentrace to kvm which is useful for
>>>> performance tuning and debugging.
>>>>
>>>> It is designed to allow debugging traces of kvm to be generated on
>>>> Up/Smp machines. Each trace entry is outputted in a trace ring buffer
>>>> for per cpu which is mapped to userspace, and the userspace tools can
>>>> analyze the data according to some formats definitions.
>>>>
>>>> Since we already have had debugfs_entries and some other kernel debug
>>>> mechanism to use, does this kvmtrace make sense? Any comment is
>>>> welcomed.         
>>> This looks very useful.  While kvm_stat provides good data, this is
>>> much more in depth.
>>>
>>> The kernel already contains a method of transferring percpu data to
>>> userspace; see Documentation/filesystems/relay.txt.  It supports
>>> mmap() and read().  Please see if it is a good fit.
>>>     
>>
>> I would also suggest to use the new kernel standard for instrumentation:
>> trace_mark().
>>
>> This would also allow to reuse the trace points with other tracer and
>> maybe even obsolete a separate transportation channel, e.g. when LTTng
>> is once :-/ merged.
>>   
> 
> Can one have markers automatically recorded?  Or do you need to connect
> the marker with a logging function?

Yes, of course you need a probe function. Either your own one, or one
provided by other tracers.

> 
> If the latter, it's too difficult to use.

I don't think it is. Check linux/samples/markers/probe-example.c. BTW,
RCU-Trace (not yet mainlined) make use of markers as well, see -rt.

And in contrast to the suggested implementation, markers have less
impact on the code if built-in but disabled. So there would be no
reasons for not shipping kvm production binaries that are prepared to be
traced.

But if you prefer the inlined version, I bet you'll have to hide it from
reviewers at LKML. ;)

Jan


[-- Attachment #1.2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 254 bytes --]

[-- Attachment #2: Type: text/plain, Size: 228 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/

[-- Attachment #3: Type: text/plain, Size: 158 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: [PATCH RFC 0/4]Porting Xentrace to kvm

[Liu, Eric E]

jan.kiszka@web.de wrote:
> Avi Kivity wrote:
>> Jan Kiszka wrote:
>>> Avi Kivity wrote:
>>> 
>>>> Liu, Eric E wrote:
>>>> 
>>>>> 
>>>>> Since we already have had debugfs_entries and some other kernel
>>>>> debug mechanism to use, does this kvmtrace make sense? Any
>>>>> comment is welcomed.
>>>> This looks very useful.  While kvm_stat provides good data, this
>>>> is much more in depth. 
>>>> 
>>>> The kernel already contains a method of transferring percpu data to
>>>> userspace; see Documentation/filesystems/relay.txt.  It supports
>>>> mmap() and read().  Please see if it is a good fit.
>>>> 
>>> 
>>> I would also suggest to use the new kernel standard for
>>> instrumentation: trace_mark(). 
>>> 
>>> This would also allow to reuse the trace points with other tracer
>>> and maybe even obsolete a separate transportation channel, e.g.
>>> when LTTng is once :-/ merged. 
>>> 
>> 
>> Can one have markers automatically recorded?  Or do you need to
>> connect the marker with a logging function?
> 
> Yes, of course you need a probe function. Either your own one, or one
> provided by other tracers.
> 
>> 
>> If the latter, it's too difficult to use.
> 
> I don't think it is. Check linux/samples/markers/probe-example.c. BTW,
> RCU-Trace (not yet mainlined) make use of markers as well, see -rt.
> 
> And in contrast to the suggested implementation, markers have less
> impact on the code if built-in but disabled. So there would be no
> reasons for not shipping kvm production binaries that are prepared to
> be traced.
> 
> But if you prefer the inlined version, I bet you'll have to hide it
> from reviewers at LKML. ;)
> 
> Jan

Thanks for your comments, and I think that using relayfs and
trace_mark() is more reasonable, if it makes sense, I will modify the
patches according to these. 



-------------------------------------------------------------------------
This SF.net email is sponsored by: Microsoft
Defy all challenges. Microsoft(R) Visual Studio 2008.
http://clk.atdmt.com/MRT/go/vse0120000070mrt/direct/01/