Re: Hi,something about the xentrace tool

[Rob Gardner <rob.gardner@hp.com>]

NAHieu wrote:
> On 6/14/06, Rob Gardner <rob.gardner@hp.com> wrote:
>> rickey berkeley wrote:
>> > Based on trace source code(xen/common/trace.c),dom0 tracing the event
>> > which are enabled.
>> >
>> > xen initialize tracing buffer for each cpu,trace buffer size (in
>> > pages) in kenel space is defined by opt_tbuf_size.
>> > The tracing buf use loop structure.
>> >
>> > I guess when the tracing data increase rapidly ,and xentrace tool in
>> > user space does not take them immediately.
>> > Some tracing data will lost.Maybe relayfs or something else can solve
>> > this problem.
>> >
>>
>> I added a basic flow control mechanism to the trace buffer system a few
>> months ago. You can see an example of how to use it in tools/xenmon. The
>> way it works is that as trace records are generated, a software
>> interrupt is generated when the trace buffer gets filled to a certain
>> point. The user space tools can use select() on an event channel to find
>> out about these interrupts. See tools/xenmon/xenbaked.c for exact
>> programming details. This code has not been copied into xentrace yet;
>> Feel free to do so yourself if you think it's necessary.
>
> Would you explain a little bit more on how you handle the overflow? If
> the userspace detects that event, what it does then?

If overflow occurs, it is not handled. The mechanism I implemented was 
just designed to drastically reduce the probability of overflow.

When xenbaked wakes up from its select() call (indicating the trace 
buffer high water mark has been reached) it simply starts reading trace 
records out of the trace buffers.

>
> Usually if there is no more space available in kernel buffer, what
> will you do? Drop some data, and hope that the userspace will come up
> to free some for more space in the future? Or if you don't drop them,
> just block there and wait for the userspace to come to free some?

Currently, the trace buffer "high water" mark is set to 50%. That is, 
when the hypervisor trace buffer becomes 1/2 full, it sends a soft 
interrupt to wake up xenbaked from its blocking select(). If nobody 
wakes up to read trace records from the trace buffer, I take that to 
mean that nobody cares about the trace records. When somebody does care, 
they will read those records in a timely manner. Obviously, the 
hypervisor cannot "block" if there is no room in the trace buffers; In 
this case, new trace records simply overwrite old ones, and the old ones 
are lost.

If you encounter a situation where trace records are being generated too 
fast, and fill up the trace buffer too quickly, then the simple next 
step is to increase the size of the trace buffers. So far, use of the 
trace records has not been linked to anything so critical that it's 
necessary to take extraordinary measures to avoid loss of data.

Rob