From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from lists.lttng.org (lists.lttng.org [167.114.26.123]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 6A9A1C433EF for ; Tue, 7 Dec 2021 20:08:11 +0000 (UTC) Received: from lists-lttng01.efficios.com (localhost [IPv6:::1]) by lists.lttng.org (Postfix) with ESMTP id 4J7rvY3f09z1rWS; Tue, 7 Dec 2021 15:08:09 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=lists.lttng.org; s=default; t=1638907690; bh=J47RgJ1D9Zq6jezeKTW4gqnVSR02QjhtY7hLd7DgssU=; h=Date:To:Cc:In-Reply-To:References:Subject:List-Id: List-Unsubscribe:List-Archive:List-Post:List-Help:List-Subscribe: From:Reply-To:From; b=F7MAVsOG3vLAJY7QJGENYOeQA55DHgqY5bDNIj81lTCVxSE2rWrda+BMjvT831IKT 9trxf8ffX5FR2mjI2B1V4rK4Sv7wGth4pIGlV29Iuj2jDWFJvkrJ/eZf7fFMo6qUvp 2jRWNlSuyDA3jpYuSS4OpWdNklKvGYXvcwxaORdPFqmm4vJEfhE7F5FQtkRoc6bR7u wBNLgYfoPq4lhJK49R+GpS0Fx8jVCbdtu3Y6L0EioSKftun1nzhboKjPLmdI9BKtkf S2GVzLRMzj6e3cDU77UGwq5f/1YTA3a1CztiTX0oR1Etp8DIXX9Ss1eqiHq4RtJkp+ 85D/FnfdQNy7w== Received: from mail.efficios.com (mail.efficios.com [167.114.26.124]) by lists.lttng.org (Postfix) with ESMTPS id 4J7rvW0Bt4z1rWQ for ; Tue, 7 Dec 2021 15:08:06 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id B976339FC17 for ; Tue, 7 Dec 2021 15:08:00 -0500 (EST) Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id MOBefnBbORru; Tue, 7 Dec 2021 15:07:59 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mail.efficios.com (Postfix) with ESMTP id D638B39FC11; Tue, 7 Dec 2021 15:07:59 -0500 (EST) DKIM-Filter: OpenDKIM Filter v2.10.3 mail.efficios.com D638B39FC11 X-Virus-Scanned: amavisd-new at efficios.com Received: from mail.efficios.com ([127.0.0.1]) by localhost (mail03.efficios.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id 9uD27mGLcjEY; Tue, 7 Dec 2021 15:07:59 -0500 (EST) Received: from mail03.efficios.com (mail03.efficios.com [167.114.26.124]) by mail.efficios.com (Postfix) with ESMTP id C620C39FC10; Tue, 7 Dec 2021 15:07:59 -0500 (EST) Date: Tue, 7 Dec 2021 15:07:59 -0500 (EST) To: "zhenyu.ren" Cc: lttng-dev Message-ID: <1503888133.17265.1638907679680.JavaMail.zimbra@efficios.com> In-Reply-To: <6ad73114-861c-46e3-8f9e-1076e482116f.zhenyu.ren@aliyun.com> References: <6ad73114-861c-46e3-8f9e-1076e482116f.zhenyu.ren@aliyun.com> MIME-Version: 1.0 X-Originating-IP: [167.114.26.124] X-Mailer: Zimbra 8.8.15_GA_4173 (ZimbraWebClient - FF94 (Linux)/8.8.15_GA_4177) Thread-Topic: lttng-consumerd can NOT get notification to consume ring buffer data Thread-Index: e3+roufjAnMbGS13hUMAXHwRr8O6Dw== Subject: Re: [lttng-dev] lttng-consumerd can NOT get notification to consume ring buffer data X-BeenThere: lttng-dev@lists.lttng.org X-Mailman-Version: 2.1.35 Precedence: list List-Id: LTTng development list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , From: Mathieu Desnoyers via lttng-dev Reply-To: Mathieu Desnoyers Content-Type: multipart/mixed; boundary="===============5366344613872067940==" Errors-To: lttng-dev-bounces@lists.lttng.org Sender: "lttng-dev" --===============5366344613872067940== Content-Type: multipart/alternative; boundary="=_fedc12fe-9489-44d5-81b6-2906f2ebede0" --=_fedc12fe-9489-44d5-81b6-2906f2ebede0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hi,=20 Can you try:=20 - Reproducing with a more recent LTTng-UST/LTTng-Tools ? (e.g. 2.13). LTTng= 2.7 is not supported anymore.=20 - Try to reproduce with "per-pid" UST buffers rather than "per-uid",=20 - Try to reproduce without the "tracefile rotation" mode (without --tracefi= le-count and --tracefile-size options to the channel).=20 Also, do you happen to have traced applications which are frequently killed= asynchronously=20 while some of their threads are actively tracing by any chance ?=20 Thanks,=20 Mathieu=20 ----- On Dec 7, 2021, at 4:03 AM, lttng-dev wro= te:=20 > Hi, Lttng-dev: > I found a strange problem related to lttng-consumed and ctf files recentl= y. The > ctf files belongs to some CPUs have been stopped rotating but other ctf > files(belong to other CPUs) keeped working as usual. I am very sure all C= PUs > were producing spans all the time. > #date; ls -ltrh channel0_0_* |tail -n 3; date;ls -ltrh channel0_1_* |tail= -n 3 > Tue Dec 7 16:25:45 CST 2021 > -rw-rw---- 1 root root 1.8M Dec 7 16:20 channel0_0_17 > -rw-rw---- 1 root root 2.0M Dec 7 16:23 channel0_0_18 > -rw-rw---- 1 root root 916K Dec 7 16:24 channel0_0_19 > Tue Dec 7 16:25:45 CST 2021 > -rw-rw---- 1 root root 1.7M Dec 6 00:30 channel0_1_8 > -rw-rw---- 1 root root 1.9M Dec 6 00:31 channel0_1_9 > -rw-rw---- 1 root root 388K Dec 6 00:32 channel0_1_10 > Notice that the ctf files with CPU0 (channel0_0_19) was modified at the t= ime > "now", but the ctf files with CPU1(channeo0_1_10) has been stopped workin= g at > Dec 6. > I gdb lttng-consumerd=EF=BC=88break at lib_ring_buffer_channel_do_read() = and > lib_ring_buffer_poll_deliver()--I configured a read timer on channels). S= ee the > followings ( lib_ring_buffer_poll_deliver() ) > 0x00007f0f2418a445 <+213>: mov 0x0(%r13),%rcx rcx:0x7f0f0402e610(handle->= table) > 0x00007f0f2418a449 <+217>: mov 0x98(%rsi),%r9 rsi:0x7f0efb86f000(buf) > r9:0x86c400000(consumed_old =3D buf->consumed) > 0x00007f0f2418a450 <+224>: lea 0x150(%rsi),%rdi rdi:0x7f0efda75150 > 0x00007f0f2418a457 <+231>: mov 0x150(%rsi),%rax rax:3(cpu?) > 0x00007f0f2418a45e <+238>: mov 0x78(%rbp),%rdx rbp:chan > rdx:0x1000000(chan->backend.buf_size) > 0x00007f0f2418a462 <+242>: mov 0x88(%rbp),%r8d > r8d:0x14(20)(chan->backend.buf_size_order) > 0x00007f0f2418a469 <+249>: cmp %rax,0x8(%rcx) > 0x00007f0f2418a46d <+253>: jbe 0x7f0f2418a80c > > 0x00007f0f2418a473 <+259>: shl $0x6,%rax rax:192 > 0x00007f0f2418a477 <+263>: sub $0x1,%rdx rdx:0xffffff 16777215 > (chan->backend.buf_size - 1) > 0x00007f0f2418a47b <+267>: lea 0x10(%rax,%rcx,1),%r10 rax:192 > rcx:0x7f0f0402e610(handle->table) r10:0x7f0f0402e6e0 > 0x00007f0f2418a480 <+272>: and %r9,%rdx r9:0x8fdc00000 rdx:0xc00000 12582= 912 > buf_trunc(consume_old, chan->backend.buf_size - 1) > 0x00007f0f2418a483 <+275>: mov 0x8(%rdi),%rax rdi:0x7f0efda75150 rax:2688 > ref_offset =3D (size_t) ref->offset > 0x00007f0f2418a487 <+279>: mov %r8d,%ecx ecx:0x14(20) > 0x00007f0f2418a48a <+282>: shr %cl,%rdx (....) > 0x00007f0f2418a48d <+285>: shl $0x7,%rdx > 0x00007f0f2418a491 <+289>: add %rax,%rdx > 0x00007f0f2418a494 <+292>: lea 0x80(%rdx),%rax > 0x00007f0f2418a49b <+299>: cmp 0x28(%r10),%rax > 0x00007f0f2418a49f <+303>: ja 0x7f0f2418a80c > > 0x00007f0f2418a4a5 <+309>: mov 0x20(%r10),%rax r10:0x7f0f0402e6e0 > rax:0x7f0efb86f000(buf) > 0x00007f0f2418a4a9 <+313>: add %rdx,%rax rdx:3200 rax:0x7f0efb86fc80 > 0x00007f0f2418a4ac <+316>: mov (%rax),%rax > rax:0x86c00000(commit_count!!!!!!Incremented _once_ at sb switch cc_sb) > 0x00007f0f2418a4af <+319>: mov 0x80(%rbp),%r8 > r8:0x100000(chan->backend.subbuf_size) > 0x00007f0f2418a4b6 <+326>: mov 0x78(%rbp),%rdx > rdx:0x1000000(chan->backend.buf_size) > 0x00007f0f2418a4ba <+330>: mov 0x8c(%rbp),%ecx ecx:4 > 0x00007f0f2418a4c0 <+336>: mov 0x80(%rsi),%rdi rdi:0x86d400000 > 0x00007f0f2418a4c7 <+343>: sub %r8,%rax rax:0x86b00000(commit_count - > chan->backend.subbuf_size) > 0x00007f0f2418a4ca <+346>: and 0x8(%rbp),%rax rax & chan->commit_count_ma= sk =3D > rax:0x86b00000 > 0x00007f0f2418a4ce <+350>: neg %rdx rdx:0x1000000(16M chan->backend.buf_s= ize) > --> 0xffffffffff000000(-16777216) > 0x00007f0f2418a4d1 <+353>: and %r9,%rdx r9:0x86c400000(consume_old) > rdx:0x86c000000 > 0x00007f0f2418a4d4 <+356>: shr %cl,%rdx cl:4 rdx:0x86c00000 > 0x00007f0f2418a4d7 <+359>: cmp %rdx,%rax rax:0x86b0000 rdx:0x86c00000 > 0x00007f0f2418a4da <+362>: jne 0x7f0f2418a411 > > 0x00007f0f2418a4e0 <+368>: mov %r8,%rax > 209 static inline > 210 int lib_ring_buffer_poll_deliver(const struct > lttng_ust_lib_ring_buffer_config *config, > 211 struct lttng_ust_lib_ring_buffer *buf, > 212 struct channel *chan, > 213 struct lttng_ust_shm_handle *handle) > 214 { > 215 unsigned long consumed_old, consumed_idx, commit_count, write_offset; > 216 > 217 consumed_old =3D uatomic_read(&buf->consumed); > 218 consumed_idx =3D subbuf_index(consumed_old, chan); > 219 commit_count =3D v_read(config, &shmp_index(handle, buf->commit_cold, > consumed_idx)->cc_sb); > 220 /* > 221 * No memory barrier here, since we are only interested > 222 * in a statistically correct polling result. The next poll will > 223 * get the data is we are racing. The mb() that ensures correct > 224 * memory order is in get_subbuf. > 225 */ > 226 write_offset =3D v_read(config, &buf->offset); > 227 > 228 /* > 229 * Check that the subbuffer we are trying to consume has been > 230 * already fully committed. > 231 */ > 232 > 233 if (((commit_count - chan->backend.subbuf_size) > 234 & chan->commit_count_mask) > 235 - (buf_trunc(consumed_old, chan) > 236 >> chan->backend.num_subbuf_order) > 237 !=3D 0) > 238 return 0; > 239 > 240 /* > 241 * Check that we are not about to read the same subbuffer in > 242 * which the writer head is. > 243 */ > 244 if (subbuf_trunc(write_offset, chan) - subbuf_trunc(consumed_old, cha= n) > 245 =3D=3D 0) > 246 return 0; > 247 > 248 return 1; > 249 > 250 } > It seems that lib_ring_buffer_channel_do_read()-> lib_ring_buffer_poll_de= liver() > returned 0 at line 238 ($consume_old:0x86c400000, $commint_count:0x86c000= 00, > $write_offset:0x86d400000). > Buffer type: per UID > Channels: > ------------- > - channel0: [enabled] > Attributes: > overwrite mode: 0 > subbufers size: 1048576 > number of subbufers: 16 > switch timer interval: 20000000 > read timer interval: 60000000 > trace file count: 80 > trace file size (bytes): 2097152 > output: mmap() > PS: Lttng version is 2.7(I know it is an very old version:) ) > Do you have any ideas about this problem? Thanks in advance > Thanks a lot > zhenyu.ren > _______________________________________________ > lttng-dev mailing list > lttng-dev@lists.lttng.org > https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --=20 Mathieu Desnoyers=20 EfficiOS Inc.=20 http://www.efficios.com=20 --=_fedc12fe-9489-44d5-81b6-2906f2ebede0 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi,

Can you try:
=

- Reproducing with a more recent L= TTng-UST/LTTng-Tools ? (e.g. 2.13). LTTng 2.7 is not supported anymore.
- Try to reproduce with "per-pid" UST buffe= rs rather than "per-uid",
- Try to repro= duce without the "tracefile rotation" mode (without --tracefile-count and -= -tracefile-size options to the channel).

Also, do you happen to have traced app= lications which are frequently killed asynchronously
while some of their threads are actively tracing by any chance= ?

T= hanks,

Mathieu


----- On Dec 7, 2021, at 4:03 AM, lttng-dev <lttng-dev@= lists.lttng.org> wrote:
Hi, Lttng-dev:<= /span>

  I found a strange problem related to lttng-consumed and= ctf files recently. The ctf files belongs to some CPUs have been stopped r= otating but other ctf files(belong to other CPUs) keeped working as usual. = I am very sure all CPUs were producing spans all the time.

=
#date; ls -ltrh channel0_0_* |tail -n 3;=  date;ls -ltrh channel0_1_* |tail -n 3
Tue=  Dec  7 16:25:45 CST 2021
-rw-rw---- = 1 root root 1.8M Dec  7 16:20 chann= el0_0_17
-rw-rw---- 1 root root 2.0M Dec &= nbsp;7 16:23 channel0_0_18
-rw-rw---- 1 root ro= ot 916K Dec  7 16:24 channel0_0_19
Tue&nbs= p;Dec  7 16:25:45 CST 2021
-rw-rw---- 1&nb= sp;root root 1.7M Dec  6 00:30 channel0_= 1_8
-rw-rw---- 1 root root 1.9M Dec  = 6 00:31 channel0_1_9
-rw-rw---- 1 root root&nbs= p;388K Dec  6 00:32 channel0_1_10

&n= bsp;   Notice that the ctf files with CPU0 (channel0_0_19) was modifie= d at the time "now", but the ctf files with CPU1(channeo0_1_10) has been st= opped working at Dec 6.
    I gdb lttng-consumerd=EF=BC= =88break at lib_ring_buffer_channel_do_read() and  lib_ring_buffe= r_poll_deliver()--I configured a read timer on channels). See the following= s (li= b_ring_buffer_poll_deliver() )

   0x000= 07f0f2418a445 <+213>: mov    0x0(%r13),= %rcx            = ;    rcx:0x7f0f0402e610(handle->table)
 &nbs= p; 0x00007f0f2418a449 <+217>: mov   &nb= sp;0x98(%rsi),%r9         &nbs= p;      rsi:0x7f0efb86f000(buf) r9:0x86c= 400000(consumed_old =3D buf->consumed)
   0x= 00007f0f2418a450 <+224>: lea    0x150(%= rsi),%rdi           =    rdi:0x7f0efda75150
   0x00007f0f2418a45= 7 <+231>: mov    0x150(%rsi),%rax =             &nb= sp;rax:3(cpu?)
   0x00007f0f2418a45e <+238>:&n= bsp;mov    0x78(%rbp),%rdx     = ;          rbp:chan =      rdx:0x1000000(chan->backend.buf_size)
&= nbsp;  0x00007f0f2418a462 <+242>: mov  =   0x88(%rbp),%r8d        =        r8d:0x14(20)(chan->backend.buf= _size_order)
   0x00007f0f2418a469 <+249>:&nbs= p;cmp    %rax,0x8(%rcx)
   0x00007f0f= 2418a46d <+253>: jbe    0x7f0f2418a80c&= nbsp;<lib_ring_buffer_channel_do_read+1180>
   0x00= 007f0f2418a473 <+259>: shl    $0x6,%rax=             &nb= sp;        rax:192
  &= nbsp;0x00007f0f2418a477 <+263>: sub    = $0x1,%rdx           =           rdx:0xffffff&nb= sp;16777215 (chan->backend.buf_size - 1)
  &= nbsp;0x00007f0f2418a47b <+267>: lea    = 0x10(%rax,%rcx,1),%r10        rax:1= 92 rcx:0x7f0f0402e610(handle->table) r10:0x7f0f0402e6e0
&nb= sp;  0x00007f0f2418a480 <+272>: and  &n= bsp; %r9,%rdx         &nb= sp;            = r9:0x8fdc00000 rdx:0xc00000 12582912  buf_trunc(consume= _old, chan->backend.buf_size - 1)
   0x= 00007f0f2418a483 <+275>: mov    0x8(%rd= i),%rax           &n= bsp;    rdi:0x7f0efda75150 rax:2688 ref_offse= t =3D (size_t) ref->offset
   0x00007f0= f2418a487 <+279>: mov    %r8d,%ecx = ;            &n= bsp;       ecx:0x14(20)
  &= nbsp;0x00007f0f2418a48a <+282>: shr    = %cl,%rdx           &= nbsp;   (....) 
   0x00007f0f2418a48d=  <+285>: shl    $0x7,%rdx
 &nbs= p; 0x00007f0f2418a491 <+289>: add   &nb= sp;%rax,%rdx
   0x00007f0f2418a494 <+292>:&nbs= p;lea    0x80(%rdx),%rax
   0x00007f0= f2418a49b <+299>: cmp    0x28(%r10),%ra= x
   0x00007f0f2418a49f <+303>: ja &= nbsp;   0x7f0f2418a80c <lib_ring_buffer_channel_do_r= ead+1180>
   0x00007f0f2418a4a5 <+309>:&nbs= p;mov    0x20(%r10),%rax     &= nbsp;         r10:0x7f0f0402e6= e0 rax:0x7f0efb86f000(buf)
   0x00007f0f2418a4a9&nbs= p;<+313>: add    %rdx,%rax   =             &nb= sp;     rdx:3200  rax:0x7f0efb86fc80
=    0x00007f0f2418a4ac <+316>: mov  = ;  (%rax),%rax        &nb= sp;          rax:0x86c000= 00(commit_count!!!!!!Incremented _once_ at sb switch&nb= sp;cc_sb)
   0x00007f0f2418a4af <+319>: m= ov    0x80(%rbp),%r8      = ;          r8:0x100000(ch= an->backend.subbuf_size)
   0x00007f0f2418a4b6 &l= t;+326>: mov    0x78(%rbp),%rdx  &nbs= p;            r= dx:0x1000000(chan->backend.buf_size)
   0x00007f0f2418= a4ba <+330>: mov    0x8c(%rbp),%ecx&nbs= p;            &= nbsp; ecx:4
   0x00007f0f2418a4c0 <+336>:=  mov    0x80(%rsi),%rdi    &nb= sp;          rdi:0x86d400= 000
   0x00007f0f2418a4c7 <+343>: sub&nbs= p;   %r8,%rax        = ;            &n= bsp; rax:0x86b00000(commit_count - chan->backend.subbuf_s= ize)
   0x00007f0f2418a4ca <+346>: and&nb= sp;   0x8(%rbp),%rax      &nbs= p;         rax & = ;chan->commit_count_mask =3D rax:0x86b00000
  &nb= sp;0x00007f0f2418a4ce <+350>: neg    %r= dx            &= nbsp;           &nbs= p; rdx:0x1000000(16M chan->backend.buf_size)  -->=  0xffffffffff000000(-16777216)
   0x00007f0f2418a4d1=  <+353>: and    %r9,%rdx  &nb= sp;            =        r9:0x86c400000(consume_old) =   rdx:0x86c000000
   0x00007f0f2418a4d4 &l= t;+356>: shr    %cl,%rdx    = ;            &n= bsp;     cl:4      &= nbsp;           &nbs= p;       rdx:0x86c00000
  &= nbsp;0x00007f0f2418a4d7 <+359>: cmp    = %rdx,%rax           =           rax:0x86b0000&n= bsp;rdx:0x86c00000
   0x00007f0f2418a4da <+362>= ;: jne    0x7f0f2418a411 <lib_ring_buffer_= channel_do_read+161>
   0x00007f0f2418a4e0 <+3= 68>: mov    %r8,%rax
   
209 static inline
210 int lib_ring_buffer_po= ll_deliver(const struct lttng_ust_lib_ring_buffer_config *co= nfig,
211          &nb= sp;       struct lttng_ust_lib_ring= _buffer *buf,
212        &n= bsp;            = ; struct channel *chan,
213     =             &nb= sp;struct lttng_ust_shm_handle *handle)
214 {
= 215     unsigned long consumed_old, = ;consumed_idx, commit_count, write_offset;
216 
217&nb= sp;    consumed_old =3D uatomic_read(&buf= ->consumed);
218     consumed_idx =3D&n= bsp;subbuf_index(consumed_old, chan);
219    &n= bsp;commit_count =3D v_read(config, &shmp_index(handle,&= nbsp;buf->commit_cold, consumed_idx)->cc_sb);
220  =    /*
221      * No&nb= sp;memory barrier here, since we are only&nbs= p;interested
222      * in a&nbs= p;statistically correct polling result. The next&n= bsp;poll will
223      * get&nbs= p;the data is we are racing. The mb()&nb= sp;that ensures correct
224      = ;* memory order is in get_subbuf.
225 &nbs= p;    */
226     write_offs= et =3D v_read(config, &buf->offset);
227 
= 228     /*
229     &nb= sp;* Check that the subbuffer we are try= ing to consume has been
230    &= nbsp; * already fully committed.
231  &nbs= p;   */
232 
233     if&= nbsp;(((commit_count - chan->backend.subbuf_size)
234 =          & chan->c= ommit_count_mask)
235        &nb= sp;- (buf_trunc(consumed_old, chan)
236    = ;        >> chan->bac= kend.num_subbuf_order)
237       &nbs= p; !=3D 0)
238        =  return 0;
239 
240     /*241      * Check that we = ;are not about to read the same subbuffe= r in
242      * which the&n= bsp;writer head is.
243      */<= br>244     if (subbuf_trunc(write_offset,&nbs= p;chan) - subbuf_trunc(consumed_old, chan)
245  = ;       =3D=3D 0)
246  = ;       return 0;
247 
2= 48     return 1;
249 
250 }
      It seems that lib_ring_buffer_channe= l_do_read()->lib_ring_buffer_poll_deliver() returned 0 at line 238 ($consume_= old:0x86c400000, $commint_count:0x86c00000, $write_offset:0x86d400000).

Buffer type: per UID
Channels:<= br>-------------
- channel0: [enabled]

  &nbs= p; Attributes:
      overwrite m= ode: 0
      subbufers size:&nbs= p;1048576
      number of subbuf= ers: 16
      switch timer = interval: 20000000
      read ti= mer interval: 60000000
      tra= ce file count: 80
      tra= ce file size (bytes): 2097152
   &nbs= p;  output: mmap()

   = ;  PS: Lttng version is 2.7(I know it is an very old version:) )
     Do you have any ideas about this problem? Thanks= in advance

Thanks a lot
zhenyu.re= n

_______________________________________________
= lttng-dev mailing list
lttng-dev@lists.lttng.org
https://lists.lttng.= org/cgi-bin/mailman/listinfo/lttng-dev

--
Mathieu Desnoyers
E= fficiOS Inc.
http://www.efficios.com
--=_fedc12fe-9489-44d5-81b6-2906f2ebede0-- --===============5366344613872067940== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ lttng-dev mailing list lttng-dev@lists.lttng.org https://lists.lttng.org/cgi-bin/mailman/listinfo/lttng-dev --===============5366344613872067940==--