Netdev List
 help / color / mirror / Atom feed
* Re: [PATCH] can: mark expected switch fall-throughs
From: Gustavo A. R. Silva @ 2019-02-14 23:14 UTC (permalink / raw)
  To: Alexandre Belloni
  Cc: Nicolas.Ferre, wg, mkl, davem, Ludovic.Desroches, linux-can,
	netdev, linux-arm-kernel, linux-kernel, Kees Cook
In-Reply-To: <20190214230756.GR10129@piout.net>



On 2/14/19 5:07 PM, Alexandre Belloni wrote:
> On 14/02/2019 17:04:03-0600, Gustavo A. R. Silva wrote:
>>
>>
>> On 2/14/19 4:17 PM, Alexandre Belloni wrote:
>>> Hi,
>>>
>>> On 14/02/2019 15:37:26-0600, Gustavo A. R. Silva wrote:
>>>>
>>>>
>>>> On 1/30/19 2:11 AM, Nicolas.Ferre@microchip.com wrote:
>>>>> On 29/01/2019 at 19:06, Gustavo A. R. Silva wrote:
>>>>>> In preparation to enabling -Wimplicit-fallthrough, mark switch cases
>>>>>> where we are expecting to fall through.
>>>>>>
>>>>>> This patch fixes the following warnings:
>>>>>>
>>>>>> drivers/net/can/peak_canfd/peak_pciefd_main.c:668:3: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>> drivers/net/can/spi/mcp251x.c:875:7: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>> drivers/net/can/usb/peak_usb/pcan_usb.c:422:6: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>> drivers/net/can/at91_can.c:895:6: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>> drivers/net/can/at91_can.c:953:15: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>> drivers/net/can/usb/peak_usb/pcan_usb.c: In function ‘pcan_usb_decode_error’:
>>>>>> drivers/net/can/usb/peak_usb/pcan_usb.c:422:6: warning: this statement may fall through [-Wimplicit-fallthrough=]
>>>>>>     if (n & PCAN_USB_ERROR_BUS_LIGHT) {
>>>>>>        ^
>>>>>> drivers/net/can/usb/peak_usb/pcan_usb.c:428:2: note: here
>>>>>>    case CAN_STATE_ERROR_WARNING:
>>>>>>    ^~~~
>>>>>>
>>>>>> Warning level 3 was used: -Wimplicit-fallthrough=3
>>>>>>
>>>>>> This patch is part of the ongoing efforts to enabling
>>>>>> -Wimplicit-fallthrough.
>>>>>>
>>>>>> Notice that in some cases spelling mistakes were fixed.
>>>>>> In other cases, the /* fall through */ comment is placed
>>>>>> at the bottom of the case statement, which is what GCC
>>>>>> is expecting to find.
>>>>>>
>>>>>> Signed-off-by: Gustavo A. R. Silva <gustavo@embeddedor.com>
>>>>>> ---
>>>>>>   drivers/net/can/at91_can.c                    | 6 ++++--
>>>>>
>>>>> For this one:
>>>>> Acked-by: Nicolas Ferre <nicolas.ferre@microchip.com>
>>>>>
>>>>
>>>> Thanks, Nicolas.
>>>>
>>>
>>> I though I had a déjà vu but you actually sent the at91 part twice.
>>>
>>
>> It wasn't intentional.
>>
>>>> Dave:
>>>>
>>>> I wonder if you can take this patch.
>>>>
>>>> Thanks
>>>> --
>>>> Gustavo
>>>>
>>>>>>   drivers/net/can/peak_canfd/peak_pciefd_main.c | 2 +-
>>>>>>   drivers/net/can/spi/mcp251x.c                 | 3 ++-
>>>>>>   drivers/net/can/usb/peak_usb/pcan_usb.c       | 2 +-
>>>>>>   4 files changed, 8 insertions(+), 5 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/net/can/at91_can.c b/drivers/net/can/at91_can.c
>>>>>> index d98c69045b17..1718c20f9c99 100644
>>>>>> --- a/drivers/net/can/at91_can.c
>>>>>> +++ b/drivers/net/can/at91_can.c
>>>>>> @@ -902,7 +902,8 @@ static void at91_irq_err_state(struct net_device *dev,
>>>>>>   				CAN_ERR_CRTL_TX_WARNING :
>>>>>>   				CAN_ERR_CRTL_RX_WARNING;
>>>>>>   		}
>>>>>> -	case CAN_STATE_ERROR_WARNING:	/* fallthrough */
>>>>>> +		/* fall through */
>>>>>> +	case CAN_STATE_ERROR_WARNING:
>>>>>>   		/*
>>>>>>   		 * from: ERROR_ACTIVE, ERROR_WARNING
>>>>>>   		 * to  : ERROR_PASSIVE, BUS_OFF
>>>>>> @@ -951,7 +952,8 @@ static void at91_irq_err_state(struct net_device *dev,
>>>>>>   		netdev_dbg(dev, "Error Active\n");
>>>>>>   		cf->can_id |= CAN_ERR_PROT;
>>>>>>   		cf->data[2] = CAN_ERR_PROT_ACTIVE;
>>>>>> -	case CAN_STATE_ERROR_WARNING:	/* fallthrough */
>>>
>>> Seriously, for that one, you should fix the compiler. The fall through
>>
>> I'll pass your feedback on to the GCC guys.
>>
>>> is not implicit, it is actually quite explicit and the warning is simply
>>> wrong.
>>>
>>> Also, the gcc documentation says that -Wimplicit-fallthrough=3
>>> recognizes /* fallthrough */ as a proper fall through comment (and I
>>> tested with gcc 8.2).
>>>
>>
>> Yeah. But that's not the relevant change in this case.  Notice that the
>> comment was moved to the very bottom of the previous case.
>>
> 
> Yes and it doesn't matter for gcc, I tested with gcc 8.2.
> 

Yeah. But, again, you are missing the relevant part of the patch.

--
Gustavo

^ permalink raw reply

* Re: [PATCH] can: mark expected switch fall-throughs
From: Alexandre Belloni @ 2019-02-14 23:21 UTC (permalink / raw)
  To: Gustavo A. R. Silva
  Cc: Nicolas.Ferre, wg, mkl, davem, Ludovic.Desroches, linux-can,
	netdev, linux-arm-kernel, linux-kernel, Kees Cook
In-Reply-To: <c597ac66-e79d-50e4-6906-923bd18834e9@embeddedor.com>

On 14/02/2019 17:14:05-0600, Gustavo A. R. Silva wrote:
> >>> Also, the gcc documentation says that -Wimplicit-fallthrough=3
> >>> recognizes /* fallthrough */ as a proper fall through comment (and I
> >>> tested with gcc 8.2).
> >>>
> >>
> >> Yeah. But that's not the relevant change in this case.  Notice that the
> >> comment was moved to the very bottom of the previous case.
> >>
> > 
> > Yes and it doesn't matter for gcc, I tested with gcc 8.2.
> > 
> 
> Yeah. But, again, you are missing the relevant part of the patch.
> 

Right, I misread the patch and though you were moving the comment after
the case statement.


-- 
Alexandre Belloni, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

^ permalink raw reply

* [RFC iproute2 v2] ip route: get: allow zero-length subnet mask
From: Luca Boccassi @ 2019-02-14 23:29 UTC (permalink / raw)
  To: netdev; +Cc: stephen, Luca Boccassi, Clément Hertling
In-Reply-To: <20190213200954.32271-1-bluca@debian.org>

A /0 subnet mask is theoretically valid, but ip route get doesn't allow
it:

$ ip route get 1.0.0.0/0
need at least a destination address

Change the check and remember whether we found an address or not, since
according to the documentation it's a mandatory parameter.

$ ip/ip route get 1.0.0.0/0
1.0.0.0 via 192.168.1.1 dev eth0 src 192.168.1.91 uid 1000
    cache

Reported-by: Clément Hertling <wxcafe@wxcafe.net>
Signed-off-by: Luca Boccassi <bluca@debian.org>
---
v2: changed the check instead of removing it, so that "ip route get"
fails since the address is a mandatory parameter according to the
manpage.

 ip/iproute.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/ip/iproute.c b/ip/iproute.c
index 5f58a3b3..cc02a3e1 100644
--- a/ip/iproute.c
+++ b/ip/iproute.c
@@ -1932,6 +1932,7 @@ static int iproute_get(int argc, char **argv)
 	int fib_match = 0;
 	int from_ok = 0;
 	unsigned int mark = 0;
+	bool address_found = false;
 
 	iproute_reset_filter(0);
 	filter.cloned = 2;
@@ -2037,11 +2038,12 @@ static int iproute_get(int argc, char **argv)
 				addattr_l(&req.n, sizeof(req),
 					  RTA_DST, &addr.data, addr.bytelen);
 			req.r.rtm_dst_len = addr.bitlen;
+			address_found = true;
 		}
 		argc--; argv++;
 	}
 
-	if (req.r.rtm_dst_len == 0) {
+	if (!address_found) {
 		fprintf(stderr, "need at least a destination address\n");
 		return -1;
 	}
-- 
2.20.1


^ permalink raw reply related

* Re: [PATCH bpf-next v2] bpf: fix memory leak in bpf_lwt_xmit_reroute
From: Alexei Starovoitov @ 2019-02-14 23:33 UTC (permalink / raw)
  To: David Ahern
  Cc: Peter Oskolkov, Alexei Starovoitov, Daniel Borkmann, netdev,
	Peter Oskolkov, Willem de Bruijn
In-Reply-To: <4f4ce7f6-a9df-b898-8de5-05dcbf15b095@gmail.com>

On Thu, Feb 14, 2019 at 12:04:47PM -0700, David Ahern wrote:
> On 2/14/19 11:39 AM, Peter Oskolkov wrote:
> > On error the skb should be freed. Tested with diff/steps
> > provided by David Ahern.
> > 
> > v2: surface routing errors to the user instead of a generic EINVAL,
> >     as suggested by David Ahern.
> > 
> > Reported-by: David Ahern <dsahern@gmail.com>
> > Fixes: 3bd0b15281af ("bpf: add handling of BPF_LWT_REROUTE to lwt_bpf.c")
> > Signed-off-by: Peter Oskolkov <posk@google.com>
> > ---
> >  net/core/lwt_bpf.c | 29 ++++++++++++++++++++---------
> >  1 file changed, 20 insertions(+), 9 deletions(-)
> > 
> 
> Reviewed-by: David Ahern <dsahern@gmail.com>

Applied, Thanks


^ permalink raw reply

* Re: [PATCH v2 bpf-next 0/2] tools/bpf: smaller clean ups
From: Alexei Starovoitov @ 2019-02-14 23:33 UTC (permalink / raw)
  To: Andrii Nakryiko
  Cc: andrii.nakryiko, netdev, kernel-team, yhs, ast, kafai, daniel,
	david.laight, acme
In-Reply-To: <20190213182554.2763867-1-andriin@fb.com>

On Wed, Feb 13, 2019 at 10:25:52AM -0800, Andrii Nakryiko wrote:
> This patchset replaces bzero() with memset() and syncs if_link.h header
> to suppress unsynchronized headers warning.

Applied, Thanks


^ permalink raw reply

* [PATCH 1/1] doc: Mention MSG_ZEROCOPY implementation for UDP
From: Petr Vorel @ 2019-02-14 23:43 UTC (permalink / raw)
  To: netdev; +Cc: Petr Vorel

MSG_ZEROCOPY implementation for UDP was merged in v5.0,
6e360f733113 ("Merge branch 'udp-msg_zerocopy'").

Signed-off-by: Petr Vorel <pvorel@suse.cz>
---
 Documentation/networking/msg_zerocopy.rst | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Documentation/networking/msg_zerocopy.rst b/Documentation/networking/msg_zerocopy.rst
index fe46d4867e2d..18c1415e7bfa 100644
--- a/Documentation/networking/msg_zerocopy.rst
+++ b/Documentation/networking/msg_zerocopy.rst
@@ -7,7 +7,7 @@ Intro
 =====
 
 The MSG_ZEROCOPY flag enables copy avoidance for socket send calls.
-The feature is currently implemented for TCP sockets.
+The feature is currently implemented for TCP and UDP sockets.
 
 
 Opportunity and Caveats
-- 
2.20.1


^ permalink raw reply related

* RE:(2) [Bug reporting] kernel panic during handle the dst unreach icmp msg.
From: 배석진 @ 2019-02-14 23:54 UTC (permalink / raw)
  To: Eric Dumazet, 배석진, netdev@vger.kernel.org
  Cc: 박종언, Steffen Klassert, Herbert Xu
In-Reply-To: <da4182b2-61ab-3ac8-b8a7-1b2b90dcbd54@gmail.com>

> I do not believe this patch is needed.
>  
> You probably hit another more serious bug, but since you do not post the full stack trace
> it is hard to help.
>  
> Are you using vti tunnel ?

there's no working logs of vpn/vti/tun on platform or kernel history.
and callstack has no functions about that.
it looks like simple ipv4 usage.

attaching full dump info.
no additional bug or warning on entire kernel history.
anything about tun or kasan even onther.

thanks.


<4>[60392.948306] I[1:    ksoftirqd/1:   19] ------------[ cut here ]------------
<0>[60392.948334] I[1:    ksoftirqd/1:   19] kernel BUG at net/ipv4/tcp_ipv4.c:519!
<2>[60392.948344] I[1:    ksoftirqd/1:   19] sec_debug_set_extra_info_fault = BUG / 0xffffff80090351d0
<0>[60392.948386] I[1:    ksoftirqd/1:   19] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
<4>[60392.948395] I[1:    ksoftirqd/1:   19] Modules linked in:
<0>[60392.948408] I[1:    ksoftirqd/1:   19] debug-snapshot: core register saved(CPU:1)
<0>[60392.948416] I[1:    ksoftirqd/1:   19] DISR_EL1: 0000000000000000
<0>[60392.948425] I[1:    ksoftirqd/1:   19] DISR_EL1 valid_bit(31) is NOT set (0x0)
<0>[60392.948433] I[1:    ksoftirqd/1:   19] CPU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
<0>[60392.948441] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
<0>[60392.948449] I[1:    ksoftirqd/1:   19] DSU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
<0>[60392.948457] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
<0>[60392.948464] I[1:    ksoftirqd/1:   19] debug-snapshot: context saved(CPU:1)
<0>[60392.950562] I[1:    ksoftirqd/1:   19] ============ CPU Error Log Info after Cache flush ================
<0>[60392.950579] I[1:    ksoftirqd/1:   19] DISR_EL1: 0000000000000000
<0>[60392.950588] I[1:    ksoftirqd/1:   19] DISR_EL1 valid_bit(31) is NOT set (0x0)
<0>[60392.950596] I[1:    ksoftirqd/1:   19] CPU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
<0>[60392.950604] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
<0>[60392.950611] I[1:    ksoftirqd/1:   19] DSU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
<0>[60392.950619] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
<6>[60392.950631] I[1:    ksoftirqd/1:   19] debug-snapshot: item - log_kevents is disabled
<6>[60392.950638] I[1:    ksoftirqd/1:   19] TIF_FOREIGN_FPSTATE: 0, FP/SIMD depth 0, cpu: 0
<4>[60392.950648] I[1:    ksoftirqd/1:   19] CPU: 1 PID: 19 Comm: ksoftirqd/1 Tainted: G        W       4.14.85-15389299 #1
<4>[60392.950655] I[1:    ksoftirqd/1:   19] Hardware name: Samsung BEYOND2LTE KOR SINGLE 26 board based on EXYNOS9820 (DT)
<4>[60392.950664] I[1:    ksoftirqd/1:   19] task: ffffffc8f0c8c400 task.stack: ffffff800b958000
<4>[60392.950676] I[1:    ksoftirqd/1:   19] PC is at tcp_v4_err+0x4b0/0x4bc
<4>[60392.950684] I[1:    ksoftirqd/1:   19] LR is at tcp_v4_err+0x3ac/0x4bc
<4>[60392.950692] I[1:    ksoftirqd/1:   19] pc : [<ffffff80090351d0>] lr : [<ffffff80090350cc>] pstate: 60c00145
<4>[60392.950698] I[1:    ksoftirqd/1:   19] sp : ffffff800b95b940
<4>[60392.950705] I[1:    ksoftirqd/1:   19] x29: ffffff800b95b990 x28: ffffffc82fe10c2c 
<4>[60392.950714] I[1:    ksoftirqd/1:   19] x27: 0000000000000003 x26: 0000000000000001 
<4>[60392.950723] I[1:    ksoftirqd/1:   19] x25: ffffffc0051e5e00 x24: ffffff800aaebc80 
<4>[60392.950732] I[1:    ksoftirqd/1:   19] x23: ffffffc0051e5f58 x22: 0000000000000000 
<4>[60392.950741] I[1:    ksoftirqd/1:   19] x21: 0000000000000071 x20: ffffffc0051e5e88 
<4>[60392.950750] I[1:    ksoftirqd/1:   19] x19: ffffffc0051e5e00 x18: 0000000000000001 
<4>[60392.950760] I[1:    ksoftirqd/1:   19] x17: 0000000000000400 x16: 0000000000000000 
<4>[60392.950769] I[1:    ksoftirqd/1:   19] x15: ffffffffffffffff x14: 0000000000000000 
<4>[60392.950778] I[1:    ksoftirqd/1:   19] x13: 0000000000000002 x12: 0000000000000000 
<4>[60392.950787] I[1:    ksoftirqd/1:   19] x11: ffffffc0051e5f58 x10: 0000000000007530 
<4>[60392.950796] I[1:    ksoftirqd/1:   19] x9 : 0000000000000020 x8 : 0000000000000200 
<4>[60392.950806] I[1:    ksoftirqd/1:   19] x7 : 0000000000000000 x6 : 0000000000000010 
<4>[60392.950815] I[1:    ksoftirqd/1:   19] x5 : 000000000000808c x4 : 000000000000bb01 
<4>[60392.950825] I[1:    ksoftirqd/1:   19] x3 : 00000000831fd9ac x2 : 000000000000808c 
<4>[60392.950834] I[1:    ksoftirqd/1:   19] x1 : 000000000200a8c0 x0 : 0000000000000040 
<4>[60392.950844] I[1:    ksoftirqd/1:   19] 
<4>[60392.950844] I[1:    ksoftirqd/1:   19] PC: 0xffffff80090350d0:
<4>[60392.950855] I[1:    ksoftirqd/1:   19] 50d0 : 92407C08 14000009 AA1303E0 97FFFE97 17FFFFD0 F940CB08 D538D089 9108E108
<4>[60392.950872] I[1:    ksoftirqd/1:   19] 50f0 : 17FFFF78 52801F48 12001D29 F940AE77 9AC92108 D344FD09 528EA60A 9105626B
<4>[60392.950888] I[1:    ksoftirqd/1:   19] 5110 : F11D4D3F 9A8A3108 EB0B02FF B9072268 54000580 B4000577 97C9A4D3 D29EF9EA
<4>[60392.950903] I[1:    ksoftirqd/1:   19] 5130 : F2BC6A6A F9444A68 F2D374AA D343FC09 F2E4188A 9BCA7D29 D344FD29 EB08013F
<4>[60392.950919] I[1:    ksoftirqd/1:   19] 5150 : 54000069 AA0903E8 F9044A69 B94022E9 B9472277 4B090100 97C94F99 CB0002E8
<4>[60392.950934] I[1:    ksoftirqd/1:   19] 5170 : 7100051F 5400022B 320003EA B000A74B 391D4A6A 93407D08 F945416A D344FD0B
<4>[60392.950951] I[1:    ksoftirqd/1:   19] 5190 : 528EA609 F11D4D7F 9A893108 8B080142 911A4261 AA1303E0 F9034662 97FCBBB7
<4>[60392.950967] I[1:    ksoftirqd/1:   19] 51b0 : AA1603E8 17FFFF86 F9455668 AA1303E0 F9402D08 94063788 AA1603E8 17FFFF80
<4>[60392.950982] I[1:    ksoftirqd/1:   19] 51d0 : D4210000 14000000 00BE7BAD F81C0FF7 A90157F6 A9024FF4 A9037BFD 9100C3FD
<4>[60392.950997] I[1:    ksoftirqd/1:   19] 51f0 : AA0003F6 B940AAC8 F9407AC9 7941BECA B9409AD7 2A2803E8 2A0203F4 2A0103F5
<4>[60392.951012] I[1:    ksoftirqd/1:   19] 5210 : 721B051F 8B0A0133 54000201 321F07E3 2A1503E0 2A1403E1 2A1703E2 2A1F03E4
<4>[60392.951028] I[1:    ksoftirqd/1:   19] 5230 : 97D8189D 13804008 0B000108 53107D08 79002268 7941BEC8 321C03E9 790166C9
<4>[60392.951043] I[1:    ksoftirqd/1:   19] 5250 : 790162C8 14000012 79401A68 B940B2C2 AA1303E0 53027D08 121E0D01 97D8177A
<4>[60392.951058] I[1:    ksoftirqd/1:   19] 5270 : 2A0003E4 321F07E3 2A1503E0 2A1403E1 2A1703E2 97D81888 13804008 0B000108
<4>[60392.951073] I[1:    ksoftirqd/1:   19] 5290 : 12800009 4A484128 79002268 A9437BFD A9424FF4 A94157F6 F84407F7 D65F03C0
<4>[60392.951089] I[1:    ksoftirqd/1:   19] 52b0 : 00BE7BAD A9BF7BFD 910003FD B9458008 B9400002 AA0103E0 2A0803E1 97FFFFC4
<4>[60392.951105] I[1:    ksoftirqd/1:   19] 
<4>[60392.951105] I[1:    ksoftirqd/1:   19] LR: 0xffffff8009034fcc:
<4>[60392.951116] I[1:    ksoftirqd/1:   19] 4fcc : 39404A69 121F1929 7100093F 540001A1 B4000068 F9407508 B4000148 B9408F28
<4>[60392.951131] I[1:    ksoftirqd/1:   19] 4fec : 35000188 F942A668 AA1303E0 B9022E75 940637FA AA1303E0 97FFAE89 14000006
<4>[60392.951147] I[1:    ksoftirqd/1:   19] 500c : B9408F28 35000068 39568268 37000208 B9023275 AA1403E0 94061649 91020260
<4>[60392.951162] I[1:    ksoftirqd/1:   19] 502c : 97D6FDEE 36000060 AA1303E0 97FCB533 A9457BFD A9444FF4 A94357F6 A9425FF8
<4>[60392.951177] I[1:    ksoftirqd/1:   19] 504c : A94167FA A8C66FFC D65F03C0 F942A668 AA1303E0 B9022E75 940637E0 17FFFFEE
<4>[60392.951193] I[1:    ksoftirqd/1:   19] 506c : D00030CB D37DF34A 9114816B B86A6975 7100075F 54FFFA68 B9400129 6B0902FF
<4>[60392.951208] I[1:    ksoftirqd/1:   19] 508c : 54FFFA01 395D4669 34FFF9C9 B5FFF9A8 395D4E69 34FFF969 B9408F2A 35FFF92A
<4>[60392.951223] I[1:    ksoftirqd/1:   19] 50ac : AA0803F6 B9489A68 51000529 391D4E69 340001C8 B948A669 0B480D20 97C94FC1
<4>[60392.951238] I[1:    ksoftirqd/1:   19] 50cc : 395D4E69 92407C08 14000009 AA1303E0 97FFFE97 17FFFFD0 F940CB08 D538D089
<4>[60392.951254] I[1:    ksoftirqd/1:   19] 50ec : 9108E108 17FFFF78 52801F48 12001D29 F940AE77 9AC92108 D344FD09 528EA60A
<4>[60392.951269] I[1:    ksoftirqd/1:   19] 510c : 9105626B F11D4D3F 9A8A3108 EB0B02FF B9072268 54000580 B4000577 97C9A4D3
<4>[60392.951283] I[1:    ksoftirqd/1:   19] 512c : D29EF9EA F2BC6A6A F9444A68 F2D374AA D343FC09 F2E4188A 9BCA7D29 D344FD29
<4>[60392.951299] I[1:    ksoftirqd/1:   19] 514c : EB08013F 54000069 AA0903E8 F9044A69 B94022E9 B9472277 4B090100 97C94F99
<4>[60392.951314] I[1:    ksoftirqd/1:   19] 516c : CB0002E8 7100051F 5400022B 320003EA B000A74B 391D4A6A 93407D08 F945416A
<4>[60392.951328] I[1:    ksoftirqd/1:   19] 518c : D344FD0B 528EA609 F11D4D7F 9A893108 8B080142 911A4261 AA1303E0 F9034662
<4>[60392.951344] I[1:    ksoftirqd/1:   19] 51ac : 97FCBBB7 AA1603E8 17FFFF86 F9455668 AA1303E0 F9402D08 94063788 AA1603E8
<4>[60392.951358] I[1:    ksoftirqd/1:   19] 
<4>[60392.951358] I[1:    ksoftirqd/1:   19] SP: 0xffffff800b95b840:
<4>[60392.951369] I[1:    ksoftirqd/1:   19] b840 : 0B95BA88 FFFFFF80 00002752 00000000 0B95B9A0 FFFFFF80 00000001 00000000
<4>[60392.951384] I[1:    ksoftirqd/1:   19] b860 : 00000001 00000000 051E5E00 FFFFFFC0 0B95B8E0 FFFFFF80 09001E50 FFFFFF80
<4>[60392.951398] I[1:    ksoftirqd/1:   19] b880 : 0B95B8E0 FFFFFF80 EE595D00 B254C55C EF89B000 FFFFFFC8 EE595D00 B254C55C
<4>[60392.951412] I[1:    ksoftirqd/1:   19] b8a0 : 2FE10C24 FFFFFFC8 36B80010 FFFFFFC0 0B2D8880 FFFFFF80 0AAEBC80 FFFFFF80
<4>[60392.951426] I[1:    ksoftirqd/1:   19] b8c0 : 00000010 00000000 00000000 00000000 0B95B930 FFFFFF80 09015704 FFFFFF80
<4>[60392.951440] I[1:    ksoftirqd/1:   19] b8e0 : 00000003 00000000 0906F3D0 FFFFFF80 00000001 00000000 EF89B000 FFFFFFC8
<4>[60392.951454] I[1:    ksoftirqd/1:   19] b900 : 0AAEBC80 FFFFFF80 2FE10C24 FFFFFFC8 00000000 00000000 00000000 FFFFFFC8
<4>[60392.951468] I[1:    ksoftirqd/1:   19] b920 : 00780078 FFFFFFC8 EE595D00 B254C55C 0B95B990 FFFFFF80 09034E40 FFFFFF80
<4>[60392.951482] I[1:    ksoftirqd/1:   19] b940 : 0A5386B0 FFFFFF80 0A540FD0 FFFFFF80 EF89B0A0 FFFFFFC8 EF89B000 FFFFFFC8
<4>[60392.951496] I[1:    ksoftirqd/1:   19] b960 : 00000008 00000000 2FE10C24 FFFFFFC8 00000000 00000000 00000006 00000000
<4>[60392.951511] I[1:    ksoftirqd/1:   19] b980 : 00000000 00000000 1EEBFB80 FFFFFFC8 0B95B9C0 FFFFFF80 0904A990 FFFFFF80
<4>[60392.951526] I[1:    ksoftirqd/1:   19] b9a0 : 0AAEBC80 FFFFFF80 36B82128 FFFFFFC0 2FE10C2C FFFFFFC8 1EEBFB80 FFFFFFC8
<4>[60392.951541] I[1:    ksoftirqd/1:   19] b9c0 : 0B95BA00 FFFFFF80 0904A6D0 FFFFFF80 1EEBFC18 FFFFFFC8 EE595D00 B254C55C
<4>[60392.951556] I[1:    ksoftirqd/1:   19] b9e0 : 0A540578 FFFFFF80 12995900 FFFFFFC0 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
<4>[60392.951570] I[1:    ksoftirqd/1:   19] ba00 : 0B95BA40 FFFFFF80 0904A3AC FFFFFF80 00000008 00000000 1EEBFC18 FFFFFFC8
<4>[60392.951586] I[1:    ksoftirqd/1:   19] ba20 : 0A540578 FFFFFF80 0964F9E8 FFFFFF80 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
<4>[60392.951605] I[1:    ksoftirqd/1:   19] 
<4>[60392.951605] I[1:    ksoftirqd/1:   19] X7: 0xffffffffffffff00:
<4>[60392.951616] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951637] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951655] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951671] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951696] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951713] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951730] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951747] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951764] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951780] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951797] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951814] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951831] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951849] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951867] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951884] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.951902] I[1:    ksoftirqd/1:   19] 
<4>[60392.951902] I[1:    ksoftirqd/1:   19] X11: 0xffffffc0051e5e58:
<4>[60392.951914] I[1:    ksoftirqd/1:   19] 5e58 : 00017031 00000000 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8
<4>[60392.951930] I[1:    ksoftirqd/1:   19] 5e78 : FFFFFFFF 00000004 00000003 00000000 00790078 00000000 00000000 00000000
<4>[60392.951945] I[1:    ksoftirqd/1:   19] 5e98 : 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0
<4>[60392.951961] I[1:    ksoftirqd/1:   19] 5eb8 : 051E5EB0 FFFFFFC0 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0
<4>[60392.951975] I[1:    ksoftirqd/1:   19] 5ed8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.951990] I[1:    ksoftirqd/1:   19] 5ef8 : 00000000 00000000 00000000 00123900 00000000 00000000 00000000 00000000
<4>[60392.952005] I[1:    ksoftirqd/1:   19] 5f18 : 00000000 00000000 00000000 00000000 42FCE000 FFFFFFC8 00000000 00000000
<4>[60392.952020] I[1:    ksoftirqd/1:   19] 5f38 : 00000000 00100000 00000000 00000001 00000000 00000000 00000000 00000000
<4>[60392.952035] I[1:    ksoftirqd/1:   19] 5f58 : 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0 00000000 00000000 FFFFFFFF 00000000
<4>[60392.952050] I[1:    ksoftirqd/1:   19] 5f78 : 00000000 00000000 FFFFFFFF 7FFFFFFF 00000000 00000000 00000000 00000000
<4>[60392.952065] I[1:    ksoftirqd/1:   19] 5f98 : 00000000 00000000 090343B0 FFFFFF80 051E5E00 FFFFFFC0 00000001 FFFFFFFF
<4>[60392.952079] I[1:    ksoftirqd/1:   19] 5fb8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000C026F
<4>[60392.952094] I[1:    ksoftirqd/1:   19] 5fd8 : 0007C913 FFFFFFFF 00000000 00000000 00000000 00000000 00006000 00000000
<4>[60392.952109] I[1:    ksoftirqd/1:   19] 5ff8 : 00000000 00000000 00000001 00000000 014000C0 43A820B3 00010600 00000001
<4>[60392.952124] I[1:    ksoftirqd/1:   19] 6018 : 00000000 00000000 0AAF9E10 FFFFFF80 00000000 00000000 00000071 00000000
<4>[60392.952139] I[1:    ksoftirqd/1:   19] 6038 : 00000000 00002752 00000000 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
<4>[60392.952155] I[1:    ksoftirqd/1:   19] 
<4>[60392.952155] I[1:    ksoftirqd/1:   19] X12: 0xffffffffffffff00:
<4>[60392.952166] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952183] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952199] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952216] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952232] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952249] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952266] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952283] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952300] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952317] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952333] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952350] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952366] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952383] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952401] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952417] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952436] I[1:    ksoftirqd/1:   19] 
<4>[60392.952436] I[1:    ksoftirqd/1:   19] X14: 0xffffffffffffff00:
<4>[60392.952447] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952467] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952483] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952499] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952516] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952532] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952548] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952563] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952580] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952596] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952612] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952628] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952645] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952661] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952677] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952693] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952710] I[1:    ksoftirqd/1:   19] 
<4>[60392.952710] I[1:    ksoftirqd/1:   19] X15: 0xfffffffffffffeff:
<4>[60392.952721] I[1:    ksoftirqd/1:   19] fefc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952737] I[1:    ksoftirqd/1:   19] ff1c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952753] I[1:    ksoftirqd/1:   19] ff3c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952768] I[1:    ksoftirqd/1:   19] ff5c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952784] I[1:    ksoftirqd/1:   19] ff7c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952799] I[1:    ksoftirqd/1:   19] ff9c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952815] I[1:    ksoftirqd/1:   19] ffbc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952830] I[1:    ksoftirqd/1:   19] ffdc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952846] I[1:    ksoftirqd/1:   19] fffc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952861] I[1:    ksoftirqd/1:   19] 001c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952877] I[1:    ksoftirqd/1:   19] 003c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952893] I[1:    ksoftirqd/1:   19] 005c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952910] I[1:    ksoftirqd/1:   19] 007c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952926] I[1:    ksoftirqd/1:   19] 009c : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952944] I[1:    ksoftirqd/1:   19] 00bc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952960] I[1:    ksoftirqd/1:   19] 00dc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952976] I[1:    ksoftirqd/1:   19] 00fc : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.952993] I[1:    ksoftirqd/1:   19] 
<4>[60392.952993] I[1:    ksoftirqd/1:   19] X16: 0xffffffffffffff00:
<4>[60392.953004] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953020] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953036] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953052] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953068] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953084] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953099] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953115] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953130] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953146] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953162] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953178] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953194] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953210] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953226] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953242] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953259] I[1:    ksoftirqd/1:   19] 
<4>[60392.953259] I[1:    ksoftirqd/1:   19] X19: 0xffffffc0051e5d00:
<4>[60392.953270] I[1:    ksoftirqd/1:   19] 5d00 : 00000000 00000000 00000000 00000000 E745D5E7 496DFBF1 00000000 00000000
<4>[60392.953285] I[1:    ksoftirqd/1:   19] 5d20 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953300] I[1:    ksoftirqd/1:   19] 5d40 : 00000000 00000000 00000000 00000000 00000000 00000000 FFFF0000 0200A8C0
<4>[60392.953314] I[1:    ksoftirqd/1:   19] 5d60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953328] I[1:    ksoftirqd/1:   19] 5d80 : 00000000 00000000 0081FF80 00000000 00000000 00080000 00000000 00000000
<4>[60392.953342] I[1:    ksoftirqd/1:   19] 5da0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953356] I[1:    ksoftirqd/1:   19] 5dc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953370] I[1:    ksoftirqd/1:   19] 5de0 : 00000000 00000000 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D
<4>[60392.953384] I[1:    ksoftirqd/1:   19] 5e00 : 831FD9AC 0200A8C0 EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000
<4>[60392.953398] I[1:    ksoftirqd/1:   19] 5e20 : BC8B82B8 FFFFFFC8 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000
<4>[60392.953412] I[1:    ksoftirqd/1:   19] 5e40 : FFFF0000 831FD9AC 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000
<4>[60392.953427] I[1:    ksoftirqd/1:   19] 5e60 : 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004
<4>[60392.953441] I[1:    ksoftirqd/1:   19] 5e80 : 00000003 00000000 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0
<4>[60392.953455] I[1:    ksoftirqd/1:   19] 5ea0 : 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0
<4>[60392.953469] I[1:    ksoftirqd/1:   19] 5ec0 : 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000
<4>[60392.953483] I[1:    ksoftirqd/1:   19] 5ee0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953498] I[1:    ksoftirqd/1:   19] 
<4>[60392.953498] I[1:    ksoftirqd/1:   19] X20: 0xffffffc0051e5d88:
<4>[60392.953509] I[1:    ksoftirqd/1:   19] 5d88 : 0081FF80 00000000 00000000 00080000 00000000 00000000 00000000 00000000
<4>[60392.953523] I[1:    ksoftirqd/1:   19] 5da8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953537] I[1:    ksoftirqd/1:   19] 5dc8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953551] I[1:    ksoftirqd/1:   19] 5de8 : 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D 831FD9AC 0200A8C0
<4>[60392.953565] I[1:    ksoftirqd/1:   19] 5e08 : EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000 BC8B82B8 FFFFFFC8
<4>[60392.953580] I[1:    ksoftirqd/1:   19] 5e28 : 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000 FFFF0000 831FD9AC
<4>[60392.953594] I[1:    ksoftirqd/1:   19] 5e48 : 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000 00004301 00000000
<4>[60392.953608] I[1:    ksoftirqd/1:   19] 5e68 : 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004 00000003 00000000
<4>[60392.953622] I[1:    ksoftirqd/1:   19] 5e88 : 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0
<4>[60392.953636] I[1:    ksoftirqd/1:   19] 5ea8 : 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0 00000000 00000000
<4>[60392.953650] I[1:    ksoftirqd/1:   19] 5ec8 : 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000 00000000 00000000
<4>[60392.953664] I[1:    ksoftirqd/1:   19] 5ee8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00123900
<4>[60392.953678] I[1:    ksoftirqd/1:   19] 5f08 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.953692] I[1:    ksoftirqd/1:   19] 5f28 : 42FCE000 FFFFFFC8 00000000 00000000 00000000 00100000 00000000 00000001
<4>[60392.953706] I[1:    ksoftirqd/1:   19] 5f48 : 00000000 00000000 00000000 00000000 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0
<4>[60392.953720] I[1:    ksoftirqd/1:   19] 5f68 : 00000000 00000000 FFFFFFFF 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
<4>[60392.953736] I[1:    ksoftirqd/1:   19] 
<4>[60392.953736] I[1:    ksoftirqd/1:   19] X22: 0xffffffffffffff00:
<4>[60392.953747] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953764] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953779] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953795] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953813] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953830] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953846] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953862] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953877] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953893] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953909] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953925] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953941] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953957] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953973] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.953989] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
<4>[60392.954007] I[1:    ksoftirqd/1:   19] 
<4>[60392.954007] I[1:    ksoftirqd/1:   19] X23: 0xffffffc0051e5e58:
<4>[60392.954019] I[1:    ksoftirqd/1:   19] 5e58 : 00017031 00000000 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8
<4>[60392.954034] I[1:    ksoftirqd/1:   19] 5e78 : FFFFFFFF 00000004 00000003 00000000 00790078 00000000 00000000 00000000
<4>[60392.954048] I[1:    ksoftirqd/1:   19] 5e98 : 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0
<4>[60392.954062] I[1:    ksoftirqd/1:   19] 5eb8 : 051E5EB0 FFFFFFC0 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0
<4>[60392.954077] I[1:    ksoftirqd/1:   19] 5ed8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954091] I[1:    ksoftirqd/1:   19] 5ef8 : 00000000 00000000 00000000 00123900 00000000 00000000 00000000 00000000
<4>[60392.954105] I[1:    ksoftirqd/1:   19] 5f18 : 00000000 00000000 00000000 00000000 42FCE000 FFFFFFC8 00000000 00000000
<4>[60392.954119] I[1:    ksoftirqd/1:   19] 5f38 : 00000000 00100000 00000000 00000001 00000000 00000000 00000000 00000000
<4>[60392.954135] I[1:    ksoftirqd/1:   19] 5f58 : 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0 00000000 00000000 FFFFFFFF 00000000
<4>[60392.954149] I[1:    ksoftirqd/1:   19] 5f78 : 00000000 00000000 FFFFFFFF 7FFFFFFF 00000000 00000000 00000000 00000000
<4>[60392.954163] I[1:    ksoftirqd/1:   19] 5f98 : 00000000 00000000 090343B0 FFFFFF80 051E5E00 FFFFFFC0 00000001 FFFFFFFF
<4>[60392.954178] I[1:    ksoftirqd/1:   19] 5fb8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000C026F
<4>[60392.954192] I[1:    ksoftirqd/1:   19] 5fd8 : 0007C913 FFFFFFFF 00000000 00000000 00000000 00000000 00006000 00000000
<4>[60392.954206] I[1:    ksoftirqd/1:   19] 5ff8 : 00000000 00000000 00000001 00000000 014000C0 43A820B3 00010600 00000001
<4>[60392.954220] I[1:    ksoftirqd/1:   19] 6018 : 00000000 00000000 0AAF9E10 FFFFFF80 00000000 00000000 00000071 00000000
<4>[60392.954234] I[1:    ksoftirqd/1:   19] 6038 : 00000000 00002752 00000000 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
<4>[60392.954249] I[1:    ksoftirqd/1:   19] 
<4>[60392.954249] I[1:    ksoftirqd/1:   19] X24: 0xffffff800aaebb80:
<4>[60392.954261] I[1:    ksoftirqd/1:   19] bb80 : 0AAEEB88 FFFFFF80 0AB08710 FFFFFF80 08F65024 FFFFFF80 08F65060 FFFFFF80
<4>[60392.954275] I[1:    ksoftirqd/1:   19] bba0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954289] I[1:    ksoftirqd/1:   19] bbc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954303] I[1:    ksoftirqd/1:   19] bbe0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954318] I[1:    ksoftirqd/1:   19] bc00 : 00000000 00000000 00000000 00000000 0AAEBC10 FFFFFF80 0AAEBC10 FFFFFF80
<4>[60392.954333] I[1:    ksoftirqd/1:   19] bc20 : 0AAEBC98 FFFFFF80 0AAEBC98 FFFFFF80 00000000 00000000 00000000 00000000
<4>[60392.954347] I[1:    ksoftirqd/1:   19] bc40 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954361] I[1:    ksoftirqd/1:   19] bc60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954376] I[1:    ksoftirqd/1:   19] bc80 : 00000002 000004F2 00020002 00000000 000174E6 00000000 0AAEBC20 FFFFFF80
<4>[60392.954391] I[1:    ksoftirqd/1:   19] bca0 : 0AAEBC20 FFFFFF80 00000000 00000000 00000000 00000000 0B61BC48 FFFFFF80
<4>[60392.954405] I[1:    ksoftirqd/1:   19] bcc0 : 0B61BC48 FFFFFF80 0A550D80 FFFFFF80 00000000 00000000 9DB59DB5 00000000
<4>[60392.954419] I[1:    ksoftirqd/1:   19] bce0 : 0C000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954433] I[1:    ksoftirqd/1:   19] bd00 : 09644DB0 FFFFFF80 F0000091 00000000 EF839800 FFFFFFC8 EF839900 FFFFFFC8
<4>[60392.954447] I[1:    ksoftirqd/1:   19] bd20 : 0919AB44 FFFFFF80 0A580018 FFFFFF80 00000000 00000001 00000002 00000000
<4>[60392.954461] I[1:    ksoftirqd/1:   19] bd40 : 00000000 00000000 0A580018 FFFFFF80 0AB08478 FFFFFF80 0AAEBD20 FFFFFF80
<4>[60392.954475] I[1:    ksoftirqd/1:   19] bd60 : 00000000 00000000 00000000 00000000 00000000 00000000 EF896858 FFFFFFC8
<4>[60392.954490] I[1:    ksoftirqd/1:   19] 
<4>[60392.954490] I[1:    ksoftirqd/1:   19] X25: 0xffffffc0051e5d00:
<4>[60392.954501] I[1:    ksoftirqd/1:   19] 5d00 : 00000000 00000000 00000000 00000000 E745D5E7 496DFBF1 00000000 00000000
<4>[60392.954516] I[1:    ksoftirqd/1:   19] 5d20 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954530] I[1:    ksoftirqd/1:   19] 5d40 : 00000000 00000000 00000000 00000000 00000000 00000000 FFFF0000 0200A8C0
<4>[60392.954544] I[1:    ksoftirqd/1:   19] 5d60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954562] I[1:    ksoftirqd/1:   19] 5d80 : 00000000 00000000 0081FF80 00000000 00000000 00080000 00000000 00000000
<4>[60392.954579] I[1:    ksoftirqd/1:   19] 5da0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954593] I[1:    ksoftirqd/1:   19] 5dc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954607] I[1:    ksoftirqd/1:   19] 5de0 : 00000000 00000000 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D
<4>[60392.954622] I[1:    ksoftirqd/1:   19] 5e00 : 831FD9AC 0200A8C0 EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000
<4>[60392.954640] I[1:    ksoftirqd/1:   19] 5e20 : BC8B82B8 FFFFFFC8 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000
<4>[60392.954673] I[1:    ksoftirqd/1:   19] 5e40 : FFFF0000 831FD9AC 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000
<4>[60392.954687] I[1:    ksoftirqd/1:   19] 5e60 : 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004
<4>[60392.954702] I[1:    ksoftirqd/1:   19] 5e80 : 00000003 00000000 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0
<4>[60392.954716] I[1:    ksoftirqd/1:   19] 5ea0 : 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0
<4>[60392.954730] I[1:    ksoftirqd/1:   19] 5ec0 : 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000
<4>[60392.954745] I[1:    ksoftirqd/1:   19] 5ee0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954761] I[1:    ksoftirqd/1:   19] 
<4>[60392.954761] I[1:    ksoftirqd/1:   19] X28: 0xffffffc82fe10b2c:
<4>[60392.954772] I[1:    ksoftirqd/1:   19] 0b2c : 05F02FA5 F6F39767 535F5E0E 0CC4A1B3 04B50A41 DE8744AE 00000040 00000020
<4>[60392.954787] I[1:    ksoftirqd/1:   19] 0b4c : 00000000 00000002 00000000 DB9D6F18 FFFFFFC8 00000000 00000000 00000000
<4>[60392.954801] I[1:    ksoftirqd/1:   19] 0b6c : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954815] I[1:    ksoftirqd/1:   19] 0b8c : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954830] I[1:    ksoftirqd/1:   19] 0bac : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954844] I[1:    ksoftirqd/1:   19] 0bcc : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954858] I[1:    ksoftirqd/1:   19] 0bec : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
<4>[60392.954872] I[1:    ksoftirqd/1:   19] 0c0c : 00080000 5000C045 000036D4 62240140 0200A8C0 0200A8C0 ABB50103 00000000
<4>[60392.954886] I[1:    ksoftirqd/1:   19] 0c2c : 34000045 004023E3 99CA0640 0200A8C0 831FD9AC BB018C80 AE46FB9F 537870A5
<4>[60392.954900] I[1:    ksoftirqd/1:   19] 0c4c : 48031180 00002D8D 0A080101 F2556142 DBE9DC24 736D692E 76726573 00656369
<4>[60392.954914] I[1:    ksoftirqd/1:   19] 0c6c : 00390100 00000000 4E000000 554C4C55 00644C4C 00000000 00000000 00000000
<4>[60392.954928] I[1:    ksoftirqd/1:   19] 0c8c : 00000000 00000000 00000000 00000000 00000000 00000000 00000001 00000000
<4>[60392.954942] I[1:    ksoftirqd/1:   19] 0cac : 00000000 00000000 00000000 00000000 00000000 00000001 00000000 00000000
<4>[60392.954956] I[1:    ksoftirqd/1:   19] 0ccc : 00000000 00000000 00000000 00000000 00000000 00100000 00000001 2FE10CF8
<4>[60392.954970] I[1:    ksoftirqd/1:   19] 0cec : FFFFFFC8 00000000 00000000 001B4880 FFFFFFBF 00001000 00000000 00000000
<4>[60392.954984] I[1:    ksoftirqd/1:   19] 0d0c : 00000002 00000001 00000001 00000000 00000000 00000001 00000001 00000000
<4>[60392.954999] I[1:    ksoftirqd/1:   19] 
<4>[60392.954999] I[1:    ksoftirqd/1:   19] X29: 0xffffff800b95b890:
<4>[60392.955010] I[1:    ksoftirqd/1:   19] b890 : EF89B000 FFFFFFC8 EE595D00 B254C55C 2FE10C24 FFFFFFC8 36B80010 FFFFFFC0
<4>[60392.955025] I[1:    ksoftirqd/1:   19] b8b0 : 0B2D8880 FFFFFF80 0AAEBC80 FFFFFF80 00000010 00000000 00000000 00000000
<4>[60392.955039] I[1:    ksoftirqd/1:   19] b8d0 : 0B95B930 FFFFFF80 09015704 FFFFFF80 00000003 00000000 0906F3D0 FFFFFF80
<4>[60392.955053] I[1:    ksoftirqd/1:   19] b8f0 : 00000001 00000000 EF89B000 FFFFFFC8 0AAEBC80 FFFFFF80 2FE10C24 FFFFFFC8
<4>[60392.955067] I[1:    ksoftirqd/1:   19] b910 : 00000000 00000000 00000000 FFFFFFC8 00780078 FFFFFFC8 EE595D00 B254C55C
<4>[60392.955081] I[1:    ksoftirqd/1:   19] b930 : 0B95B990 FFFFFF80 09034E40 FFFFFF80 0A5386B0 FFFFFF80 0A540FD0 FFFFFF80
<4>[60392.955096] I[1:    ksoftirqd/1:   19] b950 : EF89B0A0 FFFFFFC8 EF89B000 FFFFFFC8 00000008 00000000 2FE10C24 FFFFFFC8
<4>[60392.955110] I[1:    ksoftirqd/1:   19] b970 : 00000000 00000000 00000006 00000000 00000000 00000000 1EEBFB80 FFFFFFC8
<4>[60392.955124] I[1:    ksoftirqd/1:   19] b990 : 0B95B9C0 FFFFFF80 0904A990 FFFFFF80 0AAEBC80 FFFFFF80 36B82128 FFFFFFC0
<4>[60392.955138] I[1:    ksoftirqd/1:   19] b9b0 : 2FE10C2C FFFFFFC8 1EEBFB80 FFFFFFC8 0B95BA00 FFFFFF80 0904A6D0 FFFFFF80
<4>[60392.955152] I[1:    ksoftirqd/1:   19] b9d0 : 1EEBFC18 FFFFFFC8 EE595D00 B254C55C 0A540578 FFFFFF80 12995900 FFFFFFC0
<4>[60392.955167] I[1:    ksoftirqd/1:   19] b9f0 : 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8 0B95BA40 FFFFFF80 0904A3AC FFFFFF80
<4>[60392.955181] I[1:    ksoftirqd/1:   19] ba10 : 00000008 00000000 1EEBFC18 FFFFFFC8 0A540578 FFFFFF80 0964F9E8 FFFFFF80
<4>[60392.955195] I[1:    ksoftirqd/1:   19] ba30 : 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8 0B95BA70 FFFFFF80 0900B104 FFFFFF80
<4>[60392.955209] I[1:    ksoftirqd/1:   19] ba50 : 2FE10C10 FFFFFFC8 00000001 00000000 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
<4>[60392.955223] I[1:    ksoftirqd/1:   19] ba70 : 0B95BAE0 FFFFFF80 0900AFD4 FFFFFF80 00000004 00000000 00000001 00000002
<4>[60392.955238] I[1:    ksoftirqd/1:   19] 
<0>[60392.955246] I[1:    ksoftirqd/1:   19] Process ksoftirqd/1 (pid: 19, stack limit = 0xffffff800b958000)
<0>[60392.955253] I[1:    ksoftirqd/1:   19] Call trace:
<4>[60392.955263] I[1:    ksoftirqd/1:   19] Exception stack(0xffffff800b95b700 to 0xffffff800b95b840)
<4>[60392.955272] I[1:    ksoftirqd/1:   19] b700: 0000000000000040 000000000200a8c0 000000000000808c 00000000831fd9ac
<4>[60392.955281] I[1:    ksoftirqd/1:   19] b720: 000000000000bb01 000000000000808c 0000000000000010 0000000000000000
<4>[60392.955290] I[1:    ksoftirqd/1:   19] b740: 0000000000000200 0000000000000020 0000000000007530 ffffffc0051e5f58
<4>[60392.955298] I[1:    ksoftirqd/1:   19] b760: 0000000000000000 0000000000000002 0000000000000000 ffffffffffffffff
<4>[60392.955306] I[1:    ksoftirqd/1:   19] b780: 0000000000000000 0000000000000400 0000000000000001 ffffffc0051e5e00
<4>[60392.955315] I[1:    ksoftirqd/1:   19] b7a0: ffffffc0051e5e88 0000000000000071 0000000000000000 ffffffc0051e5f58
<4>[60392.955323] I[1:    ksoftirqd/1:   19] b7c0: ffffff800aaebc80 ffffffc0051e5e00 0000000000000001 0000000000000003
<4>[60392.955332] I[1:    ksoftirqd/1:   19] b7e0: ffffffc82fe10c2c ffffff800b95b990 ffffff80090350cc ffffff800b95b940
<4>[60392.955341] I[1:    ksoftirqd/1:   19] b800: ffffff80090351d0 0000000060c00145 0000000000000001 b254c55cee595d00
<4>[60392.955349] I[1:    ksoftirqd/1:   19] b820: ffffffffffffffff ffffff800b2d7db8 ffffff800b95b990 ffffff80090351d0
<0>[60392.955362] I[1:    ksoftirqd/1:   19] [<ffffff80090351d0>] tcp_v4_err+0x4b0/0x4bc
<0>[60392.955372] I[1:    ksoftirqd/1:   19] [<ffffff800904a990>] icmp_socket_deliver+0x70/0xc0
<0>[60392.955380] I[1:    ksoftirqd/1:   19] [<ffffff800904a6d0>] icmp_unreach+0x1b4/0x1f0
<0>[60392.955389] I[1:    ksoftirqd/1:   19] [<ffffff800904a3ac>] icmp_rcv+0x310/0x348
<0>[60392.955400] I[1:    ksoftirqd/1:   19] [<ffffff800900b104>] ip_local_deliver_finish+0xf8/0x1f0
<0>[60392.955409] I[1:    ksoftirqd/1:   19] [<ffffff800900afd4>] ip_local_deliver+0xc8/0x100
<0>[60392.955418] I[1:    ksoftirqd/1:   19] [<ffffff800900b790>] ip_rcv_finish+0x218/0x2e4
<0>[60392.955426] I[1:    ksoftirqd/1:   19] [<ffffff800900b4c0>] ip_rcv+0x2c4/0x37c
<0>[60392.955437] I[1:    ksoftirqd/1:   19] [<ffffff8008f84f98>] __netif_receive_skb_core+0x8f0/0xa68
<0>[60392.955446] I[1:    ksoftirqd/1:   19] [<ffffff8008f85960>] process_backlog+0x17c/0x290
<0>[60392.955455] I[1:    ksoftirqd/1:   19] [<ffffff8008f85ef0>] net_rx_action+0x224/0x464
<0>[60392.955465] I[1:    ksoftirqd/1:   19] [<ffffff80081b9564>] __do_softirq+0x200/0x39c
<0>[60392.955475] I[1:    ksoftirqd/1:   19] [<ffffff80081f1f04>] run_ksoftirqd+0x34/0x74
<0>[60392.955485] I[1:    ksoftirqd/1:   19] [<ffffff800821ad7c>] smpboot_thread_fn+0x1d0/0x2f8
<0>[60392.955495] I[1:    ksoftirqd/1:   19] [<ffffff8008215c48>] kthread+0x11c/0x130
<0>[60392.955505] I[1:    ksoftirqd/1:   19] [<ffffff80081bdaa0>] ret_from_fork+0x10/0x1c
<0>[60392.955515] I[1:    ksoftirqd/1:   19] Code: f9402d08 94063788 aa1603e8 17ffff80 (d4210000) 
<4>[60392.955524] I[1:    ksoftirqd/1:   19] ---[ end trace c1d9b9267f6ef3c2 ]---


^ permalink raw reply

* [PATCH v2 perf,bpf 00/11] perf annotation of BPF programs
From: Song Liu @ 2019-02-14 23:56 UTC (permalink / raw)
  To: linux-kernel, netdev
  Cc: acme, ast, daniel, jolsa, kernel-team, namhyung, peterz, Song Liu

Changes v1 to v2:
1. Fix compilation error with different feature-disassembler-four-args;
2. Fix a segfault in perf-record;
3. Split patches 5/9 and 6/9 so that perf_env changes and perf.data changes
   are in separate patches.

This series enables annotation of BPF programs in perf.

perf tool gathers information via sys_bpf and (optionally) stores them in
perf.data as headers.

Patch 1/11 fixes a minor issue in kernel;
Patch 2/11 to 4/11 introduce new helper functions and use them in perf and
     bpftool;
Patch 5/11 to 8/11 saves information of bpf program in perf_env;
Patch 9/11 adds --bpf-event options to perf-top;
Patch 10/11 enables annotation of bpf programs based on information
     gathered in 5/11 to 8/11;
Patch 11/11 handles information of short living BPF program that are loaded
     during perf-record or perf-top.

Commands tested during developments are perf-top, perf-record, perf-report,
and perf-annotate.

===================== Note on patch dependency  ========================
This set has dependency in both bpf-next tree and tip/perf/core. Current
version is developed on bpf-next tree with the following commits
cherry-picked from tip/perf/core:

(from 1/10 to 10/10)
commit 76193a94522f ("perf, bpf: Introduce PERF_RECORD_KSYMBOL")
commit d764ac646491 ("tools headers uapi: Sync tools/include/uapi/linux/perf_event.h")
commit 6ee52e2a3fe4 ("perf, bpf: Introduce PERF_RECORD_BPF_EVENT")
commit df063c83aa2c ("tools headers uapi: Sync tools/include/uapi/linux/perf_event.h")
commit 9aa0bfa370b2 ("perf tools: Handle PERF_RECORD_KSYMBOL")
commit 45178a928a4b ("perf tools: Handle PERF_RECORD_BPF_EVENT")
commit 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs")
commit a40b95bcd30c ("perf top: Synthesize BPF events for pre-existing loaded BPF programs")
commit 6934058d9fb6 ("bpf: Add module name [bpf] to ksymbols for bpf programs")
commit 811184fb6977 ("perf bpf: Fix synthesized PERF_RECORD_KSYMBOL/BPF_EVENT")
========================================================================

This set is also available at:

https://github.com/liu-song-6/linux/tree/bpf-annotation

Thanks!!

Song Liu (11):
  perf, bpf: consider events with attr.bpf_event as side-band events
  bpf: libbpf: introduce bpf_program__get_prog_info_linear()
  bpf: bpftool: use bpf_program__get_prog_info_linear() in
    prog.c:do_dump()
  perf, bpf: synthesize bpf events with
    bpf_program__get_prog_info_linear()
  perf, bpf: save bpf_prog_info in a rbtree in perf_env
  perf, bpf: save bpf_prog_info information as headers to perf.data
  perf, bpf: save btf in a rbtree in perf_env
  perf, bpf: save btf information as headers to perf.data
  perf-top: add option --bpf-event
  perf, bpf: enable annotation of bpf program
  perf, bpf: save information about short living bpf programs

 kernel/events/core.c        |   3 +-
 tools/bpf/bpftool/prog.c    | 266 ++++++---------------------
 tools/lib/bpf/libbpf.c      | 251 +++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h      |  63 +++++++
 tools/lib/bpf/libbpf.map    |   3 +
 tools/perf/Makefile.config  |   6 +-
 tools/perf/builtin-record.c |  15 +-
 tools/perf/builtin-top.c    |  15 +-
 tools/perf/util/annotate.c  | 149 ++++++++++++++-
 tools/perf/util/bpf-event.c | 354 +++++++++++++++++++++++++++---------
 tools/perf/util/bpf-event.h |  48 ++++-
 tools/perf/util/dso.c       |   1 +
 tools/perf/util/dso.h       |  33 ++--
 tools/perf/util/env.c       | 148 +++++++++++++++
 tools/perf/util/env.h       |  17 ++
 tools/perf/util/evlist.c    |  20 ++
 tools/perf/util/evlist.h    |   2 +
 tools/perf/util/header.c    | 231 ++++++++++++++++++++++-
 tools/perf/util/header.h    |   2 +
 tools/perf/util/symbol.c    |   1 +
 20 files changed, 1316 insertions(+), 312 deletions(-)

--
2.17.1

^ permalink raw reply

* [PATCH v2 perf,bpf 01/11] perf, bpf: consider events with attr.bpf_event as side-band events
From: Song Liu @ 2019-02-14 23:58 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190214235624.2579307-1-songliubraving@fb.com>

Events with bpf_event should be considered as side-band event, as they
carry information about BPF programs.

Fixes: 6ee52e2a3fe4 ("perf, bpf: Introduce PERF_RECORD_BPF_EVENT")
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 kernel/events/core.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/kernel/events/core.c b/kernel/events/core.c
index 0a8dab322111..9403bdda5f8c 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -4238,7 +4238,8 @@ static bool is_sb_event(struct perf_event *event)
 	if (attr->mmap || attr->mmap_data || attr->mmap2 ||
 	    attr->comm || attr->comm_exec ||
 	    attr->task || attr->ksymbol ||
-	    attr->context_switch)
+	    attr->context_switch ||
+	    attr->bpf_event)
 		return true;
 	return false;
 }
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 04/11] perf, bpf: synthesize bpf events with bpf_program__get_prog_info_linear()
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

With bpf_program__get_prog_info_linear, we can simplify the logic that
synthesizes bpf events.

This patch doesn't change the behavior of the code.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/util/bpf-event.c | 118 ++++++++++++------------------------
 1 file changed, 40 insertions(+), 78 deletions(-)

diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index 796ef793f4ce..e6dfb95029e5 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -3,7 +3,9 @@
 #include <stdlib.h>
 #include <bpf/bpf.h>
 #include <bpf/btf.h>
+#include <bpf/libbpf.h>
 #include <linux/btf.h>
+#include <linux/err.h>
 #include "bpf-event.h"
 #include "debug.h"
 #include "symbol.h"
@@ -49,99 +51,62 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 {
 	struct ksymbol_event *ksymbol_event = &event->ksymbol_event;
 	struct bpf_event *bpf_event = &event->bpf_event;
-	u32 sub_prog_cnt, i, func_info_rec_size = 0;
-	u8 (*prog_tags)[BPF_TAG_SIZE] = NULL;
-	struct bpf_prog_info info = { .type = 0, };
-	u32 info_len = sizeof(info);
-	void *func_infos = NULL;
-	u64 *prog_addrs = NULL;
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_info *info;
 	struct btf *btf = NULL;
-	u32 *prog_lens = NULL;
 	bool has_btf = false;
-	char errbuf[512];
+	u32 sub_prog_cnt, i;
 	int err = 0;
+	u64 arrays;
 
-	/* Call bpf_obj_get_info_by_fd() to get sizes of arrays */
-	err = bpf_obj_get_info_by_fd(fd, &info, &info_len);
+	arrays = 1UL << BPF_PROG_INFO_JITED_KSYMS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_FUNC_LENS;
+	arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_PROG_TAGS;
 
-	if (err) {
-		pr_debug("%s: failed to get BPF program info: %s, aborting\n",
-			 __func__, str_error_r(errno, errbuf, sizeof(errbuf)));
+	info_linear = bpf_program__get_prog_info_linear(fd, arrays);
+	if (IS_ERR_OR_NULL(info_linear)) {
+		info_linear = NULL;
+		pr_debug("%s: failed to get BPF program info. aborting\n", __func__);
 		return -1;
 	}
-	if (info_len < offsetof(struct bpf_prog_info, prog_tags)) {
+
+	if (info_linear->info_len < offsetof(struct bpf_prog_info, prog_tags)) {
 		pr_debug("%s: the kernel is too old, aborting\n", __func__);
 		return -2;
 	}
 
+	info = &info_linear->info;
+
 	/* number of ksyms, func_lengths, and tags should match */
-	sub_prog_cnt = info.nr_jited_ksyms;
-	if (sub_prog_cnt != info.nr_prog_tags ||
-	    sub_prog_cnt != info.nr_jited_func_lens)
+	sub_prog_cnt = info->nr_jited_ksyms;
+	if (sub_prog_cnt != info->nr_prog_tags ||
+	    sub_prog_cnt != info->nr_jited_func_lens)
 		return -1;
 
 	/* check BTF func info support */
-	if (info.btf_id && info.nr_func_info && info.func_info_rec_size) {
+	if (info->btf_id && info->nr_func_info && info->func_info_rec_size) {
 		/* btf func info number should be same as sub_prog_cnt */
-		if (sub_prog_cnt != info.nr_func_info) {
+		if (sub_prog_cnt != info->nr_func_info) {
 			pr_debug("%s: mismatch in BPF sub program count and BTF function info count, aborting\n", __func__);
-			return -1;
-		}
-		if (btf__get_from_id(info.btf_id, &btf)) {
-			pr_debug("%s: failed to get BTF of id %u, aborting\n", __func__, info.btf_id);
-			return -1;
+			err = -1;
+			goto out;
 		}
-		func_info_rec_size = info.func_info_rec_size;
-		func_infos = calloc(sub_prog_cnt, func_info_rec_size);
-		if (!func_infos) {
-			pr_debug("%s: failed to allocate memory for func_infos, aborting\n", __func__);
-			return -1;
+		if (btf__get_from_id(info->btf_id, &btf)) {
+			pr_debug("%s: failed to get BTF of id %u, aborting\n", __func__, info->btf_id);
+			err = -1;
+			btf = NULL;
+			goto out;
 		}
 		has_btf = true;
 	}
 
-	/*
-	 * We need address, length, and tag for each sub program.
-	 * Allocate memory and call bpf_obj_get_info_by_fd() again
-	 */
-	prog_addrs = calloc(sub_prog_cnt, sizeof(u64));
-	if (!prog_addrs) {
-		pr_debug("%s: failed to allocate memory for prog_addrs, aborting\n", __func__);
-		goto out;
-	}
-	prog_lens = calloc(sub_prog_cnt, sizeof(u32));
-	if (!prog_lens) {
-		pr_debug("%s: failed to allocate memory for prog_lens, aborting\n", __func__);
-		goto out;
-	}
-	prog_tags = calloc(sub_prog_cnt, BPF_TAG_SIZE);
-	if (!prog_tags) {
-		pr_debug("%s: failed to allocate memory for prog_tags, aborting\n", __func__);
-		goto out;
-	}
-
-	memset(&info, 0, sizeof(info));
-	info.nr_jited_ksyms = sub_prog_cnt;
-	info.nr_jited_func_lens = sub_prog_cnt;
-	info.nr_prog_tags = sub_prog_cnt;
-	info.jited_ksyms = ptr_to_u64(prog_addrs);
-	info.jited_func_lens = ptr_to_u64(prog_lens);
-	info.prog_tags = ptr_to_u64(prog_tags);
-	info_len = sizeof(info);
-	if (has_btf) {
-		info.nr_func_info = sub_prog_cnt;
-		info.func_info_rec_size = func_info_rec_size;
-		info.func_info = ptr_to_u64(func_infos);
-	}
-
-	err = bpf_obj_get_info_by_fd(fd, &info, &info_len);
-	if (err) {
-		pr_debug("%s: failed to get BPF program info, aborting\n", __func__);
-		goto out;
-	}
-
 	/* Synthesize PERF_RECORD_KSYMBOL */
 	for (i = 0; i < sub_prog_cnt; i++) {
+		u8 (*prog_tags)[BPF_TAG_SIZE] = (void *)(info->prog_tags);
+		__u32 *prog_lens = (__u32 *)(info->jited_func_lens);
+		__u64 *prog_addrs = (__u64 *)(info->jited_ksyms);
+		void *func_infos = (void *)(info->func_info);
 		const struct bpf_func_info *finfo;
 		const char *short_name = NULL;
 		const struct btf_type *t;
@@ -163,13 +128,13 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 					 KSYM_NAME_LEN - name_len,
 					 prog_tags[i], BPF_TAG_SIZE);
 		if (has_btf) {
-			finfo = func_infos + i * info.func_info_rec_size;
+			finfo = func_infos + i * info->func_info_rec_size;
 			t = btf__type_by_id(btf, finfo->type_id);
 			short_name = btf__name_by_offset(btf, t->name_off);
 		} else if (i == 0 && sub_prog_cnt == 1) {
 			/* no subprog */
-			if (info.name[0])
-				short_name = info.name;
+			if (info->name[0])
+				short_name = info->name;
 		} else
 			short_name = "F";
 		if (short_name)
@@ -195,9 +160,9 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 			},
 			.type = PERF_BPF_EVENT_PROG_LOAD,
 			.flags = 0,
-			.id = info.id,
+			.id = info->id,
 		};
-		memcpy(bpf_event->tag, prog_tags[i], BPF_TAG_SIZE);
+		memcpy(bpf_event->tag, info->tag, BPF_TAG_SIZE);
 		memset((void *)event + event->header.size, 0, machine->id_hdr_size);
 		event->header.size += machine->id_hdr_size;
 		err = perf_tool__process_synth_event(tool, event,
@@ -205,10 +170,7 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 	}
 
 out:
-	free(prog_tags);
-	free(prog_lens);
-	free(prog_addrs);
-	free(func_infos);
+	free(info_linear);
 	free(btf);
 	return err ? -1 : 0;
 }
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 08/11] perf, bpf: save btf information as headers to perf.data
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

This patch enables perf-record to save btf information as headers to
perf.data A new header type HEADER_BTF is introduced for this data.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/util/header.c | 99 +++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/header.h |  1 +
 2 files changed, 99 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index 2ae76a9d06f6..3f1562afe8e5 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -1125,6 +1125,45 @@ static int write_bpf_prog_info(struct feat_fd *ff,
 	return ret;
 }
 
+static int write_btf(struct feat_fd *ff,
+		     struct perf_evlist *evlist __maybe_unused)
+{
+	struct perf_env *env = &ff->ph->env;
+	struct rb_root *root;
+	struct rb_node *next;
+	u32 count = 0;
+	int ret;
+
+	down_read(&env->bpf_info_lock);
+
+	root = &env->btfs;
+	next = rb_first(root);
+	while (next) {
+		++count;
+		next = rb_next(next);
+	}
+
+	ret = do_write(ff, &count, sizeof(count));
+
+	if (ret < 0)
+		goto out;
+
+	next = rb_first(root);
+	while (next) {
+		struct btf_node *node;
+
+		node = rb_entry(next, struct btf_node, rb_node);
+		next = rb_next(&node->rb_node);
+		ret = do_write(ff, node,
+			       sizeof(struct btf_node) + node->data_size);
+		if (ret < 0)
+			goto out;
+	}
+out:
+	up_read(&env->bpf_info_lock);
+	return ret;
+}
+
 static int cpu_cache_level__sort(const void *a, const void *b)
 {
 	struct cpu_cache_level *cache_a = (struct cpu_cache_level *)a;
@@ -1628,6 +1667,28 @@ static void print_bpf_prog_info(struct feat_fd *ff, FILE *fp)
 	up_read(&env->bpf_info_lock);
 }
 
+static void print_btf(struct feat_fd *ff, FILE *fp)
+{
+	struct perf_env *env = &ff->ph->env;
+	struct rb_root *root;
+	struct rb_node *next;
+
+	down_read(&env->bpf_info_lock);
+
+	root = &env->btfs;
+	next = rb_first(root);
+
+	while (next) {
+		struct btf_node *node;
+
+		node = rb_entry(next, struct btf_node, rb_node);
+		next = rb_next(&node->rb_node);
+		fprintf(fp, "# bpf_prog_info of id %u\n", node->id);
+	}
+
+	up_read(&env->bpf_info_lock);
+}
+
 static void free_event_desc(struct perf_evsel *events)
 {
 	struct perf_evsel *evsel;
@@ -2723,6 +2784,41 @@ static int process_bpf_prog_info(struct feat_fd *ff,
 	return err;
 }
 
+static int process_btf(struct feat_fd *ff, void *data __maybe_unused)
+{
+	struct perf_env *env = &ff->ph->env;
+	u32 count, i;
+
+	if (do_read_u32(ff, &count))
+		return -1;
+
+	down_write(&env->bpf_info_lock);
+
+	for (i = 0; i < count; ++i) {
+		struct btf_node btf_node;
+		struct btf_node *node;
+
+		if (__do_read(ff, &btf_node, sizeof(struct btf_node)))
+			return -1;
+
+		node = malloc(sizeof(struct btf_node) + btf_node.data_size);
+		if (!node)
+			return -1;
+
+		node->id = btf_node.id;
+		node->data_size = btf_node.data_size;
+
+		if (__do_read(ff, node->data, btf_node.data_size)) {
+			free(node);
+			return -1;
+		}
+		perf_env__insert_btf(env, node);
+	}
+
+	up_write(&env->bpf_info_lock);
+	return 0;
+}
+
 struct feature_ops {
 	int (*write)(struct feat_fd *ff, struct perf_evlist *evlist);
 	void (*print)(struct feat_fd *ff, FILE *fp);
@@ -2783,7 +2879,8 @@ static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
 	FEAT_OPR(SAMPLE_TIME,	sample_time,	false),
 	FEAT_OPR(MEM_TOPOLOGY,	mem_topology,	true),
 	FEAT_OPR(CLOCKID,       clockid,        false),
-	FEAT_OPR(BPF_PROG_INFO, bpf_prog_info,  false)
+	FEAT_OPR(BPF_PROG_INFO, bpf_prog_info,  false),
+	FEAT_OPR(BTF,           btf,            false)
 };
 
 struct header_print_data {
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index 0785c91b4c3a..ba51d8e43c53 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -40,6 +40,7 @@ enum {
 	HEADER_MEM_TOPOLOGY,
 	HEADER_CLOCKID,
 	HEADER_BPF_PROG_INFO,
+	HEADER_BTF,
 	HEADER_LAST_FEATURE,
 	HEADER_FEAT_BITS	= 256,
 };
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 07/11] perf, bpf: save btf in a rbtree in perf_env
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

btf contains information necessary to annotate bpf programs. This patch
saves btf for bpf programs loaded in the system.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/util/bpf-event.c | 22 +++++++++++++
 tools/perf/util/bpf-event.h |  7 ++++
 tools/perf/util/env.c       | 65 +++++++++++++++++++++++++++++++++++++
 tools/perf/util/env.h       |  3 ++
 4 files changed, 97 insertions(+)

diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index ead599bc4f4e..37a5b8134e00 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -30,6 +30,27 @@ int machine__process_bpf_event(struct machine *machine __maybe_unused,
 	return 0;
 }
 
+static int perf_fetch_btf(struct perf_env *env, u32 btf_id, struct btf *btf)
+{
+	struct btf_node *node;
+	u32 data_size;
+	const void *data;
+
+	data = btf__get_raw_data(btf, &data_size);
+
+	node = malloc(data_size + sizeof(struct btf_node));
+
+	if (!node)
+		return -1;
+
+	node->id = btf_id;
+	node->data_size = data_size;
+	memcpy(node->data, data, data_size);
+
+	perf_env__insert_btf(env, node);
+	return 0;
+}
+
 /*
  * Synthesize PERF_RECORD_KSYMBOL and PERF_RECORD_BPF_EVENT for one bpf
  * program. One PERF_RECORD_BPF_EVENT is generated for the program. And
@@ -109,6 +130,7 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_session *session,
 			goto out;
 		}
 		has_btf = true;
+		perf_fetch_btf(env, info->btf_id, btf);
 	}
 
 	/* Synthesize PERF_RECORD_KSYMBOL */
diff --git a/tools/perf/util/bpf-event.h b/tools/perf/util/bpf-event.h
index 11e6730b6105..60ce24e4e5c6 100644
--- a/tools/perf/util/bpf-event.h
+++ b/tools/perf/util/bpf-event.h
@@ -20,6 +20,13 @@ struct bpf_prog_info_node {
 	struct rb_node			rb_node;
 };
 
+struct btf_node {
+	struct rb_node	rb_node;
+	u32		id;
+	u32		data_size;
+	char		data[];
+};
+
 #ifdef HAVE_LIBBPF_SUPPORT
 int machine__process_bpf_event(struct machine *machine, union perf_event *event,
 			       struct perf_sample *sample);
diff --git a/tools/perf/util/env.c b/tools/perf/util/env.c
index 665b6fe3c7b2..6f9e3d4b94bc 100644
--- a/tools/perf/util/env.c
+++ b/tools/perf/util/env.c
@@ -61,6 +61,57 @@ struct bpf_prog_info_node *perf_env__find_bpf_prog_info(struct perf_env *env,
 	return node;
 }
 
+void perf_env__insert_btf(struct perf_env *env, struct btf_node *btf_node)
+{
+	struct rb_node *parent = NULL;
+	__u32 btf_id = btf_node->id;
+	struct btf_node *node;
+	struct rb_node **p;
+
+	down_write(&env->bpf_info_lock);
+	p = &env->btfs.rb_node;
+
+	while (*p != NULL) {
+		parent = *p;
+		node = rb_entry(parent, struct btf_node, rb_node);
+		if (btf_id < node->id) {
+			p = &(*p)->rb_left;
+		} else if (btf_id > node->id) {
+			p = &(*p)->rb_right;
+		} else {
+			pr_debug("duplicated btf %u\n", btf_id);
+			up_write(&env->bpf_info_lock);
+			return;
+		}
+	}
+
+	rb_link_node(&btf_node->rb_node, parent, p);
+	rb_insert_color(&btf_node->rb_node, &env->btfs);
+	up_write(&env->bpf_info_lock);
+}
+
+struct btf_node *perf_env__find_btf(struct perf_env *env, __u32 btf_id)
+{
+	struct btf_node *node = NULL;
+	struct rb_node *n;
+
+	down_read(&env->bpf_info_lock);
+	n = env->btfs.rb_node;
+
+	while (n) {
+		node = rb_entry(n, struct btf_node, rb_node);
+		if (btf_id < node->id)
+			n = n->rb_left;
+		else if (btf_id > node->id)
+			n = n->rb_right;
+		else
+			break;
+	}
+
+	up_read(&env->bpf_info_lock);
+	return node;
+}
+
 /* purge data in bpf_prog_infos tree */
 static void purge_bpf_info(struct perf_env *env)
 {
@@ -80,6 +131,19 @@ static void purge_bpf_info(struct perf_env *env)
 		rb_erase_init(&node->rb_node, root);
 		free(node);
 	}
+
+	root = &env->btfs;
+	next = rb_first(root);
+
+	while (next) {
+		struct btf_node *node;
+
+		node = rb_entry(next, struct btf_node, rb_node);
+		next = rb_next(&node->rb_node);
+		rb_erase_init(&node->rb_node, root);
+		free(node);
+	}
+
 	up_write(&env->bpf_info_lock);
 }
 
@@ -117,6 +181,7 @@ void perf_env__exit(struct perf_env *env)
 static void init_bpf_rb_trees(struct perf_env *env)
 {
 	env->bpf_prog_infos = RB_ROOT;
+	env->btfs = RB_ROOT;
 	init_rwsem(&env->bpf_info_lock);
 }
 
diff --git a/tools/perf/util/env.h b/tools/perf/util/env.h
index a6d25e91bc98..f0c74255423c 100644
--- a/tools/perf/util/env.h
+++ b/tools/perf/util/env.h
@@ -74,6 +74,7 @@ struct perf_env {
 	 */
 	struct rw_semaphore	bpf_info_lock;
 	struct rb_root		bpf_prog_infos;
+	struct rb_root		btfs;
 };
 
 extern struct perf_env perf_env;
@@ -94,4 +95,6 @@ void perf_env__insert_bpf_prog_info(struct perf_env *env,
 				    struct bpf_prog_info_node *info_node);
 struct bpf_prog_info_node *perf_env__find_bpf_prog_info(struct perf_env *env,
 							__u32 prog_id);
+void perf_env__insert_btf(struct perf_env *env, struct btf_node *btf_node);
+struct btf_node *perf_env__find_btf(struct perf_env *env, __u32 btf_id);
 #endif /* __PERF_ENV_H */
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 09/11] perf-top: add option --bpf-event
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

bpf events are only tracked when opts->bpf_event is enabled. This patch
adds command line flag to enable this for perf-top.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/builtin-top.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 27d8d42e0a4d..5271d7211b9c 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1492,6 +1492,7 @@ int cmd_top(int argc, const char **argv)
 		    "Display raw encoding of assembly instructions (default)"),
 	OPT_BOOLEAN(0, "demangle-kernel", &symbol_conf.demangle_kernel,
 		    "Enable kernel symbol demangling"),
+	OPT_BOOLEAN(0, "bpf-event", &opts->bpf_event, "record bpf events"),
 	OPT_STRING(0, "objdump", &top.annotation_opts.objdump_path, "path",
 		    "objdump binary to use for disassembly and annotations"),
 	OPT_STRING('M', "disassembler-style", &top.annotation_opts.disassembler_style, "disassembler style",
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 11/11] perf, bpf: save information about short living bpf programs
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000045.2592135-1-songliubraving@fb.com>

To annotate bpf programs in perf, it is necessary to save information in
bpf_prog_info and btf. For short living bpf program, it is necessary to
save these information before it is unloaded.

This patch saves these information in a separate thread. This thread
creates its own evlist, that only tracks bpf events. This evlists uses
ring buffer with very low watermark for lower latency. When bpf load
events are received, this thread tries to gather information via sys_bpf
and save it in perf_env.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/builtin-record.c |  13 ++++
 tools/perf/builtin-top.c    |  12 ++++
 tools/perf/util/bpf-event.c | 129 ++++++++++++++++++++++++++++++++++++
 tools/perf/util/bpf-event.h |  22 ++++++
 tools/perf/util/evlist.c    |  20 ++++++
 tools/perf/util/evlist.h    |   2 +
 6 files changed, 198 insertions(+)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 2355e0a9eda0..46abb44aaaab 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1106,6 +1106,8 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 	struct perf_data *data = &rec->data;
 	struct perf_session *session;
 	bool disabled = false, draining = false;
+	struct bpf_event_poll_args poll_args;
+	bool bpf_thread_running = false;
 	int fd;
 
 	atexit(record__sig_exit);
@@ -1206,6 +1208,14 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 		goto out_child;
 	}
 
+	if (rec->opts.bpf_event) {
+		poll_args.env = &session->header.env;
+		poll_args.target = &rec->opts.target;
+		poll_args.done = &done;
+		if (bpf_event__start_polling_thread(&poll_args) == 0)
+			bpf_thread_running = true;
+	}
+
 	err = record__synthesize(rec, false);
 	if (err < 0)
 		goto out_child;
@@ -1456,6 +1466,9 @@ static int __cmd_record(struct record *rec, int argc, const char **argv)
 
 out_delete_session:
 	perf_session__delete(session);
+
+	if (bpf_thread_running)
+		bpf_event__stop_polling_thread(&poll_args);
 	return status;
 }
 
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 5271d7211b9c..2586ee081967 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1524,10 +1524,12 @@ int cmd_top(int argc, const char **argv)
 			"number of thread to run event synthesize"),
 	OPT_END()
 	};
+	struct bpf_event_poll_args poll_args;
 	const char * const top_usage[] = {
 		"perf top [<options>]",
 		NULL
 	};
+	bool bpf_thread_running = false;
 	int status = hists__init();
 
 	if (status < 0)
@@ -1652,8 +1654,18 @@ int cmd_top(int argc, const char **argv)
 		signal(SIGWINCH, winch_sig);
 	}
 
+	if (top.record_opts.bpf_event) {
+		poll_args.env = &perf_env;
+		poll_args.target = target;
+		poll_args.done = &done;
+		if (bpf_event__start_polling_thread(&poll_args) == 0)
+			bpf_thread_running = true;
+	}
 	status = __cmd_top(&top);
 
+	if (bpf_thread_running)
+		bpf_event__stop_polling_thread(&poll_args);
+
 out_delete_evlist:
 	perf_evlist__delete(top.evlist);
 
diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index 4f347d61ed96..0caf137c515b 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -8,6 +8,7 @@
 #include "machine.h"
 #include "env.h"
 #include "session.h"
+#include "evlist.h"
 
 #define ptr_to_u64(ptr)    ((__u64)(unsigned long)(ptr))
 
@@ -316,3 +317,131 @@ int perf_event__synthesize_bpf_events(struct perf_session *session,
 	free(event);
 	return err;
 }
+
+static void perf_env_add_bpf_info(struct perf_env *env, u32 id)
+{
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_info_node *info_node;
+	struct btf *btf = NULL;
+	u64 arrays;
+	u32 btf_id;
+	int fd;
+
+	fd = bpf_prog_get_fd_by_id(id);
+	if (fd < 0)
+		return;
+
+	arrays = 1UL << BPF_PROG_INFO_JITED_KSYMS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_FUNC_LENS;
+	arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_PROG_TAGS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_INSNS;
+	arrays |= 1UL << BPF_PROG_INFO_LINE_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_LINE_INFO;
+
+	info_linear = bpf_program__get_prog_info_linear(fd, arrays);
+	if (IS_ERR_OR_NULL(info_linear)) {
+		pr_debug("%s: failed to get BPF program info. aborting\n", __func__);
+		goto out;
+	}
+
+	btf_id = info_linear->info.btf_id;
+
+	info_node = malloc(sizeof(struct bpf_prog_info_node));
+	if (info_node) {
+		info_node->info_linear = info_linear;
+		perf_env__insert_bpf_prog_info(env, info_node);
+	} else
+		free(info_linear);
+
+	if (btf_id == 0)
+		goto out;
+
+	if (btf__get_from_id(btf_id, &btf)) {
+		pr_debug("%s: failed to get BTF of id %u, aborting\n",
+			 __func__, btf_id);
+		goto out;
+	}
+	perf_fetch_btf(env, btf_id, btf);
+
+out:
+	free(btf);
+	close(fd);
+}
+
+static void *bpf_poll_thread(void *arg)
+{
+	struct bpf_event_poll_args *args = arg;
+	int i;
+
+	while (!*(args->done)) {
+		perf_evlist__poll(args->evlist, 1000);
+
+		for (i = 0; i < args->evlist->nr_mmaps; i++) {
+			struct perf_mmap *map = &args->evlist->mmap[i];
+			union perf_event *event;
+
+			if (perf_mmap__read_init(map))
+				continue;
+			while ((event = perf_mmap__read_event(map)) != NULL) {
+				pr_debug("processing vip event of type %d\n",
+					 event->header.type);
+				switch (event->header.type) {
+				case PERF_RECORD_BPF_EVENT:
+					if (event->bpf_event.type != PERF_BPF_EVENT_PROG_LOAD)
+						break;
+					perf_env_add_bpf_info(args->env, event->bpf_event.id);
+					break;
+				default:
+					break;
+				}
+				perf_mmap__consume(map);
+			}
+			perf_mmap__read_done(map);
+		}
+	}
+	return NULL;
+}
+
+pthread_t poll_thread;
+
+int bpf_event__start_polling_thread(struct bpf_event_poll_args *args)
+{
+	struct perf_evsel *counter;
+
+	args->evlist = perf_evlist__new();
+
+	if (args->evlist == NULL)
+		return -1;
+
+	if (perf_evlist__create_maps(args->evlist, args->target))
+		return -1;
+
+	if (perf_evlist__add_bpf_tracker(args->evlist))
+		return -1;
+
+	evlist__for_each_entry(args->evlist, counter) {
+		if (perf_evsel__open(counter, args->evlist->cpus,
+				     args->evlist->threads) < 0)
+			return -1;
+	}
+
+	if (perf_evlist__mmap(args->evlist, UINT_MAX))
+		return -1;
+
+	evlist__for_each_entry(args->evlist, counter) {
+		if (perf_evsel__enable(counter))
+			return -1;
+	}
+
+	if (pthread_create(&poll_thread, NULL, bpf_poll_thread, args))
+		return -1;
+
+	return 0;
+}
+
+void bpf_event__stop_polling_thread(struct bpf_event_poll_args *args)
+{
+	pthread_join(poll_thread, NULL);
+	perf_evlist__exit(args->evlist);
+}
diff --git a/tools/perf/util/bpf-event.h b/tools/perf/util/bpf-event.h
index c4f0f1395ea5..61914827c1e3 100644
--- a/tools/perf/util/bpf-event.h
+++ b/tools/perf/util/bpf-event.h
@@ -12,12 +12,17 @@
 #include <bpf/libbpf.h>
 #include <linux/btf.h>
 #include <linux/rbtree.h>
+#include <pthread.h>
+#include <api/fd/array.h>
 #include "event.h"
 
 struct machine;
 union perf_event;
+struct perf_env;
 struct perf_sample;
 struct record_opts;
+struct evlist;
+struct target;
 
 struct bpf_prog_info_node {
 	struct bpf_prog_info_linear	*info_linear;
@@ -31,6 +36,13 @@ struct btf_node {
 	char		data[];
 };
 
+struct bpf_event_poll_args {
+	struct perf_env		*env;
+	struct perf_evlist	*evlist;
+	struct target		*target;
+	volatile int		*done;
+};
+
 #ifdef HAVE_LIBBPF_SUPPORT
 int machine__process_bpf_event(struct machine *machine, union perf_event *event,
 			       struct perf_sample *sample);
@@ -39,6 +51,8 @@ int perf_event__synthesize_bpf_events(struct perf_session *session,
 				      perf_event__handler_t process,
 				      struct machine *machine,
 				      struct record_opts *opts);
+int bpf_event__start_polling_thread(struct bpf_event_poll_args *args);
+void bpf_event__stop_polling_thread(struct bpf_event_poll_args *args);
 #else
 static inline int machine__process_bpf_event(struct machine *machine __maybe_unused,
 					     union perf_event *event __maybe_unused,
@@ -54,5 +68,13 @@ static inline int perf_event__synthesize_bpf_events(struct perf_session *session
 {
 	return 0;
 }
+
+static inline int bpf_event__start_polling_thread(struct bpf_event_poll_args *args __maybe_unused)
+{
+	return 0;
+}
+void bpf_event__stop_polling_thread(struct bpf_event_poll_args *args __maybe_unused)
+{
+}
 #endif // HAVE_LIBBPF_SUPPORT
 #endif
diff --git a/tools/perf/util/evlist.c b/tools/perf/util/evlist.c
index 8c902276d4b4..612c079579ce 100644
--- a/tools/perf/util/evlist.c
+++ b/tools/perf/util/evlist.c
@@ -271,6 +271,26 @@ int perf_evlist__add_dummy(struct perf_evlist *evlist)
 	return 0;
 }
 
+int perf_evlist__add_bpf_tracker(struct perf_evlist *evlist)
+{
+	struct perf_event_attr attr = {
+		.type	          = PERF_TYPE_SOFTWARE,
+		.config           = PERF_COUNT_SW_DUMMY,
+		.watermark        = 1,
+		.bpf_event        = 1,
+		.wakeup_watermark = 1,
+		.size	   = sizeof(attr), /* to capture ABI version */
+	};
+	struct perf_evsel *evsel = perf_evsel__new_idx(&attr,
+						       evlist->nr_entries);
+
+	if (evsel == NULL)
+		return -ENOMEM;
+
+	perf_evlist__add(evlist, evsel);
+	return 0;
+}
+
 static int perf_evlist__add_attrs(struct perf_evlist *evlist,
 				  struct perf_event_attr *attrs, size_t nr_attrs)
 {
diff --git a/tools/perf/util/evlist.h b/tools/perf/util/evlist.h
index 868294491194..a2d22715188e 100644
--- a/tools/perf/util/evlist.h
+++ b/tools/perf/util/evlist.h
@@ -84,6 +84,8 @@ int __perf_evlist__add_default_attrs(struct perf_evlist *evlist,
 
 int perf_evlist__add_dummy(struct perf_evlist *evlist);
 
+int perf_evlist__add_bpf_tracker(struct perf_evlist *evlist);
+
 int perf_evlist__add_newtp(struct perf_evlist *evlist,
 			   const char *sys, const char *name, void *handler);
 
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 10/11] perf, bpf: enable annotation of bpf program
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190214235624.2579307-1-songliubraving@fb.com>

This patch enables the annotation of bpf program.

A new dso type DSO_BINARY_TYPE__BPF_PROG_INFO is introduced to for BPF
programs. In symbol__disassemble(), DSO_BINARY_TYPE__BPF_PROG_INFO dso
calls into a new function symbol__disassemble_bpf(), where annotation
line information is filled based bpf_prog_info and btf saved in given
perf_env.

symbol__disassemble_bpf() uses libbfd to disassemble bpf programs.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/Makefile.config  |   6 +-
 tools/perf/util/annotate.c  | 149 +++++++++++++++++++++++++++++++++++-
 tools/perf/util/bpf-event.c |  48 ++++++++++++
 tools/perf/util/bpf-event.h |   4 +
 tools/perf/util/dso.c       |   1 +
 tools/perf/util/dso.h       |  33 +++++---
 tools/perf/util/symbol.c    |   1 +
 7 files changed, 229 insertions(+), 13 deletions(-)

diff --git a/tools/perf/Makefile.config b/tools/perf/Makefile.config
index b441c88cafa1..ab223239f1fb 100644
--- a/tools/perf/Makefile.config
+++ b/tools/perf/Makefile.config
@@ -701,7 +701,7 @@ else
 endif
 
 ifeq ($(feature-libbfd), 1)
-  EXTLIBS += -lbfd
+  EXTLIBS += -lbfd -lopcodes
 else
   # we are on a system that requires -liberty and (maybe) -lz
   # to link against -lbfd; test each case individually here
@@ -796,6 +796,10 @@ ifdef HAVE_KVM_STAT_SUPPORT
     CFLAGS += -DHAVE_KVM_STAT_SUPPORT
 endif
 
+ifeq ($(feature-disassembler-four-args), 1)
+    CFLAGS += -DDISASM_FOUR_ARGS_SIGNATURE
+endif
+
 ifeq (${IS_64_BIT}, 1)
   ifndef NO_PERF_READ_VDSO32
     $(call feature_check,compile-32)
diff --git a/tools/perf/util/annotate.c b/tools/perf/util/annotate.c
index 70de8f6b3aee..078017d31ca9 100644
--- a/tools/perf/util/annotate.c
+++ b/tools/perf/util/annotate.c
@@ -22,6 +22,7 @@
 #include "annotate.h"
 #include "evsel.h"
 #include "evlist.h"
+#include "bpf-event.h"
 #include "block-range.h"
 #include "string2.h"
 #include "arch/common.h"
@@ -29,6 +30,9 @@
 #include <pthread.h>
 #include <linux/bitops.h>
 #include <linux/kernel.h>
+#include <bfd.h>
+#include <dis-asm.h>
+#include <bpf/libbpf.h>
 
 /* FIXME: For the HE_COLORSET */
 #include "ui/browser.h"
@@ -1672,6 +1676,147 @@ static int dso__disassemble_filename(struct dso *dso, char *filename, size_t fil
 	return 0;
 }
 
+static void get_exec_path(char *tpath, size_t size)
+{
+	const char *path = "/proc/self/exe";
+	ssize_t len;
+
+	len = readlink(path, tpath, size - 1);
+	assert(len > 0);
+	tpath[len] = 0;
+}
+
+static int symbol__disassemble_bpf(struct symbol *sym,
+				   struct annotate_args *args)
+{
+	struct annotation *notes = symbol__annotation(sym);
+	struct annotation_options *opts = args->options;
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_linfo *prog_linfo = NULL;
+	struct bpf_prog_info_node *info_node;
+	int len = sym->end - sym->start;
+	disassembler_ftype disassemble;
+	struct map *map = args->ms.map;
+	struct disassemble_info info;
+	struct dso *dso = map->dso;
+	int pc = 0, count, sub_id;
+	struct btf *btf = NULL;
+	char tpath[PATH_MAX];
+	size_t buf_size;
+	int nr_skip = 0;
+	__u64 arrays;
+	char *buf;
+	bfd *bfdf;
+	FILE *s;
+
+	if (dso->binary_type != DSO_BINARY_TYPE__BPF_PROG_INFO)
+		return -1;
+
+	pr_debug("%s: handling sym %s addr %lx len %lx\n", __func__,
+		 sym->name, sym->start, sym->end - sym->start);
+
+	memset(tpath, 0, sizeof(tpath));
+	get_exec_path(tpath, sizeof(tpath));
+
+	bfdf = bfd_openr(tpath, NULL);
+	assert(bfdf);
+	assert(bfd_check_format(bfdf, bfd_object));
+
+	s = open_memstream(&buf, &buf_size);
+	init_disassemble_info(&info, s,
+			      (fprintf_ftype) fprintf);
+
+	info.arch = bfd_get_arch(bfdf);
+	info.mach = bfd_get_mach(bfdf);
+
+	arrays = 1UL << BPF_PROG_INFO_JITED_INSNS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_KSYMS;
+	arrays |= 1UL << BPF_PROG_INFO_LINE_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
+
+	info_node = perf_env__find_bpf_prog_info(dso->bpf_prog.env,
+						 dso->bpf_prog.id);
+	if (!info_node)
+		return -1;
+	info_linear = info_node->info_linear;
+	sub_id = dso->bpf_prog.sub_id;
+
+	info.buffer = (void *)(info_linear->info.jited_prog_insns);
+	info.buffer_length = info_linear->info.jited_prog_len;
+
+	if (info_linear->info.nr_line_info)
+		prog_linfo = bpf_prog_linfo__new(&info_linear->info);
+	prog_linfo = prog_linfo;
+
+	if (info_linear->info.btf_id) {
+		struct btf_node *node;
+
+		node = perf_env__find_btf(dso->bpf_prog.env,
+					  info_linear->info.btf_id);
+		if (node)
+			btf = btf__new((__u8 *)(node->data),
+				       node->data_size);
+	}
+
+	disassemble_init_for_target(&info);
+
+#ifdef DISASM_FOUR_ARGS_SIGNATURE
+	disassemble = disassembler(info.arch,
+				   bfd_big_endian(bfdf),
+				   info.mach,
+				   bfdf);
+#else
+	disassemble = disassembler(bfdf);
+#endif
+	assert(disassemble);
+
+	fflush(s);
+	do {
+		const struct bpf_line_info *linfo = NULL;
+		struct disasm_line *dl;
+		size_t prev_buf_size;
+		const char *srcline;
+		u64 addr;
+
+		addr = pc + ((u64 *)(info_linear->info.jited_ksyms))[sub_id];
+		count = disassemble(pc, &info);
+
+		linfo = bpf_prog_linfo__lfind_addr_func(prog_linfo, addr, sub_id,
+							nr_skip);
+
+		if (linfo) {
+			srcline = btf__name_by_offset(btf, linfo->line_off);
+			nr_skip++;
+		} else
+			srcline = NULL;
+
+		fprintf(s, "\n");
+		prev_buf_size = buf_size;
+		fflush(s);
+
+		if (!opts->hide_src_code && srcline) {
+			args->offset = -1;
+			args->line = strdup(srcline);
+			args->line_nr = 0;
+			args->ms.sym  = sym;
+			dl = disasm_line__new(args);
+			annotation_line__add(&dl->al, &notes->src->source);
+		}
+
+		args->offset = pc;
+		args->line = buf + prev_buf_size;
+		args->line_nr = 0;
+		args->ms.sym  = sym;
+		dl = disasm_line__new(args);
+		annotation_line__add(&dl->al, &notes->src->source);
+
+		pc += count;
+	} while (count > 0 && pc < len);
+
+	bfd_close(bfdf);
+	return 0;
+}
+
 static int symbol__disassemble(struct symbol *sym, struct annotate_args *args)
 {
 	struct annotation_options *opts = args->options;
@@ -1699,7 +1844,9 @@ static int symbol__disassemble(struct symbol *sym, struct annotate_args *args)
 	pr_debug("annotating [%p] %30s : [%p] %30s\n",
 		 dso, dso->long_name, sym, sym->name);
 
-	if (dso__is_kcore(dso)) {
+	if (dso->binary_type == DSO_BINARY_TYPE__BPF_PROG_INFO) {
+		return symbol__disassemble_bpf(sym, args);
+	} else if (dso__is_kcore(dso)) {
 		kce.kcore_filename = symfs_filename;
 		kce.addr = map__rip_2objdump(map, sym->start);
 		kce.offs = sym->start;
diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index 37a5b8134e00..4f347d61ed96 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -21,12 +21,60 @@ static int snprintf_hex(char *buf, size_t size, unsigned char *data, size_t len)
 	return ret;
 }
 
+static int machine__process_bpf_event_load(struct machine *machine __maybe_unused,
+					   union perf_event *event,
+					   struct perf_sample *sample __maybe_unused)
+{
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_info_node *info_node;
+	struct perf_env *env = machine->env;
+	int id = event->bpf_event.id;
+	unsigned int i;
+
+	/* perf-record, no need to handle bpf-event */
+	if (env == NULL)
+		return 0;
+
+	info_node = perf_env__find_bpf_prog_info(env, id);
+	if (!info_node)
+		return 0;
+	info_linear = info_node->info_linear;
+
+	for (i = 0; i < info_linear->info.nr_jited_ksyms; i++) {
+		u64 *addrs = (u64 *)(info_linear->info.jited_ksyms);
+		u64 addr = addrs[i];
+		struct map *map;
+
+		map = map_groups__find(&machine->kmaps, addr);
+
+		if (map) {
+			map->dso->binary_type = DSO_BINARY_TYPE__BPF_PROG_INFO;
+			map->dso->bpf_prog.id = id;
+			map->dso->bpf_prog.sub_id = i;
+			map->dso->bpf_prog.env = env;
+		}
+	}
+	return 0;
+}
+
 int machine__process_bpf_event(struct machine *machine __maybe_unused,
 			       union perf_event *event,
 			       struct perf_sample *sample __maybe_unused)
 {
 	if (dump_trace)
 		perf_event__fprintf_bpf_event(event, stdout);
+
+	switch (event->bpf_event.type) {
+	case PERF_BPF_EVENT_PROG_LOAD:
+		return machine__process_bpf_event_load(machine, event, sample);
+
+	case PERF_BPF_EVENT_PROG_UNLOAD:
+		break;
+	default:
+		pr_debug("unexpected bpf_event type of %d\n",
+			 event->bpf_event.type);
+		break;
+	}
 	return 0;
 }
 
diff --git a/tools/perf/util/bpf-event.h b/tools/perf/util/bpf-event.h
index 60ce24e4e5c6..c4f0f1395ea5 100644
--- a/tools/perf/util/bpf-event.h
+++ b/tools/perf/util/bpf-event.h
@@ -2,6 +2,10 @@
 #ifndef __PERF_BPF_EVENT_H
 #define __PERF_BPF_EVENT_H
 
+#include <bpf/bpf.h>
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+#include <linux/btf.h>
 #include <linux/compiler.h>
 #include <bpf/bpf.h>
 #include <bpf/btf.h>
diff --git a/tools/perf/util/dso.c b/tools/perf/util/dso.c
index 62c8cf622607..1798192bf0f9 100644
--- a/tools/perf/util/dso.c
+++ b/tools/perf/util/dso.c
@@ -181,6 +181,7 @@ int dso__read_binary_type_filename(const struct dso *dso,
 	case DSO_BINARY_TYPE__KALLSYMS:
 	case DSO_BINARY_TYPE__GUEST_KALLSYMS:
 	case DSO_BINARY_TYPE__JAVA_JIT:
+	case DSO_BINARY_TYPE__BPF_PROG_INFO:
 	case DSO_BINARY_TYPE__NOT_FOUND:
 		ret = -1;
 		break;
diff --git a/tools/perf/util/dso.h b/tools/perf/util/dso.h
index 8c8a7abe809d..f20d319463f1 100644
--- a/tools/perf/util/dso.h
+++ b/tools/perf/util/dso.h
@@ -14,6 +14,8 @@
 #include "namespaces.h"
 #include "build-id.h"
 
+struct perf_env;
+
 enum dso_binary_type {
 	DSO_BINARY_TYPE__KALLSYMS = 0,
 	DSO_BINARY_TYPE__GUEST_KALLSYMS,
@@ -34,6 +36,7 @@ enum dso_binary_type {
 	DSO_BINARY_TYPE__KCORE,
 	DSO_BINARY_TYPE__GUEST_KCORE,
 	DSO_BINARY_TYPE__OPENEMBEDDED_DEBUGINFO,
+	DSO_BINARY_TYPE__BPF_PROG_INFO,
 	DSO_BINARY_TYPE__NOT_FOUND,
 };
 
@@ -177,17 +180,25 @@ struct dso {
 	struct auxtrace_cache *auxtrace_cache;
 	int		 comp;
 
-	/* dso data file */
-	struct {
-		struct rb_root	 cache;
-		int		 fd;
-		int		 status;
-		u32		 status_seen;
-		size_t		 file_size;
-		struct list_head open_entry;
-		u64		 debug_frame_offset;
-		u64		 eh_frame_hdr_offset;
-	} data;
+	union {
+		/* dso data file */
+		struct {
+			struct rb_root	 cache;
+			int		 fd;
+			int		 status;
+			u32		 status_seen;
+			size_t		 file_size;
+			struct list_head open_entry;
+			u64		 debug_frame_offset;
+			u64		 eh_frame_hdr_offset;
+		} data;
+		/* bpf prog information */
+		struct {
+			u32		id;
+			u32		sub_id;
+			struct perf_env	*env;
+		} bpf_prog;
+	};
 
 	union { /* Tool specific area */
 		void	 *priv;
diff --git a/tools/perf/util/symbol.c b/tools/perf/util/symbol.c
index 48efad6d0f90..33ae59e89da2 100644
--- a/tools/perf/util/symbol.c
+++ b/tools/perf/util/symbol.c
@@ -1441,6 +1441,7 @@ static bool dso__is_compatible_symtab_type(struct dso *dso, bool kmod,
 	case DSO_BINARY_TYPE__BUILD_ID_CACHE_DEBUGINFO:
 		return true;
 
+	case DSO_BINARY_TYPE__BPF_PROG_INFO:
 	case DSO_BINARY_TYPE__NOT_FOUND:
 	default:
 		return false;
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 06/11] perf, bpf: save bpf_prog_info information as headers to perf.data
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

This patch enables perf-record to save bpf_prog_info information as
headers to perf.data. A new header type HEADER_BPF_PROG_INFO is
introduced for this data.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/util/header.c | 134 ++++++++++++++++++++++++++++++++++++++-
 tools/perf/util/header.h |   1 +
 2 files changed, 134 insertions(+), 1 deletion(-)

diff --git a/tools/perf/util/header.c b/tools/perf/util/header.c
index dec6d218c31c..2ae76a9d06f6 100644
--- a/tools/perf/util/header.c
+++ b/tools/perf/util/header.c
@@ -1080,6 +1080,51 @@ static int write_clockid(struct feat_fd *ff,
 			sizeof(ff->ph->env.clockid_res_ns));
 }
 
+static int write_bpf_prog_info(struct feat_fd *ff,
+			       struct perf_evlist *evlist __maybe_unused)
+{
+	struct perf_env *env = &ff->ph->env;
+	struct rb_root *root;
+	struct rb_node *next;
+	u32 count = 0;
+	int ret;
+
+	down_read(&env->bpf_info_lock);
+
+	root = &env->bpf_prog_infos;
+	next = rb_first(root);
+	while (next) {
+		++count;
+		next = rb_next(next);
+	}
+
+	ret = do_write(ff, &count, sizeof(count));
+
+	if (ret < 0)
+		goto out;
+
+	next = rb_first(root);
+	while (next) {
+		struct bpf_prog_info_node *node;
+		size_t len;
+
+		node = rb_entry(next, struct bpf_prog_info_node, rb_node);
+		next = rb_next(&node->rb_node);
+		len = sizeof(struct bpf_prog_info_linear) +
+			node->info_linear->data_len;
+
+		/* before writing to file, translate address to offset */
+		bpf_program__bpil_addr_to_offs(node->info_linear);
+		ret = do_write(ff, node->info_linear, len);
+		bpf_program__bpil_offs_to_addr(node->info_linear);
+		if (ret < 0)
+			goto out;
+	}
+out:
+	up_read(&env->bpf_info_lock);
+	return ret;
+}
+
 static int cpu_cache_level__sort(const void *a, const void *b)
 {
 	struct cpu_cache_level *cache_a = (struct cpu_cache_level *)a;
@@ -1560,6 +1605,29 @@ static void print_clockid(struct feat_fd *ff, FILE *fp)
 		ff->ph->env.clockid_res_ns * 1000);
 }
 
+static void print_bpf_prog_info(struct feat_fd *ff, FILE *fp)
+{
+	struct perf_env *env = &ff->ph->env;
+	struct rb_root *root;
+	struct rb_node *next;
+
+	down_read(&env->bpf_info_lock);
+
+	root = &env->bpf_prog_infos;
+	next = rb_first(root);
+
+	while (next) {
+		struct bpf_prog_info_node *node;
+
+		node = rb_entry(next, struct bpf_prog_info_node, rb_node);
+		next = rb_next(&node->rb_node);
+		fprintf(fp, "# bpf_prog_info of id %u\n",
+			node->info_linear->info.id);
+	}
+
+	up_read(&env->bpf_info_lock);
+}
+
 static void free_event_desc(struct perf_evsel *events)
 {
 	struct perf_evsel *evsel;
@@ -2592,6 +2660,69 @@ static int process_clockid(struct feat_fd *ff,
 	return 0;
 }
 
+static int process_bpf_prog_info(struct feat_fd *ff,
+				 void *data __maybe_unused)
+{
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_info_node *info_node;
+	struct perf_env *env = &ff->ph->env;
+	u32 count, i;
+	int err = -1;
+
+	if (do_read_u32(ff, &count))
+		return -1;
+
+	down_write(&env->bpf_info_lock);
+
+	for (i = 0; i < count; ++i) {
+		u32 info_len, data_len;
+
+		info_linear = NULL;
+		info_node = NULL;
+		if (do_read_u32(ff, &info_len))
+			goto out;
+		if (do_read_u32(ff, &data_len))
+			goto out;
+
+		if (info_len > sizeof(struct bpf_prog_info)) {
+			pr_warning("detected invalid bpf_prog_info\n");
+			goto out;
+		}
+
+		info_linear = malloc(sizeof(struct bpf_prog_info_linear) +
+				     data_len);
+		if (!info_linear)
+			goto out;
+		info_linear->info_len = sizeof(struct bpf_prog_info);
+		info_linear->data_len = data_len;
+		if (do_read_u64(ff, (u64 *)(&info_linear->arrays)))
+			goto out;
+		if (__do_read(ff, &info_linear->info, info_len))
+			goto out;
+		if (info_len < sizeof(struct bpf_prog_info))
+			memset(((void *)(&info_linear->info)) + info_len, 0,
+			       sizeof(struct bpf_prog_info) - info_len);
+
+		if (__do_read(ff, info_linear->data, data_len))
+			goto out;
+		info_node = malloc(sizeof(struct bpf_prog_info_node));
+		if (!info_node)
+			goto out;
+
+		/* after reading from file, translate offset to address */
+		bpf_program__bpil_offs_to_addr(info_linear);
+		info_node->info_linear = info_linear;
+		perf_env__insert_bpf_prog_info(env, info_node);
+	}
+
+	return 0;
+out:
+	free(info_linear);
+	free(info_node);
+	up_write(&env->bpf_info_lock);
+	return err;
+}
+
 struct feature_ops {
 	int (*write)(struct feat_fd *ff, struct perf_evlist *evlist);
 	void (*print)(struct feat_fd *ff, FILE *fp);
@@ -2651,7 +2782,8 @@ static const struct feature_ops feat_ops[HEADER_LAST_FEATURE] = {
 	FEAT_OPN(CACHE,		cache,		true),
 	FEAT_OPR(SAMPLE_TIME,	sample_time,	false),
 	FEAT_OPR(MEM_TOPOLOGY,	mem_topology,	true),
-	FEAT_OPR(CLOCKID,       clockid,        false)
+	FEAT_OPR(CLOCKID,       clockid,        false),
+	FEAT_OPR(BPF_PROG_INFO, bpf_prog_info,  false)
 };
 
 struct header_print_data {
diff --git a/tools/perf/util/header.h b/tools/perf/util/header.h
index 0d553ddca0a3..0785c91b4c3a 100644
--- a/tools/perf/util/header.h
+++ b/tools/perf/util/header.h
@@ -39,6 +39,7 @@ enum {
 	HEADER_SAMPLE_TIME,
 	HEADER_MEM_TOPOLOGY,
 	HEADER_CLOCKID,
+	HEADER_BPF_PROG_INFO,
 	HEADER_LAST_FEATURE,
 	HEADER_FEAT_BITS	= 256,
 };
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 03/11] bpf: bpftool: use bpf_program__get_prog_info_linear() in prog.c:do_dump()
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

This patches uses bpf_program__get_prog_info_linear() to simplify the
logic in prog.c do_dump().

Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/bpf/bpftool/prog.c | 266 +++++++++------------------------------
 1 file changed, 59 insertions(+), 207 deletions(-)

diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 0640e9bc0ada..206b820df7c2 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -393,41 +393,31 @@ static int do_show(int argc, char **argv)
 
 static int do_dump(int argc, char **argv)
 {
-	unsigned int finfo_rec_size, linfo_rec_size, jited_linfo_rec_size;
-	void *func_info = NULL, *linfo = NULL, *jited_linfo = NULL;
-	unsigned int nr_finfo, nr_linfo = 0, nr_jited_linfo = 0;
+	struct bpf_prog_info_linear *info_linear;
 	struct bpf_prog_linfo *prog_linfo = NULL;
-	unsigned long *func_ksyms = NULL;
-	struct bpf_prog_info info = {};
-	unsigned int *func_lens = NULL;
+	enum {DUMP_JITED, DUMP_XLATED} mode;
 	const char *disasm_opt = NULL;
-	unsigned int nr_func_ksyms;
-	unsigned int nr_func_lens;
+	struct bpf_prog_info *info;
 	struct dump_data dd = {};
-	__u32 len = sizeof(info);
+	void *func_info = NULL;
 	struct btf *btf = NULL;
-	unsigned int buf_size;
 	char *filepath = NULL;
 	bool opcodes = false;
 	bool visual = false;
 	char func_sig[1024];
 	unsigned char *buf;
 	bool linum = false;
-	__u32 *member_len;
-	__u64 *member_ptr;
+	__u32 member_len;
+	__u64 arrays;
 	ssize_t n;
-	int err;
 	int fd;
 
 	if (is_prefix(*argv, "jited")) {
 		if (disasm_init())
 			return -1;
-
-		member_len = &info.jited_prog_len;
-		member_ptr = &info.jited_prog_insns;
+		mode = DUMP_JITED;
 	} else if (is_prefix(*argv, "xlated")) {
-		member_len = &info.xlated_prog_len;
-		member_ptr = &info.xlated_prog_insns;
+		mode = DUMP_XLATED;
 	} else {
 		p_err("expected 'xlated' or 'jited', got: %s", *argv);
 		return -1;
@@ -466,175 +456,50 @@ static int do_dump(int argc, char **argv)
 		return -1;
 	}
 
-	err = bpf_obj_get_info_by_fd(fd, &info, &len);
-	if (err) {
-		p_err("can't get prog info: %s", strerror(errno));
-		return -1;
-	}
-
-	if (!*member_len) {
-		p_info("no instructions returned");
-		close(fd);
-		return 0;
-	}
+	if (mode == DUMP_JITED)
+		arrays = 1UL << BPF_PROG_INFO_JITED_INSNS;
+	else
+		arrays = 1UL << BPF_PROG_INFO_XLATED_INSNS;
 
-	buf_size = *member_len;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_KSYMS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_FUNC_LENS;
+	arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_LINE_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_LINE_INFO;
 
-	buf = malloc(buf_size);
-	if (!buf) {
-		p_err("mem alloc failed");
-		close(fd);
+	info_linear = bpf_program__get_prog_info_linear(fd, arrays);
+	close(fd);
+	if (IS_ERR_OR_NULL(info_linear)) {
+		p_err("can't get prog info: %s", strerror(errno));
 		return -1;
 	}
 
-	nr_func_ksyms = info.nr_jited_ksyms;
-	if (nr_func_ksyms) {
-		func_ksyms = malloc(nr_func_ksyms * sizeof(__u64));
-		if (!func_ksyms) {
-			p_err("mem alloc failed");
-			close(fd);
-			goto err_free;
-		}
-	}
-
-	nr_func_lens = info.nr_jited_func_lens;
-	if (nr_func_lens) {
-		func_lens = malloc(nr_func_lens * sizeof(__u32));
-		if (!func_lens) {
-			p_err("mem alloc failed");
-			close(fd);
+	info = &info_linear->info;
+	if (mode == DUMP_JITED) {
+		if (info->jited_prog_len == 0) {
+			p_info("no instructions returned");
 			goto err_free;
 		}
-	}
-
-	nr_finfo = info.nr_func_info;
-	finfo_rec_size = info.func_info_rec_size;
-	if (nr_finfo && finfo_rec_size) {
-		func_info = malloc(nr_finfo * finfo_rec_size);
-		if (!func_info) {
-			p_err("mem alloc failed");
-			close(fd);
+		buf = (unsigned char *)(info->jited_prog_insns);
+		member_len = info->jited_prog_len;
+	} else {	/* DUMP_XLATED */
+		if (info->xlated_prog_len == 0) {
+			p_err("error retrieving insn dump: kernel.kptr_restrict set?");
 			goto err_free;
 		}
+		buf = (unsigned char *)info->xlated_prog_insns;
+		member_len = info->xlated_prog_len;
 	}
 
-	linfo_rec_size = info.line_info_rec_size;
-	if (info.nr_line_info && linfo_rec_size && info.btf_id) {
-		nr_linfo = info.nr_line_info;
-		linfo = malloc(nr_linfo * linfo_rec_size);
-		if (!linfo) {
-			p_err("mem alloc failed");
-			close(fd);
-			goto err_free;
-		}
-	}
-
-	jited_linfo_rec_size = info.jited_line_info_rec_size;
-	if (info.nr_jited_line_info &&
-	    jited_linfo_rec_size &&
-	    info.nr_jited_ksyms &&
-	    info.nr_jited_func_lens &&
-	    info.btf_id) {
-		nr_jited_linfo = info.nr_jited_line_info;
-		jited_linfo = malloc(nr_jited_linfo * jited_linfo_rec_size);
-		if (!jited_linfo) {
-			p_err("mem alloc failed");
-			close(fd);
-			goto err_free;
-		}
-	}
-
-	memset(&info, 0, sizeof(info));
-
-	*member_ptr = ptr_to_u64(buf);
-	*member_len = buf_size;
-	info.jited_ksyms = ptr_to_u64(func_ksyms);
-	info.nr_jited_ksyms = nr_func_ksyms;
-	info.jited_func_lens = ptr_to_u64(func_lens);
-	info.nr_jited_func_lens = nr_func_lens;
-	info.nr_func_info = nr_finfo;
-	info.func_info_rec_size = finfo_rec_size;
-	info.func_info = ptr_to_u64(func_info);
-	info.nr_line_info = nr_linfo;
-	info.line_info_rec_size = linfo_rec_size;
-	info.line_info = ptr_to_u64(linfo);
-	info.nr_jited_line_info = nr_jited_linfo;
-	info.jited_line_info_rec_size = jited_linfo_rec_size;
-	info.jited_line_info = ptr_to_u64(jited_linfo);
-
-	err = bpf_obj_get_info_by_fd(fd, &info, &len);
-	close(fd);
-	if (err) {
-		p_err("can't get prog info: %s", strerror(errno));
-		goto err_free;
-	}
-
-	if (*member_len > buf_size) {
-		p_err("too many instructions returned");
-		goto err_free;
-	}
-
-	if (info.nr_jited_ksyms > nr_func_ksyms) {
-		p_err("too many addresses returned");
-		goto err_free;
-	}
-
-	if (info.nr_jited_func_lens > nr_func_lens) {
-		p_err("too many values returned");
-		goto err_free;
-	}
-
-	if (info.nr_func_info != nr_finfo) {
-		p_err("incorrect nr_func_info %d vs. expected %d",
-		      info.nr_func_info, nr_finfo);
-		goto err_free;
-	}
-
-	if (info.func_info_rec_size != finfo_rec_size) {
-		p_err("incorrect func_info_rec_size %d vs. expected %d",
-		      info.func_info_rec_size, finfo_rec_size);
-		goto err_free;
-	}
-
-	if (linfo && info.nr_line_info != nr_linfo) {
-		p_err("incorrect nr_line_info %u vs. expected %u",
-		      info.nr_line_info, nr_linfo);
-		goto err_free;
-	}
-
-	if (info.line_info_rec_size != linfo_rec_size) {
-		p_err("incorrect line_info_rec_size %u vs. expected %u",
-		      info.line_info_rec_size, linfo_rec_size);
-		goto err_free;
-	}
-
-	if (jited_linfo && info.nr_jited_line_info != nr_jited_linfo) {
-		p_err("incorrect nr_jited_line_info %u vs. expected %u",
-		      info.nr_jited_line_info, nr_jited_linfo);
-		goto err_free;
-	}
-
-	if (info.jited_line_info_rec_size != jited_linfo_rec_size) {
-		p_err("incorrect jited_line_info_rec_size %u vs. expected %u",
-		      info.jited_line_info_rec_size, jited_linfo_rec_size);
-		goto err_free;
-	}
-
-	if ((member_len == &info.jited_prog_len &&
-	     info.jited_prog_insns == 0) ||
-	    (member_len == &info.xlated_prog_len &&
-	     info.xlated_prog_insns == 0)) {
-		p_err("error retrieving insn dump: kernel.kptr_restrict set?");
-		goto err_free;
-	}
-
-	if (info.btf_id && btf__get_from_id(info.btf_id, &btf)) {
+	if (info->btf_id && btf__get_from_id(info->btf_id, &btf)) {
 		p_err("failed to get btf");
 		goto err_free;
 	}
 
-	if (nr_linfo) {
-		prog_linfo = bpf_prog_linfo__new(&info);
+	func_info = (void *)info->func_info;
+
+	if (info->nr_line_info) {
+		prog_linfo = bpf_prog_linfo__new(info);
 		if (!prog_linfo)
 			p_info("error in processing bpf_line_info.  continue without it.");
 	}
@@ -647,9 +512,9 @@ static int do_dump(int argc, char **argv)
 			goto err_free;
 		}
 
-		n = write(fd, buf, *member_len);
+		n = write(fd, buf, member_len);
 		close(fd);
-		if (n != *member_len) {
+		if (n != member_len) {
 			p_err("error writing output file: %s",
 			      n < 0 ? strerror(errno) : "short write");
 			goto err_free;
@@ -657,19 +522,19 @@ static int do_dump(int argc, char **argv)
 
 		if (json_output)
 			jsonw_null(json_wtr);
-	} else if (member_len == &info.jited_prog_len) {
+	} else if (mode == DUMP_JITED) {
 		const char *name = NULL;
 
-		if (info.ifindex) {
-			name = ifindex_to_bfd_params(info.ifindex,
-						     info.netns_dev,
-						     info.netns_ino,
+		if (info->ifindex) {
+			name = ifindex_to_bfd_params(info->ifindex,
+						     info->netns_dev,
+						     info->netns_ino,
 						     &disasm_opt);
 			if (!name)
 				goto err_free;
 		}
 
-		if (info.nr_jited_func_lens && info.jited_func_lens) {
+		if (info->nr_jited_func_lens && info->jited_func_lens) {
 			struct kernel_sym *sym = NULL;
 			struct bpf_func_info *record;
 			char sym_name[SYM_MAX_NAME];
@@ -677,17 +542,16 @@ static int do_dump(int argc, char **argv)
 			__u64 *ksyms = NULL;
 			__u32 *lens;
 			__u32 i;
-
-			if (info.nr_jited_ksyms) {
+			if (info->nr_jited_ksyms) {
 				kernel_syms_load(&dd);
-				ksyms = (__u64 *) info.jited_ksyms;
+				ksyms = (__u64 *) info->jited_ksyms;
 			}
 
 			if (json_output)
 				jsonw_start_array(json_wtr);
 
-			lens = (__u32 *) info.jited_func_lens;
-			for (i = 0; i < info.nr_jited_func_lens; i++) {
+			lens = (__u32 *) info->jited_func_lens;
+			for (i = 0; i < info->nr_jited_func_lens; i++) {
 				if (ksyms) {
 					sym = kernel_syms_search(&dd, ksyms[i]);
 					if (sym)
@@ -699,7 +563,7 @@ static int do_dump(int argc, char **argv)
 				}
 
 				if (func_info) {
-					record = func_info + i * finfo_rec_size;
+					record = func_info + i * info->func_info_rec_size;
 					btf_dumper_type_only(btf, record->type_id,
 							     func_sig,
 							     sizeof(func_sig));
@@ -736,49 +600,37 @@ static int do_dump(int argc, char **argv)
 			if (json_output)
 				jsonw_end_array(json_wtr);
 		} else {
-			disasm_print_insn(buf, *member_len, opcodes, name,
+			disasm_print_insn(buf, member_len, opcodes, name,
 					  disasm_opt, btf, NULL, 0, 0, false);
 		}
 	} else if (visual) {
 		if (json_output)
 			jsonw_null(json_wtr);
 		else
-			dump_xlated_cfg(buf, *member_len);
+			dump_xlated_cfg(buf, member_len);
 	} else {
 		kernel_syms_load(&dd);
-		dd.nr_jited_ksyms = info.nr_jited_ksyms;
-		dd.jited_ksyms = (__u64 *) info.jited_ksyms;
+		dd.nr_jited_ksyms = info->nr_jited_ksyms;
+		dd.jited_ksyms = (__u64 *) info->jited_ksyms;
 		dd.btf = btf;
 		dd.func_info = func_info;
-		dd.finfo_rec_size = finfo_rec_size;
+		dd.finfo_rec_size = info->func_info_rec_size;
 		dd.prog_linfo = prog_linfo;
 
 		if (json_output)
-			dump_xlated_json(&dd, buf, *member_len, opcodes,
+			dump_xlated_json(&dd, buf, member_len, opcodes,
 					 linum);
 		else
-			dump_xlated_plain(&dd, buf, *member_len, opcodes,
+			dump_xlated_plain(&dd, buf, member_len, opcodes,
 					  linum);
 		kernel_syms_destroy(&dd);
 	}
 
-	free(buf);
-	free(func_ksyms);
-	free(func_lens);
-	free(func_info);
-	free(linfo);
-	free(jited_linfo);
-	bpf_prog_linfo__free(prog_linfo);
+	free(info_linear);
 	return 0;
 
 err_free:
-	free(buf);
-	free(func_ksyms);
-	free(func_lens);
-	free(func_info);
-	free(linfo);
-	free(jited_linfo);
-	bpf_prog_linfo__free(prog_linfo);
+	free(info_linear);
 	return -1;
 }
 
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 02/11] bpf: libbpf: introduce bpf_program__get_prog_info_linear()
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190214235624.2579307-1-songliubraving@fb.com>

Currently, bpf_prog_info includes 9 arrays. The user has the option to
fetch any combination of these arrays. However, this requires a lot of
handling of these arrays. This work becomes more tricky when we need to
store bpf_prog_info to a file, because these arrays are allocated
independently.

This patch introduces struct bpf_prog_info_linear, which stores arrays
of bpf_prog_info in continues memory. Helper functions are introduced
to unify the work to get different information of bpf_prog_info.
Specifically, bpf_program__get_prog_info_linear() allows the user to
select which arrays to fetch, and handles details for the user.

Plesae see the comments before enum bpf_prog_info_array for more details
and examples.

Cc: Daniel Borkmann <daniel@iogearbox.net>
Cc: Alexei Starovoitov <ast@kernel.org>
Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/lib/bpf/libbpf.c   | 251 +++++++++++++++++++++++++++++++++++++++
 tools/lib/bpf/libbpf.h   |  63 ++++++++++
 tools/lib/bpf/libbpf.map |   3 +
 3 files changed, 317 insertions(+)

diff --git a/tools/lib/bpf/libbpf.c b/tools/lib/bpf/libbpf.c
index e3c39edfb9d3..1f808c555747 100644
--- a/tools/lib/bpf/libbpf.c
+++ b/tools/lib/bpf/libbpf.c
@@ -113,6 +113,11 @@ void libbpf_print(enum libbpf_print_level level, const char *format, ...)
 # define LIBBPF_ELF_C_READ_MMAP ELF_C_READ
 #endif
 
+static inline __u64 ptr_to_u64(const void *ptr)
+{
+	return (__u64) (unsigned long) ptr;
+}
+
 struct bpf_capabilities {
 	/* v4.14: kernel support for program & map names. */
 	__u32 name:1;
@@ -2979,3 +2984,249 @@ bpf_perf_event_read_simple(void *mmap_mem, size_t mmap_size, size_t page_size,
 	ring_buffer_write_tail(header, data_tail);
 	return ret;
 }
+
+struct bpf_prog_info_array_desc {
+	int	array_offset;	/* e.g. offset of jited_prog_insns */
+	int	count_offset;	/* e.g. offset of jited_prog_len */
+	int	size_offset;	/* > 0: offset of rec size,
+				 * < 0: fix size of -size_offset
+				 */
+};
+
+static struct bpf_prog_info_array_desc bpf_prog_info_array_desc[] = {
+	[BPF_PROG_INFO_JITED_INSNS] = {
+		offsetof(struct bpf_prog_info, jited_prog_insns),
+		offsetof(struct bpf_prog_info, jited_prog_len),
+		-1,
+	},
+	[BPF_PROG_INFO_XLATED_INSNS] = {
+		offsetof(struct bpf_prog_info, xlated_prog_insns),
+		offsetof(struct bpf_prog_info, xlated_prog_len),
+		-1,
+	},
+	[BPF_PROG_INFO_MAP_IDS] = {
+		offsetof(struct bpf_prog_info, map_ids),
+		offsetof(struct bpf_prog_info, nr_map_ids),
+		-(int)sizeof(__u32),
+	},
+	[BPF_PROG_INFO_JITED_KSYMS] = {
+		offsetof(struct bpf_prog_info, jited_ksyms),
+		offsetof(struct bpf_prog_info, nr_jited_ksyms),
+		-(int)sizeof(__u64),
+	},
+	[BPF_PROG_INFO_JITED_FUNC_LENS] = {
+		offsetof(struct bpf_prog_info, jited_func_lens),
+		offsetof(struct bpf_prog_info, nr_jited_func_lens),
+		-(int)sizeof(__u32),
+	},
+	[BPF_PROG_INFO_FUNC_INFO] = {
+		offsetof(struct bpf_prog_info, func_info),
+		offsetof(struct bpf_prog_info, nr_func_info),
+		offsetof(struct bpf_prog_info, func_info_rec_size),
+	},
+	[BPF_PROG_INFO_LINE_INFO] = {
+		offsetof(struct bpf_prog_info, line_info),
+		offsetof(struct bpf_prog_info, nr_line_info),
+		offsetof(struct bpf_prog_info, line_info_rec_size),
+	},
+	[BPF_PROG_INFO_JITED_LINE_INFO] = {
+		offsetof(struct bpf_prog_info, jited_line_info),
+		offsetof(struct bpf_prog_info, nr_jited_line_info),
+		offsetof(struct bpf_prog_info, jited_line_info_rec_size),
+	},
+	[BPF_PROG_INFO_PROG_TAGS] = {
+		offsetof(struct bpf_prog_info, prog_tags),
+		offsetof(struct bpf_prog_info, nr_prog_tags),
+		-(int)sizeof(__u8) * BPF_TAG_SIZE,
+	},
+
+};
+
+static __u32 bpf_prog_info_read_offset_u32(struct bpf_prog_info *info, int offset)
+{
+	__u32 *array = (__u32 *)info;
+
+	if (offset >= 0)
+		return array[offset / sizeof(__u32)];
+	return -(int)offset;
+}
+
+static __u64 bpf_prog_info_read_offset_u64(struct bpf_prog_info *info, int offset)
+{
+	__u64 *array = (__u64 *)info;
+
+	if (offset >= 0)
+		return array[offset / sizeof(__u64)];
+	return -(int)offset;
+}
+
+static void bpf_prog_info_set_offset_u32(struct bpf_prog_info *info, int offset,
+					 __u32 val)
+{
+	__u32 *array = (__u32 *)info;
+
+	if (offset >= 0)
+		array[offset / sizeof(__u32)] = val;
+}
+
+static void bpf_prog_info_set_offset_u64(struct bpf_prog_info *info, int offset,
+					 __u64 val)
+{
+	__u64 *array = (__u64 *)info;
+
+	if (offset >= 0)
+		array[offset / sizeof(__u64)] = val;
+}
+
+struct bpf_prog_info_linear *
+bpf_program__get_prog_info_linear(int fd, __u64 arrays)
+{
+	struct bpf_prog_info_linear *info_linear;
+	struct bpf_prog_info info = {};
+	__u32 info_len = sizeof(info);
+	__u32 data_len = 0;
+	int i, err;
+	void *ptr;
+
+	if (arrays >> BPF_PROG_INFO_LAST_ARRAY)
+		return ERR_PTR(-EINVAL);
+
+	/* step 1: get array dimensions */
+	err = bpf_obj_get_info_by_fd(fd, &info, &info_len);
+	if (err) {
+		pr_debug("can't get prog info: %s", strerror(errno));
+		return ERR_PTR(-EFAULT);
+	}
+
+	/* step 2: calculate total size of all arrays */
+	for (i = BPF_PROG_INFO_FIRST_ARRAY; i < BPF_PROG_INFO_LAST_ARRAY; ++i) {
+		bool include_array = (arrays & (1UL << i)) > 0;
+		struct bpf_prog_info_array_desc *desc;
+		__u32 count, size;
+
+		desc = bpf_prog_info_array_desc + i;
+
+		/* kernel is too old to support this field */
+		if (info_len < desc->array_offset + sizeof(__u32) ||
+		    info_len < desc->count_offset + sizeof(__u32) ||
+		    (desc->size_offset > 0 && info_len < desc->size_offset))
+			include_array = false;
+
+		if (!include_array) {
+			arrays &= ~(1UL << i);	/* clear the bit */
+			continue;
+		}
+
+		count = bpf_prog_info_read_offset_u32(&info, desc->count_offset);
+		size = bpf_prog_info_read_offset_u32(&info, desc->size_offset);
+
+		data_len += count * size;
+	}
+
+	/* step 3: allocate continuous memory */
+	data_len = roundup(data_len, sizeof(__u64));
+	info_linear = malloc(sizeof(struct bpf_prog_info_linear) + data_len);
+	if (!info_linear)
+		return ERR_PTR(-ENOMEM);
+
+	/* step 4: fill data to info_linear->info */
+	info_linear->arrays = arrays;
+	memset(&info_linear->info, 0, sizeof(info));
+	ptr = info_linear->data;
+
+	for (i = BPF_PROG_INFO_FIRST_ARRAY; i < BPF_PROG_INFO_LAST_ARRAY; ++i) {
+		struct bpf_prog_info_array_desc *desc;
+		__u32 count, size;
+
+		if ((arrays & (1UL << i)) == 0)
+			continue;
+
+		desc = bpf_prog_info_array_desc + i;
+		count = bpf_prog_info_read_offset_u32(&info, desc->count_offset);
+		size = bpf_prog_info_read_offset_u32(&info, desc->size_offset);
+		bpf_prog_info_set_offset_u32(&info_linear->info,
+					     desc->count_offset, count);
+		bpf_prog_info_set_offset_u32(&info_linear->info,
+					     desc->size_offset, size);
+		bpf_prog_info_set_offset_u64(&info_linear->info,
+					     desc->array_offset,
+					     ptr_to_u64(ptr));
+		ptr += count * size;
+	}
+
+	/* step 5: call syscall again to get required arrays */
+	err = bpf_obj_get_info_by_fd(fd, &info_linear->info, &info_len);
+	if (err) {
+		pr_debug("can't get prog info: %s", strerror(errno));
+		free(info_linear);
+		return ERR_PTR(-EFAULT);
+	}
+
+	/* step 6: verify the data */
+	for (i = BPF_PROG_INFO_FIRST_ARRAY; i < BPF_PROG_INFO_LAST_ARRAY; ++i) {
+		struct bpf_prog_info_array_desc *desc;
+		__u32 v1, v2;
+
+		if ((arrays & (1UL << i)) == 0)
+			continue;
+
+		desc = bpf_prog_info_array_desc + i;
+		v1 = bpf_prog_info_read_offset_u32(&info, desc->count_offset);
+		v2 = bpf_prog_info_read_offset_u32(&info_linear->info,
+						   desc->count_offset);
+		if (v1 != v2)
+			pr_warning("%s: mismatch in element count\n", __func__);
+
+		v1 = bpf_prog_info_read_offset_u32(&info, desc->size_offset);
+		v2 = bpf_prog_info_read_offset_u32(&info_linear->info,
+						   desc->size_offset);
+		if (v1 != v2)
+			pr_warning("%s: mismatch in rec size\n", __func__);
+	}
+
+	/* step 7: update info_len and data_len */
+	info_linear->info_len = sizeof(struct bpf_prog_info);
+	info_linear->data_len = data_len;
+
+	return info_linear;
+}
+
+void bpf_program__bpil_addr_to_offs(struct bpf_prog_info_linear *info_linear)
+{
+	int i;
+
+	for (i = BPF_PROG_INFO_FIRST_ARRAY; i < BPF_PROG_INFO_LAST_ARRAY; ++i) {
+		struct bpf_prog_info_array_desc *desc;
+		__u64 addr, offs;
+
+		if ((info_linear->arrays & (1UL << i)) == 0)
+			continue;
+
+		desc = bpf_prog_info_array_desc + i;
+		addr = bpf_prog_info_read_offset_u64(&info_linear->info,
+						     desc->array_offset);
+		offs = addr - ptr_to_u64(info_linear->data);
+		bpf_prog_info_set_offset_u64(&info_linear->info,
+					     desc->array_offset, offs);
+	}
+}
+
+void bpf_program__bpil_offs_to_addr(struct bpf_prog_info_linear *info_linear)
+{
+	int i;
+
+	for (i = BPF_PROG_INFO_FIRST_ARRAY; i < BPF_PROG_INFO_LAST_ARRAY; ++i) {
+		struct bpf_prog_info_array_desc *desc;
+		__u64 addr, offs;
+
+		if ((info_linear->arrays & (1UL << i)) == 0)
+			continue;
+
+		desc = bpf_prog_info_array_desc + i;
+		offs = bpf_prog_info_read_offset_u64(&info_linear->info,
+						     desc->array_offset);
+		addr = offs + ptr_to_u64(info_linear->data);
+		bpf_prog_info_set_offset_u64(&info_linear->info,
+					     desc->array_offset, addr);
+	}
+}
diff --git a/tools/lib/bpf/libbpf.h b/tools/lib/bpf/libbpf.h
index 69a7c25eaccc..4d357c26fbd2 100644
--- a/tools/lib/bpf/libbpf.h
+++ b/tools/lib/bpf/libbpf.h
@@ -372,6 +372,69 @@ LIBBPF_API bool bpf_probe_map_type(enum bpf_map_type map_type, __u32 ifindex);
 LIBBPF_API bool bpf_probe_helper(enum bpf_func_id id,
 				 enum bpf_prog_type prog_type, __u32 ifindex);
 
+/*
+ * Get bpf_prog_info in continuous memory
+ *
+ * struct bpf_prog_info has multiple arrays. The user has option to choose
+ * arrays to fetch from kernel. The following APIs provide uniform way to
+ * fetch these data. All arrays in bpf_prog_info are stored in singile
+ * continuous memory region. This makes it easy to store the info in a
+ * file.
+ *
+ * Before writing bpf_prog_info_linear to files, it is necessary to
+ * translate pointers bpf_prog_info to offsets. Helper functions
+ * bpf_program__bpil_addr_to_offs() and bpf_program__bpil_offs_to_addr()
+ * are introduced to switch between pointers and offsets.
+ *
+ * Examples:
+ *   # To fetch map_ids and prog_tags:
+ *   __u64 arrays = (1UL << BPF_PROG_INFO_MAP_IDS) |
+ *           (1UL << BPF_PROG_INFO_PROG_TAGS);
+ *   struct bpf_prog_info_linear *info_linear =
+ *           bpf_program__get_prog_info_linear(fd, arrays);
+ *
+ *   # To save data in file
+ *   bpf_program__bpil_addr_to_offs(info_linear);
+ *   write(f, info_linear, sizeof(*info_linear) + info_linear->data_len);
+ *
+ *   # To read data from file
+ *   read(f, info_linear, <proper_size>);
+ *   bpf_program__bpil_offs_to_addr(info_linear);
+ */
+enum bpf_prog_info_array {
+	BPF_PROG_INFO_FIRST_ARRAY = 0,
+	BPF_PROG_INFO_JITED_INSNS = 0,
+	BPF_PROG_INFO_XLATED_INSNS,
+	BPF_PROG_INFO_MAP_IDS,
+	BPF_PROG_INFO_JITED_KSYMS,
+	BPF_PROG_INFO_JITED_FUNC_LENS,
+	BPF_PROG_INFO_FUNC_INFO,
+	BPF_PROG_INFO_LINE_INFO,
+	BPF_PROG_INFO_JITED_LINE_INFO,
+	BPF_PROG_INFO_PROG_TAGS,
+	BPF_PROG_INFO_LAST_ARRAY,
+};
+
+struct bpf_prog_info_linear {
+	/* size of struct bpf_prog_info, when the tool is compiled */
+	__u32			info_len;
+	/* total bytes allocated for data, round up to 8 bytes */
+	__u32			data_len;
+	/* which arrays are included in data */
+	__u64			arrays;
+	struct bpf_prog_info	info;
+	__u8			data[];
+};
+
+LIBBPF_API struct bpf_prog_info_linear *
+bpf_program__get_prog_info_linear(int fd, __u64 arrays);
+
+LIBBPF_API void
+bpf_program__bpil_addr_to_offs(struct bpf_prog_info_linear *info_linear);
+
+LIBBPF_API void
+bpf_program__bpil_offs_to_addr(struct bpf_prog_info_linear *info_linear);
+
 #ifdef __cplusplus
 } /* extern "C" */
 #endif
diff --git a/tools/lib/bpf/libbpf.map b/tools/lib/bpf/libbpf.map
index 5fc8222209f8..5f2fceec6ea7 100644
--- a/tools/lib/bpf/libbpf.map
+++ b/tools/lib/bpf/libbpf.map
@@ -145,4 +145,7 @@ LIBBPF_0.0.2 {
 		btf_ext__new;
 		btf_ext__reloc_func_info;
 		btf_ext__reloc_line_info;
+		bpf_program__get_prog_info_linear;
+		bpf_program__bpil_addr_to_offs;
+		bpf_program__bpil_offs_to_addr;
 } LIBBPF_0.0.1;
-- 
2.17.1


^ permalink raw reply related

* [PATCH v2 perf,bpf 05/11] perf, bpf: save bpf_prog_info in a rbtree in perf_env
From: Song Liu @ 2019-02-15  0:00 UTC (permalink / raw)
  To: netdev, linux-kernel
  Cc: ast, daniel, kernel-team, peterz, acme, jolsa, namhyung, Song Liu
In-Reply-To: <20190215000010.2590505-1-songliubraving@fb.com>

bpf_prog_info contains information necessary to annotate bpf programs.
This patch saves bpf_prog_info for bpf programs loaded in the system.

Signed-off-by: Song Liu <songliubraving@fb.com>
---
 tools/perf/builtin-record.c |  2 +-
 tools/perf/builtin-top.c    |  2 +-
 tools/perf/util/bpf-event.c | 39 +++++++++++++----
 tools/perf/util/bpf-event.h | 15 +++++--
 tools/perf/util/env.c       | 83 +++++++++++++++++++++++++++++++++++++
 tools/perf/util/env.h       | 14 +++++++
 6 files changed, 142 insertions(+), 13 deletions(-)

diff --git a/tools/perf/builtin-record.c b/tools/perf/builtin-record.c
index 88ea11d57c6f..2355e0a9eda0 100644
--- a/tools/perf/builtin-record.c
+++ b/tools/perf/builtin-record.c
@@ -1083,7 +1083,7 @@ static int record__synthesize(struct record *rec, bool tail)
 		return err;
 	}
 
-	err = perf_event__synthesize_bpf_events(tool, process_synthesized_event,
+	err = perf_event__synthesize_bpf_events(session, process_synthesized_event,
 						machine, opts);
 	if (err < 0)
 		pr_warning("Couldn't synthesize bpf events.\n");
diff --git a/tools/perf/builtin-top.c b/tools/perf/builtin-top.c
index 5a486d4de56e..27d8d42e0a4d 100644
--- a/tools/perf/builtin-top.c
+++ b/tools/perf/builtin-top.c
@@ -1216,7 +1216,7 @@ static int __cmd_top(struct perf_top *top)
 
 	init_process_thread(top);
 
-	ret = perf_event__synthesize_bpf_events(&top->tool, perf_event__process,
+	ret = perf_event__synthesize_bpf_events(top->session, perf_event__process,
 						&top->session->machines.host,
 						&top->record_opts);
 	if (ret < 0)
diff --git a/tools/perf/util/bpf-event.c b/tools/perf/util/bpf-event.c
index e6dfb95029e5..ead599bc4f4e 100644
--- a/tools/perf/util/bpf-event.c
+++ b/tools/perf/util/bpf-event.c
@@ -1,15 +1,13 @@
 // SPDX-License-Identifier: GPL-2.0
 #include <errno.h>
 #include <stdlib.h>
-#include <bpf/bpf.h>
-#include <bpf/btf.h>
-#include <bpf/libbpf.h>
-#include <linux/btf.h>
 #include <linux/err.h>
 #include "bpf-event.h"
 #include "debug.h"
 #include "symbol.h"
 #include "machine.h"
+#include "env.h"
+#include "session.h"
 
 #define ptr_to_u64(ptr)    ((__u64)(unsigned long)(ptr))
 
@@ -42,7 +40,7 @@ int machine__process_bpf_event(struct machine *machine __maybe_unused,
  *   -1 for failures;
  *   -2 for lack of kernel support.
  */
-static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
+static int perf_event__synthesize_one_bpf_prog(struct perf_session *session,
 					       perf_event__handler_t process,
 					       struct machine *machine,
 					       int fd,
@@ -52,17 +50,29 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 	struct ksymbol_event *ksymbol_event = &event->ksymbol_event;
 	struct bpf_event *bpf_event = &event->bpf_event;
 	struct bpf_prog_info_linear *info_linear;
+	struct perf_tool *tool = session->tool;
+	struct bpf_prog_info_node *info_node;
 	struct bpf_prog_info *info;
 	struct btf *btf = NULL;
 	bool has_btf = false;
+	struct perf_env *env;
 	u32 sub_prog_cnt, i;
 	int err = 0;
 	u64 arrays;
 
+	/*
+	 * for perf-record and perf-report use header.env;
+	 * otherwise, use global perf_env.
+	 */
+	env = session->data ? &session->header.env : &perf_env;
+
 	arrays = 1UL << BPF_PROG_INFO_JITED_KSYMS;
 	arrays |= 1UL << BPF_PROG_INFO_JITED_FUNC_LENS;
 	arrays |= 1UL << BPF_PROG_INFO_FUNC_INFO;
 	arrays |= 1UL << BPF_PROG_INFO_PROG_TAGS;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_INSNS;
+	arrays |= 1UL << BPF_PROG_INFO_LINE_INFO;
+	arrays |= 1UL << BPF_PROG_INFO_JITED_LINE_INFO;
 
 	info_linear = bpf_program__get_prog_info_linear(fd, arrays);
 	if (IS_ERR_OR_NULL(info_linear)) {
@@ -151,8 +161,8 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 						     machine, process);
 	}
 
-	/* Synthesize PERF_RECORD_BPF_EVENT */
 	if (opts->bpf_event) {
+		/* Synthesize PERF_RECORD_BPF_EVENT */
 		*bpf_event = (struct bpf_event){
 			.header = {
 				.type = PERF_RECORD_BPF_EVENT,
@@ -165,6 +175,19 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 		memcpy(bpf_event->tag, info->tag, BPF_TAG_SIZE);
 		memset((void *)event + event->header.size, 0, machine->id_hdr_size);
 		event->header.size += machine->id_hdr_size;
+
+		/* save bpf_prog_info to env */
+		info_node = malloc(sizeof(struct bpf_prog_info_node));
+		if (info_node) {
+			info_node->info_linear = info_linear;
+			perf_env__insert_bpf_prog_info(env, info_node);
+			info_linear = NULL;
+		}
+
+		/*
+		 * process after saving bpf_prog_info to env, so that
+		 * required information is ready for look up
+		 */
 		err = perf_tool__process_synth_event(tool, event,
 						     machine, process);
 	}
@@ -175,7 +198,7 @@ static int perf_event__synthesize_one_bpf_prog(struct perf_tool *tool,
 	return err ? -1 : 0;
 }
 
-int perf_event__synthesize_bpf_events(struct perf_tool *tool,
+int perf_event__synthesize_bpf_events(struct perf_session *session,
 				      perf_event__handler_t process,
 				      struct machine *machine,
 				      struct record_opts *opts)
@@ -209,7 +232,7 @@ int perf_event__synthesize_bpf_events(struct perf_tool *tool,
 			continue;
 		}
 
-		err = perf_event__synthesize_one_bpf_prog(tool, process,
+		err = perf_event__synthesize_one_bpf_prog(session, process,
 							  machine, fd,
 							  event, opts);
 		close(fd);
diff --git a/tools/perf/util/bpf-event.h b/tools/perf/util/bpf-event.h
index 7890067e1a37..11e6730b6105 100644
--- a/tools/perf/util/bpf-event.h
+++ b/tools/perf/util/bpf-event.h
@@ -3,19 +3,28 @@
 #define __PERF_BPF_EVENT_H
 
 #include <linux/compiler.h>
+#include <bpf/bpf.h>
+#include <bpf/btf.h>
+#include <bpf/libbpf.h>
+#include <linux/btf.h>
+#include <linux/rbtree.h>
 #include "event.h"
 
 struct machine;
 union perf_event;
 struct perf_sample;
-struct perf_tool;
 struct record_opts;
 
+struct bpf_prog_info_node {
+	struct bpf_prog_info_linear	*info_linear;
+	struct rb_node			rb_node;
+};
+
 #ifdef HAVE_LIBBPF_SUPPORT
 int machine__process_bpf_event(struct machine *machine, union perf_event *event,
 			       struct perf_sample *sample);
 
-int perf_event__synthesize_bpf_events(struct perf_tool *tool,
+int perf_event__synthesize_bpf_events(struct perf_session *session,
 				      perf_event__handler_t process,
 				      struct machine *machine,
 				      struct record_opts *opts);
@@ -27,7 +36,7 @@ static inline int machine__process_bpf_event(struct machine *machine __maybe_unu
 	return 0;
 }
 
-static inline int perf_event__synthesize_bpf_events(struct perf_tool *tool __maybe_unused,
+static inline int perf_event__synthesize_bpf_events(struct perf_session *session __maybe_unused,
 						    perf_event__handler_t process __maybe_unused,
 						    struct machine *machine __maybe_unused,
 						    struct record_opts *opts __maybe_unused)
diff --git a/tools/perf/util/env.c b/tools/perf/util/env.c
index 4c23779e271a..665b6fe3c7b2 100644
--- a/tools/perf/util/env.c
+++ b/tools/perf/util/env.c
@@ -8,10 +8,86 @@
 
 struct perf_env perf_env;
 
+void perf_env__insert_bpf_prog_info(struct perf_env *env,
+				    struct bpf_prog_info_node *info_node)
+{
+	__u32 prog_id = info_node->info_linear->info.id;
+	struct bpf_prog_info_node *node;
+	struct rb_node *parent = NULL;
+	struct rb_node **p;
+
+	down_write(&env->bpf_info_lock);
+	p = &env->bpf_prog_infos.rb_node;
+
+	while (*p != NULL) {
+		parent = *p;
+		node = rb_entry(parent, struct bpf_prog_info_node, rb_node);
+		if (prog_id < node->info_linear->info.id) {
+			p = &(*p)->rb_left;
+		} else if (prog_id > node->info_linear->info.id) {
+			p = &(*p)->rb_right;
+		} else {
+			pr_debug("duplicated bpf prog info %u\n", prog_id);
+			up_write(&env->bpf_info_lock);
+			return;
+		}
+	}
+
+	rb_link_node(&info_node->rb_node, parent, p);
+	rb_insert_color(&info_node->rb_node, &env->bpf_prog_infos);
+	up_write(&env->bpf_info_lock);
+}
+
+struct bpf_prog_info_node *perf_env__find_bpf_prog_info(struct perf_env *env,
+							__u32 prog_id)
+{
+	struct bpf_prog_info_node *node = NULL;
+	struct rb_node *n;
+
+	down_read(&env->bpf_info_lock);
+	n = env->bpf_prog_infos.rb_node;
+
+	while (n) {
+		node = rb_entry(n, struct bpf_prog_info_node, rb_node);
+		if (prog_id < node->info_linear->info.id)
+			n = n->rb_left;
+		else if (prog_id > node->info_linear->info.id)
+			n = n->rb_right;
+		else
+			break;
+	}
+
+	up_read(&env->bpf_info_lock);
+	return node;
+}
+
+/* purge data in bpf_prog_infos tree */
+static void purge_bpf_info(struct perf_env *env)
+{
+	struct rb_root *root;
+	struct rb_node *next;
+
+	down_write(&env->bpf_info_lock);
+
+	root = &env->bpf_prog_infos;
+	next = rb_first(root);
+
+	while (next) {
+		struct bpf_prog_info_node *node;
+
+		node = rb_entry(next, struct bpf_prog_info_node, rb_node);
+		next = rb_next(&node->rb_node);
+		rb_erase_init(&node->rb_node, root);
+		free(node);
+	}
+	up_write(&env->bpf_info_lock);
+}
+
 void perf_env__exit(struct perf_env *env)
 {
 	int i;
 
+	purge_bpf_info(env);
 	zfree(&env->hostname);
 	zfree(&env->os_release);
 	zfree(&env->version);
@@ -38,6 +114,12 @@ void perf_env__exit(struct perf_env *env)
 	zfree(&env->memory_nodes);
 }
 
+static void init_bpf_rb_trees(struct perf_env *env)
+{
+	env->bpf_prog_infos = RB_ROOT;
+	init_rwsem(&env->bpf_info_lock);
+}
+
 int perf_env__set_cmdline(struct perf_env *env, int argc, const char *argv[])
 {
 	int i;
@@ -59,6 +141,7 @@ int perf_env__set_cmdline(struct perf_env *env, int argc, const char *argv[])
 
 	env->nr_cmdline = argc;
 
+	init_bpf_rb_trees(env);
 	return 0;
 out_free:
 	zfree(&env->cmdline_argv);
diff --git a/tools/perf/util/env.h b/tools/perf/util/env.h
index d01b8355f4ca..a6d25e91bc98 100644
--- a/tools/perf/util/env.h
+++ b/tools/perf/util/env.h
@@ -3,7 +3,10 @@
 #define __PERF_ENV_H
 
 #include <linux/types.h>
+#include <linux/rbtree.h>
 #include "cpumap.h"
+#include "rwsem.h"
+#include "bpf-event.h"
 
 struct cpu_topology_map {
 	int	socket_id;
@@ -64,6 +67,13 @@ struct perf_env {
 	struct memory_node	*memory_nodes;
 	unsigned long long	 memory_bsize;
 	u64                     clockid_res_ns;
+
+	/*
+	 * bpf_info_lock protects bpf rbtrees. This is needed because the
+	 * trees are accessed by different threads in perf-top
+	 */
+	struct rw_semaphore	bpf_info_lock;
+	struct rb_root		bpf_prog_infos;
 };
 
 extern struct perf_env perf_env;
@@ -80,4 +90,8 @@ const char *perf_env__arch(struct perf_env *env);
 const char *perf_env__raw_arch(struct perf_env *env);
 int perf_env__nr_cpus_avail(struct perf_env *env);
 
+void perf_env__insert_bpf_prog_info(struct perf_env *env,
+				    struct bpf_prog_info_node *info_node);
+struct bpf_prog_info_node *perf_env__find_bpf_prog_info(struct perf_env *env,
+							__u32 prog_id);
 #endif /* __PERF_ENV_H */
-- 
2.17.1


^ permalink raw reply related

* Re: [PATCH] net: phy: at803x: disable delay only for RGMII mode
From: Florian Fainelli @ 2019-02-15  0:14 UTC (permalink / raw)
  To: Niklas Cassel, Peter Ujfalusi
  Cc: Marc Gonzalez, Andrew Lunn, Vinod Koul, David S Miller,
	linux-arm-msm, Bjorn Andersson, netdev, Nori, Sekhar
In-Reply-To: <20190214150649.GA29957@centauri.lan>

On 2/14/19 7:06 AM, Niklas Cassel wrote:
> On Thu, Feb 14, 2019 at 03:22:28PM +0200, Peter Ujfalusi wrote:
>> Hi Niklas,
>>
>> On 14/02/2019 14.39, Niklas Cassel wrote:
>>>>> So, I've rebased your old patch, see attachment.
>>>>> I suggest that Peter test it on am335x-evm.
>>>>
>>>> with the patch + s/rgmii-txid/rgmii-id in the am335x-evmsk.dts ethernet
>>>> is working.
>>>> I don't have am335x-evm to test, but it has the same PHY as evmsk.
>>>>
>>>
>>> Florian's concern was that this PHY driver looked at "phy-mode" from the
>>> perspective of the MAC rather than the PHY.
>>> However, if s/rgmii-txid/rgmii-id is the correct fix for am335x-evm,
>>> then this means that this PHY driver was just broken.
>>>
>>> If the driver had misinterpreted the perspective, then the correct
>>> fix for am335x-evm would have been s/rgmii-txid/rgmii-rxid.
>>
>> Not sure if I got this right, but:
>> rgmii-id/txid/rxid is the delay mode between PHY and MAC, right?
>> on the PHY node it is from the PHY perspective, right?
> 
> Yes, from the PHY perspective.
> 
> (According to Florian, IIUC, some old PHY drivers were implemented before
> it was decided that it is from PHY perspective, rather than from MAC
> perspective.)
> 
>>
>> The errata I have mentioned for am335x say:
>> "The reset state of RGMII1_IDMODE (bit 4) and RGMII2_IDMODE (bit 5) in
>> the GMII_SEL register enables internal delay mode on the transmit clock
>> of the respective RGMII port. The AM335x device does not support
>> internal delay mode, so RGMII1_IDMODE and RGMII2_IDMODE must be set to 1b."
>>
>> If the delay mode on the transmit clock is not working on the am335x,
>> then this translate that the rxid needs to be enabled on the PHY side?
> 
> IIUC what Florian explained, then either MAC or PHY needs to add delays,
> so if the PHY only adds delay on e.g. TX, then the MAC needs to add delay
> on RX.
> 
> However, in your case, the errata says that your MAC is not capable of
> adding a delay on TX, therefore the PHY needs to add a delay on TX.

So that mandates specifying either 'rgmii-id' or 'rgmii-txid'.

> 
>>
>> But then why it worked when only the txid was enabled and rxid was not
>> on the PHY side, and why it works if both txid and rxid is enabled?
> 
> Because the PHY driver was broken, so the PHY driver always enabled
> delays on both TX and RX.
> 
> This is how the driver looked before Vinod's change:
> 
>         if (phydev->interface == PHY_INTERFACE_MODE_RGMII_RXID ||
>                         phydev->interface == PHY_INTERFACE_MODE_RGMII_ID) {
>                 ret = at803x_enable_rx_delay(phydev);
>                 if (ret < 0)
>                         return ret;
>         }
>  
>         if (phydev->interface == PHY_INTERFACE_MODE_RGMII_TXID ||
>                         phydev->interface == PHY_INTERFACE_MODE_RGMII_ID) {
>                 ret = at803x_enable_tx_delay(phydev);
>                 if (ret < 0)
>                         return ret;
>         }
> 
> 
> Yet, the initial value for this PHY is that both TX and RX delay is
> enabled, and since this driver never disabled TX/RX delays,
> the TX and RX delays were always enabled, no matter what phy-mode
> you specified.

Fixing all PHY drivers is probably too much, but since we seem to have
traction and people assigned to QCOM working on this at803x PHY driver,
then let's fix it correctly, I agree with that. So that means:

- disable both RX and TX delay by default
- if RGMII_RXID or RGMII_ID is specified: turn on RX delay
- if RGMII_TXID or RGMII_ID is specified: turn on TX delay

so basically the patch above, with an initial delay disabling for both
RX and TX. Then we fix all DTSes to be correct, at least those in tree
and we provide appropriate Fixes: tag so things get backported
automatically.

How does that sound?
-- 
Florian

^ permalink raw reply

* Re: [PATCH net] net: dsa: bcm_sf2: potential array overflow in bcm_sf2_sw_suspend()
From: Florian Fainelli @ 2019-02-15  0:31 UTC (permalink / raw)
  To: David Miller, dan.carpenter
  Cc: andrew, vivien.didelot, netdev, kernel-janitors
In-Reply-To: <20190213.221323.1768150771970232999.davem@davemloft.net>

On 2/13/19 10:13 PM, David Miller wrote:
> From: Dan Carpenter <dan.carpenter@oracle.com>
> Date: Wed, 13 Feb 2019 11:23:04 +0300
> 
>> The value of ->num_ports comes from bcm_sf2_sw_probe() and it is less
>> than or equal to DSA_MAX_PORTS.  The ds->ports[] array is used inside
>> the dsa_is_user_port() and dsa_is_cpu_port() functions.  The ds->ports[]
>> array is allocated in dsa_switch_alloc() and it has ds->num_ports
>> elements so this leads to a static checker warning about a potential out
>> of bounds read.

This would not happen here because bcm_sf2 calls b53_switch_alloc()
which does allocate the full port range (not for a good reason), but
it's good to fix that anyways.

>>
>> Fixes: 8cfa94984c9c ("net: dsa: bcm_sf2: add suspend/resume callbacks")
>> Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com>
> 
> Applied.
> 


-- 
Florian

^ permalink raw reply

* Re: Three questions about busy poll
From: Willem de Bruijn @ 2019-02-15  0:39 UTC (permalink / raw)
  To: Cong Wang
  Cc: Alexander Duyck, Eric Dumazet, sridhar.samudrala,
	Linux Kernel Network Developers
In-Reply-To: <CAM_iQpX3fAHwX7u4Nne5zdjXXnMY=Y5HYde2rYGSHSQugqc_YQ@mail.gmail.com>

On Thu, Feb 14, 2019 at 3:15 PM Cong Wang <xiyou.wangcong@gmail.com> wrote:
>
> Hello,
>
> While looking into the busy polling in Linux kernel, three questions
> come into my mind:
>
> 1. In the document[1], it claims sysctl.net.busy_poll depends on
> either SO_BUSY_POLL or sysctl.net.busy_read. However, from the code in
> ep_set_busy_poll_napi_id(), I don't see such a dependency. It simply
> checks sysctl_net_busy_poll and sk->sk_napi_id, but sk->sk_napi_id is
> always set as long as we enable CONFIG_NET_RX_BUSY_POLL. So what I am
> missing here?

That documentation refers to sock_poll. This does call sk_busy_loop
individually on each socket in the pollset and thus respects those values.
Epoll was added later, after both sock_poll and that documentation.

> 2. Why there is no socket option for sysctl.net.busy_poll? Clearly
> sysctl_net_busy_poll is global and SO_BUSY_POLL only works for
> sysctl.net.busy_read.

I guess because of how sock_poll works. In that case it is not needed.
The poll duration applies more to the pollset than any of the
individual sockets, too.

> 3. How is SO_INCOMING_NAPI_ID supposed to be used? I can't find any
> useful documents online. Any example or more detailed doc?

From the commit message of 6d4339028b35 ("net: Introduce
SO_INCOMING_NAPI_ID") it sounds like a sharding mechanism that
maintains flow affinity by sharding based on rxqueue (assuming that
something like RSS was used to ensure flow affinity in the first
place).

^ permalink raw reply

* Re: [PATCH v2] kcm: remove any offset before parsing messages
From: Dominique Martinet @ 2019-02-15  1:00 UTC (permalink / raw)
  To: David Miller; +Cc: doronrk, tom, davejwatson, netdev, linux-kernel
In-Reply-To: <20181031025657.GA17861@nautica>

Dominique Martinet wrote on Wed, Oct 31, 2018:
> Anyway, that probably explains I have no problem with bigger VM
> (uselessly more memory available) or without KASAN (I guess there's
> overhead?), but I'm sending at most 300k of data and the VM has a 1.5GB
> of ram, so if there's an allocation failure there I think there's a
> problem ! . . .
> 
> So, well, I'm not sure on the way forward. Adding a bpf helper and
> document that kcm users should mind the offset?

bump on this - I had mostly forgotten about it but the nfs-ganesha
community that could make use of KCM reminded me of my patch that's
waiting for this.

Summary for people coming back after four months:
 - kcm is great, but the bpf function that's supposed to be called for
each packet does not automatically adjust the offset so that it can
assume the skb starts with the packet it needs to look at

 - there is some workaround code that is far from obvious and
undocumented, see the original thread[1]:
[1] http://lkml.kernel.org/r/20180822183852.jnwlxnz54gbbf6po@davejwatson-mba.dhcp.thefacebook.com

 - my patch here tried to automatically pull the corresponding packet to
the front, but apparently with KASAN can trigger out of memory
behaviours on "small" VMs, so even if it doesn't seem to impact
performance much without KASAN I don't think it's really ok to
potentially hang the connection due to oom under severe conditions.


The best alternative I see is adding a proper helper to get
"kcm_rx_msg(skb)->offset" from bpf and document it so users aren't as
lost as I have been; I'm not quite sure how/where to add such a helper
though as I've barely looked at the bpf code until now, but should we go
for that?


(it really feels wrong to me that some random person who just started by
trying to use kcm has to put this much effort to keep the ball rolling,
if nobody cares about kcm I'm also open to have it removed completely
despite the obvious performance gain I benchmarked for ganesha[2] ;
barely maintained feature is worse than no feature)

[2] https://review.gerrithub.io/c/ffilz/nfs-ganesha/+/421314

Thanks,
-- 
Dominique

^ permalink raw reply

* Re: [PATCH net] net: stmmac: Fix NAPI poll in TX path when in multi-queue
From: Florian Fainelli @ 2019-02-15  1:06 UTC (permalink / raw)
  To: David Miller, jose.abreu
  Cc: netdev, linux-kernel, joao.pinto, peppe.cavallaro,
	alexandre.torgue
In-Reply-To: <20190214.090107.1088098621242009425.davem@davemloft.net>

On 2/14/19 9:01 AM, David Miller wrote:
> From: Jose Abreu <jose.abreu@synopsys.com>
> Date: Wed, 13 Feb 2019 18:00:43 +0100
> 
>> Commit 8fce33317023 introduced the concept of NAPI per-channel and
>> independent cleaning of TX path.
>>
>> This is currently breaking performance in some cases. The scenario
>> happens when all packets are being received in Queue 0 but the TX is
>> performed in Queue != 0.
>>
>> I didn't look very deep but it seems that NAPI for Queue 0 will clean
>> the RX path but as TX is in different NAPI, this last one is called at a
>> slower rate which kills performance in TX. I suspect this is due to TX
>> cleaning takes much longer than RX and because NAPI will get canceled
>> once we return with 0 budget consumed (e.g. when TX is still not done it
>> will return 0 budget).
>>
>> Fix this by looking at all TX channels in NAPI poll function.
>>
>> Signed-off-by: Jose Abreu <joabreu@synopsys.com>
>> Fixes: 8fce33317023 ("net: stmmac: Rework coalesce timer and fix multi-queue races")
> 
> No this isn't right.
> 
> The TX interrupt events for Queue != 0 should clean up the TX packets
> on those queues.
> 
> Furthermore you are breaking the locality of the TX processing.
> 
> I'm not applying this, sorry.

Agreed, why don't you create per-queue NAPI instances such that they are
all independent and can complete their TX completion/RX processing
entirely separately?
-- 
Florian

^ permalink raw reply

* Re: (2) [Bug reporting] kernel panic during handle the dst unreach icmp msg.
From: Eric Dumazet @ 2019-02-15  1:08 UTC (permalink / raw)
  To: soukjin.bae, netdev@vger.kernel.org
  Cc: 박종언, Steffen Klassert, Herbert Xu
In-Reply-To: <20190214235427epcms1p25712530506811b6279ffbe6c54914e72@epcms1p2>



On 02/14/2019 03:54 PM, 배석진 wrote:
>> I do not believe this patch is needed.
>>  
>> You probably hit another more serious bug, but since you do not post the full stack trace
>> it is hard to help.
>>  
>> Are you using vti tunnel ?
> 
> there's no working logs of vpn/vti/tun on platform or kernel history.
> and callstack has no functions about that.
> it looks like simple ipv4 usage.
> 
> attaching full dump info.
> no additional bug or warning on entire kernel history.
> anything about tun or kasan even onther.
> 
> thanks.
> 
> 
> <4>[60392.948306] I[1:    ksoftirqd/1:   19] ------------[ cut here ]------------
> <0>[60392.948334] I[1:    ksoftirqd/1:   19] kernel BUG at net/ipv4/tcp_ipv4.c:519!

What the code looks like at line 519 of net/ipv4/tcp_ipv4.c ?

This is not a pristine kernel, anything could be wrong, you might have missed some critical backports.


> <2>[60392.948344] I[1:    ksoftirqd/1:   19] sec_debug_set_extra_info_fault = BUG / 0xffffff80090351d0
> <0>[60392.948386] I[1:    ksoftirqd/1:   19] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> <4>[60392.948395] I[1:    ksoftirqd/1:   19] Modules linked in:
> <0>[60392.948408] I[1:    ksoftirqd/1:   19] debug-snapshot: core register saved(CPU:1)
> <0>[60392.948416] I[1:    ksoftirqd/1:   19] DISR_EL1: 0000000000000000
> <0>[60392.948425] I[1:    ksoftirqd/1:   19] DISR_EL1 valid_bit(31) is NOT set (0x0)
> <0>[60392.948433] I[1:    ksoftirqd/1:   19] CPU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
> <0>[60392.948441] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
> <0>[60392.948449] I[1:    ksoftirqd/1:   19] DSU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
> <0>[60392.948457] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
> <0>[60392.948464] I[1:    ksoftirqd/1:   19] debug-snapshot: context saved(CPU:1)
> <0>[60392.950562] I[1:    ksoftirqd/1:   19] ============ CPU Error Log Info after Cache flush ================
> <0>[60392.950579] I[1:    ksoftirqd/1:   19] DISR_EL1: 0000000000000000
> <0>[60392.950588] I[1:    ksoftirqd/1:   19] DISR_EL1 valid_bit(31) is NOT set (0x0)
> <0>[60392.950596] I[1:    ksoftirqd/1:   19] CPU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
> <0>[60392.950604] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
> <0>[60392.950611] I[1:    ksoftirqd/1:   19] DSU : ERXSTATUS_EL1: 0000000000000000, ERXADDR_EL1: 0000000000000000, ERXMISC0_EL1: 0000000000000000
> <0>[60392.950619] I[1:    ksoftirqd/1:   19] ERXSTATUS_EL1 valid_bit(30) is NOT set (0x0)
> <6>[60392.950631] I[1:    ksoftirqd/1:   19] debug-snapshot: item - log_kevents is disabled
> <6>[60392.950638] I[1:    ksoftirqd/1:   19] TIF_FOREIGN_FPSTATE: 0, FP/SIMD depth 0, cpu: 0
> <4>[60392.950648] I[1:    ksoftirqd/1:   19] CPU: 1 PID: 19 Comm: ksoftirqd/1 Tainted: G        W       4.14.85-15389299 #1
> <4>[60392.950655] I[1:    ksoftirqd/1:   19] Hardware name: Samsung BEYOND2LTE KOR SINGLE 26 board based on EXYNOS9820 (DT)
> <4>[60392.950664] I[1:    ksoftirqd/1:   19] task: ffffffc8f0c8c400 task.stack: ffffff800b958000
> <4>[60392.950676] I[1:    ksoftirqd/1:   19] PC is at tcp_v4_err+0x4b0/0x4bc
> <4>[60392.950684] I[1:    ksoftirqd/1:   19] LR is at tcp_v4_err+0x3ac/0x4bc
> <4>[60392.950692] I[1:    ksoftirqd/1:   19] pc : [<ffffff80090351d0>] lr : [<ffffff80090350cc>] pstate: 60c00145
> <4>[60392.950698] I[1:    ksoftirqd/1:   19] sp : ffffff800b95b940
> <4>[60392.950705] I[1:    ksoftirqd/1:   19] x29: ffffff800b95b990 x28: ffffffc82fe10c2c 
> <4>[60392.950714] I[1:    ksoftirqd/1:   19] x27: 0000000000000003 x26: 0000000000000001 
> <4>[60392.950723] I[1:    ksoftirqd/1:   19] x25: ffffffc0051e5e00 x24: ffffff800aaebc80 
> <4>[60392.950732] I[1:    ksoftirqd/1:   19] x23: ffffffc0051e5f58 x22: 0000000000000000 
> <4>[60392.950741] I[1:    ksoftirqd/1:   19] x21: 0000000000000071 x20: ffffffc0051e5e88 
> <4>[60392.950750] I[1:    ksoftirqd/1:   19] x19: ffffffc0051e5e00 x18: 0000000000000001 
> <4>[60392.950760] I[1:    ksoftirqd/1:   19] x17: 0000000000000400 x16: 0000000000000000 
> <4>[60392.950769] I[1:    ksoftirqd/1:   19] x15: ffffffffffffffff x14: 0000000000000000 
> <4>[60392.950778] I[1:    ksoftirqd/1:   19] x13: 0000000000000002 x12: 0000000000000000 
> <4>[60392.950787] I[1:    ksoftirqd/1:   19] x11: ffffffc0051e5f58 x10: 0000000000007530 
> <4>[60392.950796] I[1:    ksoftirqd/1:   19] x9 : 0000000000000020 x8 : 0000000000000200 
> <4>[60392.950806] I[1:    ksoftirqd/1:   19] x7 : 0000000000000000 x6 : 0000000000000010 
> <4>[60392.950815] I[1:    ksoftirqd/1:   19] x5 : 000000000000808c x4 : 000000000000bb01 
> <4>[60392.950825] I[1:    ksoftirqd/1:   19] x3 : 00000000831fd9ac x2 : 000000000000808c 
> <4>[60392.950834] I[1:    ksoftirqd/1:   19] x1 : 000000000200a8c0 x0 : 0000000000000040 
> <4>[60392.950844] I[1:    ksoftirqd/1:   19] 
> <4>[60392.950844] I[1:    ksoftirqd/1:   19] PC: 0xffffff80090350d0:
> <4>[60392.950855] I[1:    ksoftirqd/1:   19] 50d0 : 92407C08 14000009 AA1303E0 97FFFE97 17FFFFD0 F940CB08 D538D089 9108E108
> <4>[60392.950872] I[1:    ksoftirqd/1:   19] 50f0 : 17FFFF78 52801F48 12001D29 F940AE77 9AC92108 D344FD09 528EA60A 9105626B
> <4>[60392.950888] I[1:    ksoftirqd/1:   19] 5110 : F11D4D3F 9A8A3108 EB0B02FF B9072268 54000580 B4000577 97C9A4D3 D29EF9EA
> <4>[60392.950903] I[1:    ksoftirqd/1:   19] 5130 : F2BC6A6A F9444A68 F2D374AA D343FC09 F2E4188A 9BCA7D29 D344FD29 EB08013F
> <4>[60392.950919] I[1:    ksoftirqd/1:   19] 5150 : 54000069 AA0903E8 F9044A69 B94022E9 B9472277 4B090100 97C94F99 CB0002E8
> <4>[60392.950934] I[1:    ksoftirqd/1:   19] 5170 : 7100051F 5400022B 320003EA B000A74B 391D4A6A 93407D08 F945416A D344FD0B
> <4>[60392.950951] I[1:    ksoftirqd/1:   19] 5190 : 528EA609 F11D4D7F 9A893108 8B080142 911A4261 AA1303E0 F9034662 97FCBBB7
> <4>[60392.950967] I[1:    ksoftirqd/1:   19] 51b0 : AA1603E8 17FFFF86 F9455668 AA1303E0 F9402D08 94063788 AA1603E8 17FFFF80
> <4>[60392.950982] I[1:    ksoftirqd/1:   19] 51d0 : D4210000 14000000 00BE7BAD F81C0FF7 A90157F6 A9024FF4 A9037BFD 9100C3FD
> <4>[60392.950997] I[1:    ksoftirqd/1:   19] 51f0 : AA0003F6 B940AAC8 F9407AC9 7941BECA B9409AD7 2A2803E8 2A0203F4 2A0103F5
> <4>[60392.951012] I[1:    ksoftirqd/1:   19] 5210 : 721B051F 8B0A0133 54000201 321F07E3 2A1503E0 2A1403E1 2A1703E2 2A1F03E4
> <4>[60392.951028] I[1:    ksoftirqd/1:   19] 5230 : 97D8189D 13804008 0B000108 53107D08 79002268 7941BEC8 321C03E9 790166C9
> <4>[60392.951043] I[1:    ksoftirqd/1:   19] 5250 : 790162C8 14000012 79401A68 B940B2C2 AA1303E0 53027D08 121E0D01 97D8177A
> <4>[60392.951058] I[1:    ksoftirqd/1:   19] 5270 : 2A0003E4 321F07E3 2A1503E0 2A1403E1 2A1703E2 97D81888 13804008 0B000108
> <4>[60392.951073] I[1:    ksoftirqd/1:   19] 5290 : 12800009 4A484128 79002268 A9437BFD A9424FF4 A94157F6 F84407F7 D65F03C0
> <4>[60392.951089] I[1:    ksoftirqd/1:   19] 52b0 : 00BE7BAD A9BF7BFD 910003FD B9458008 B9400002 AA0103E0 2A0803E1 97FFFFC4
> <4>[60392.951105] I[1:    ksoftirqd/1:   19] 
> <4>[60392.951105] I[1:    ksoftirqd/1:   19] LR: 0xffffff8009034fcc:
> <4>[60392.951116] I[1:    ksoftirqd/1:   19] 4fcc : 39404A69 121F1929 7100093F 540001A1 B4000068 F9407508 B4000148 B9408F28
> <4>[60392.951131] I[1:    ksoftirqd/1:   19] 4fec : 35000188 F942A668 AA1303E0 B9022E75 940637FA AA1303E0 97FFAE89 14000006
> <4>[60392.951147] I[1:    ksoftirqd/1:   19] 500c : B9408F28 35000068 39568268 37000208 B9023275 AA1403E0 94061649 91020260
> <4>[60392.951162] I[1:    ksoftirqd/1:   19] 502c : 97D6FDEE 36000060 AA1303E0 97FCB533 A9457BFD A9444FF4 A94357F6 A9425FF8
> <4>[60392.951177] I[1:    ksoftirqd/1:   19] 504c : A94167FA A8C66FFC D65F03C0 F942A668 AA1303E0 B9022E75 940637E0 17FFFFEE
> <4>[60392.951193] I[1:    ksoftirqd/1:   19] 506c : D00030CB D37DF34A 9114816B B86A6975 7100075F 54FFFA68 B9400129 6B0902FF
> <4>[60392.951208] I[1:    ksoftirqd/1:   19] 508c : 54FFFA01 395D4669 34FFF9C9 B5FFF9A8 395D4E69 34FFF969 B9408F2A 35FFF92A
> <4>[60392.951223] I[1:    ksoftirqd/1:   19] 50ac : AA0803F6 B9489A68 51000529 391D4E69 340001C8 B948A669 0B480D20 97C94FC1
> <4>[60392.951238] I[1:    ksoftirqd/1:   19] 50cc : 395D4E69 92407C08 14000009 AA1303E0 97FFFE97 17FFFFD0 F940CB08 D538D089
> <4>[60392.951254] I[1:    ksoftirqd/1:   19] 50ec : 9108E108 17FFFF78 52801F48 12001D29 F940AE77 9AC92108 D344FD09 528EA60A
> <4>[60392.951269] I[1:    ksoftirqd/1:   19] 510c : 9105626B F11D4D3F 9A8A3108 EB0B02FF B9072268 54000580 B4000577 97C9A4D3
> <4>[60392.951283] I[1:    ksoftirqd/1:   19] 512c : D29EF9EA F2BC6A6A F9444A68 F2D374AA D343FC09 F2E4188A 9BCA7D29 D344FD29
> <4>[60392.951299] I[1:    ksoftirqd/1:   19] 514c : EB08013F 54000069 AA0903E8 F9044A69 B94022E9 B9472277 4B090100 97C94F99
> <4>[60392.951314] I[1:    ksoftirqd/1:   19] 516c : CB0002E8 7100051F 5400022B 320003EA B000A74B 391D4A6A 93407D08 F945416A
> <4>[60392.951328] I[1:    ksoftirqd/1:   19] 518c : D344FD0B 528EA609 F11D4D7F 9A893108 8B080142 911A4261 AA1303E0 F9034662
> <4>[60392.951344] I[1:    ksoftirqd/1:   19] 51ac : 97FCBBB7 AA1603E8 17FFFF86 F9455668 AA1303E0 F9402D08 94063788 AA1603E8
> <4>[60392.951358] I[1:    ksoftirqd/1:   19] 
> <4>[60392.951358] I[1:    ksoftirqd/1:   19] SP: 0xffffff800b95b840:
> <4>[60392.951369] I[1:    ksoftirqd/1:   19] b840 : 0B95BA88 FFFFFF80 00002752 00000000 0B95B9A0 FFFFFF80 00000001 00000000
> <4>[60392.951384] I[1:    ksoftirqd/1:   19] b860 : 00000001 00000000 051E5E00 FFFFFFC0 0B95B8E0 FFFFFF80 09001E50 FFFFFF80
> <4>[60392.951398] I[1:    ksoftirqd/1:   19] b880 : 0B95B8E0 FFFFFF80 EE595D00 B254C55C EF89B000 FFFFFFC8 EE595D00 B254C55C
> <4>[60392.951412] I[1:    ksoftirqd/1:   19] b8a0 : 2FE10C24 FFFFFFC8 36B80010 FFFFFFC0 0B2D8880 FFFFFF80 0AAEBC80 FFFFFF80
> <4>[60392.951426] I[1:    ksoftirqd/1:   19] b8c0 : 00000010 00000000 00000000 00000000 0B95B930 FFFFFF80 09015704 FFFFFF80
> <4>[60392.951440] I[1:    ksoftirqd/1:   19] b8e0 : 00000003 00000000 0906F3D0 FFFFFF80 00000001 00000000 EF89B000 FFFFFFC8
> <4>[60392.951454] I[1:    ksoftirqd/1:   19] b900 : 0AAEBC80 FFFFFF80 2FE10C24 FFFFFFC8 00000000 00000000 00000000 FFFFFFC8
> <4>[60392.951468] I[1:    ksoftirqd/1:   19] b920 : 00780078 FFFFFFC8 EE595D00 B254C55C 0B95B990 FFFFFF80 09034E40 FFFFFF80
> <4>[60392.951482] I[1:    ksoftirqd/1:   19] b940 : 0A5386B0 FFFFFF80 0A540FD0 FFFFFF80 EF89B0A0 FFFFFFC8 EF89B000 FFFFFFC8
> <4>[60392.951496] I[1:    ksoftirqd/1:   19] b960 : 00000008 00000000 2FE10C24 FFFFFFC8 00000000 00000000 00000006 00000000
> <4>[60392.951511] I[1:    ksoftirqd/1:   19] b980 : 00000000 00000000 1EEBFB80 FFFFFFC8 0B95B9C0 FFFFFF80 0904A990 FFFFFF80
> <4>[60392.951526] I[1:    ksoftirqd/1:   19] b9a0 : 0AAEBC80 FFFFFF80 36B82128 FFFFFFC0 2FE10C2C FFFFFFC8 1EEBFB80 FFFFFFC8
> <4>[60392.951541] I[1:    ksoftirqd/1:   19] b9c0 : 0B95BA00 FFFFFF80 0904A6D0 FFFFFF80 1EEBFC18 FFFFFFC8 EE595D00 B254C55C
> <4>[60392.951556] I[1:    ksoftirqd/1:   19] b9e0 : 0A540578 FFFFFF80 12995900 FFFFFFC0 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
> <4>[60392.951570] I[1:    ksoftirqd/1:   19] ba00 : 0B95BA40 FFFFFF80 0904A3AC FFFFFF80 00000008 00000000 1EEBFC18 FFFFFFC8
> <4>[60392.951586] I[1:    ksoftirqd/1:   19] ba20 : 0A540578 FFFFFF80 0964F9E8 FFFFFF80 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
> <4>[60392.951605] I[1:    ksoftirqd/1:   19] 
> <4>[60392.951605] I[1:    ksoftirqd/1:   19] X7: 0xffffffffffffff00:
> <4>[60392.951616] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951637] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951655] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951671] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951696] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951713] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951730] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951747] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951764] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951780] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951797] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951814] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951831] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951849] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951867] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951884] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.951902] I[1:    ksoftirqd/1:   19] 
> <4>[60392.951902] I[1:    ksoftirqd/1:   19] X11: 0xffffffc0051e5e58:
> <4>[60392.951914] I[1:    ksoftirqd/1:   19] 5e58 : 00017031 00000000 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8
> <4>[60392.951930] I[1:    ksoftirqd/1:   19] 5e78 : FFFFFFFF 00000004 00000003 00000000 00790078 00000000 00000000 00000000
> <4>[60392.951945] I[1:    ksoftirqd/1:   19] 5e98 : 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0
> <4>[60392.951961] I[1:    ksoftirqd/1:   19] 5eb8 : 051E5EB0 FFFFFFC0 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0
> <4>[60392.951975] I[1:    ksoftirqd/1:   19] 5ed8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.951990] I[1:    ksoftirqd/1:   19] 5ef8 : 00000000 00000000 00000000 00123900 00000000 00000000 00000000 00000000
> <4>[60392.952005] I[1:    ksoftirqd/1:   19] 5f18 : 00000000 00000000 00000000 00000000 42FCE000 FFFFFFC8 00000000 00000000
> <4>[60392.952020] I[1:    ksoftirqd/1:   19] 5f38 : 00000000 00100000 00000000 00000001 00000000 00000000 00000000 00000000
> <4>[60392.952035] I[1:    ksoftirqd/1:   19] 5f58 : 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0 00000000 00000000 FFFFFFFF 00000000
> <4>[60392.952050] I[1:    ksoftirqd/1:   19] 5f78 : 00000000 00000000 FFFFFFFF 7FFFFFFF 00000000 00000000 00000000 00000000
> <4>[60392.952065] I[1:    ksoftirqd/1:   19] 5f98 : 00000000 00000000 090343B0 FFFFFF80 051E5E00 FFFFFFC0 00000001 FFFFFFFF
> <4>[60392.952079] I[1:    ksoftirqd/1:   19] 5fb8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000C026F
> <4>[60392.952094] I[1:    ksoftirqd/1:   19] 5fd8 : 0007C913 FFFFFFFF 00000000 00000000 00000000 00000000 00006000 00000000
> <4>[60392.952109] I[1:    ksoftirqd/1:   19] 5ff8 : 00000000 00000000 00000001 00000000 014000C0 43A820B3 00010600 00000001
> <4>[60392.952124] I[1:    ksoftirqd/1:   19] 6018 : 00000000 00000000 0AAF9E10 FFFFFF80 00000000 00000000 00000071 00000000
> <4>[60392.952139] I[1:    ksoftirqd/1:   19] 6038 : 00000000 00002752 00000000 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
> <4>[60392.952155] I[1:    ksoftirqd/1:   19] 
> <4>[60392.952155] I[1:    ksoftirqd/1:   19] X12: 0xffffffffffffff00:
> <4>[60392.952166] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952183] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952199] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952216] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952232] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952249] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952266] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952283] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952300] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952317] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952333] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952350] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952366] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952383] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952401] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952417] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952436] I[1:    ksoftirqd/1:   19] 
> <4>[60392.952436] I[1:    ksoftirqd/1:   19] X14: 0xffffffffffffff00:
> <4>[60392.952447] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952467] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952483] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952499] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952516] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952532] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952548] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952563] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952580] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952596] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952612] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952628] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952645] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952661] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952677] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952693] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952710] I[1:    ksoftirqd/1:   19] 
> <4>[60392.952710] I[1:    ksoftirqd/1:   19] X15: 0xfffffffffffffeff:
> <4>[60392.952721] I[1:    ksoftirqd/1:   19] fefc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952737] I[1:    ksoftirqd/1:   19] ff1c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952753] I[1:    ksoftirqd/1:   19] ff3c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952768] I[1:    ksoftirqd/1:   19] ff5c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952784] I[1:    ksoftirqd/1:   19] ff7c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952799] I[1:    ksoftirqd/1:   19] ff9c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952815] I[1:    ksoftirqd/1:   19] ffbc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952830] I[1:    ksoftirqd/1:   19] ffdc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952846] I[1:    ksoftirqd/1:   19] fffc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952861] I[1:    ksoftirqd/1:   19] 001c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952877] I[1:    ksoftirqd/1:   19] 003c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952893] I[1:    ksoftirqd/1:   19] 005c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952910] I[1:    ksoftirqd/1:   19] 007c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952926] I[1:    ksoftirqd/1:   19] 009c : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952944] I[1:    ksoftirqd/1:   19] 00bc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952960] I[1:    ksoftirqd/1:   19] 00dc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952976] I[1:    ksoftirqd/1:   19] 00fc : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.952993] I[1:    ksoftirqd/1:   19] 
> <4>[60392.952993] I[1:    ksoftirqd/1:   19] X16: 0xffffffffffffff00:
> <4>[60392.953004] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953020] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953036] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953052] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953068] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953084] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953099] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953115] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953130] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953146] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953162] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953178] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953194] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953210] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953226] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953242] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953259] I[1:    ksoftirqd/1:   19] 
> <4>[60392.953259] I[1:    ksoftirqd/1:   19] X19: 0xffffffc0051e5d00:
> <4>[60392.953270] I[1:    ksoftirqd/1:   19] 5d00 : 00000000 00000000 00000000 00000000 E745D5E7 496DFBF1 00000000 00000000
> <4>[60392.953285] I[1:    ksoftirqd/1:   19] 5d20 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953300] I[1:    ksoftirqd/1:   19] 5d40 : 00000000 00000000 00000000 00000000 00000000 00000000 FFFF0000 0200A8C0
> <4>[60392.953314] I[1:    ksoftirqd/1:   19] 5d60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953328] I[1:    ksoftirqd/1:   19] 5d80 : 00000000 00000000 0081FF80 00000000 00000000 00080000 00000000 00000000
> <4>[60392.953342] I[1:    ksoftirqd/1:   19] 5da0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953356] I[1:    ksoftirqd/1:   19] 5dc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953370] I[1:    ksoftirqd/1:   19] 5de0 : 00000000 00000000 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D
> <4>[60392.953384] I[1:    ksoftirqd/1:   19] 5e00 : 831FD9AC 0200A8C0 EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000
> <4>[60392.953398] I[1:    ksoftirqd/1:   19] 5e20 : BC8B82B8 FFFFFFC8 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000
> <4>[60392.953412] I[1:    ksoftirqd/1:   19] 5e40 : FFFF0000 831FD9AC 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000
> <4>[60392.953427] I[1:    ksoftirqd/1:   19] 5e60 : 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004
> <4>[60392.953441] I[1:    ksoftirqd/1:   19] 5e80 : 00000003 00000000 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0
> <4>[60392.953455] I[1:    ksoftirqd/1:   19] 5ea0 : 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0
> <4>[60392.953469] I[1:    ksoftirqd/1:   19] 5ec0 : 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000
> <4>[60392.953483] I[1:    ksoftirqd/1:   19] 5ee0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953498] I[1:    ksoftirqd/1:   19] 
> <4>[60392.953498] I[1:    ksoftirqd/1:   19] X20: 0xffffffc0051e5d88:
> <4>[60392.953509] I[1:    ksoftirqd/1:   19] 5d88 : 0081FF80 00000000 00000000 00080000 00000000 00000000 00000000 00000000
> <4>[60392.953523] I[1:    ksoftirqd/1:   19] 5da8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953537] I[1:    ksoftirqd/1:   19] 5dc8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953551] I[1:    ksoftirqd/1:   19] 5de8 : 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D 831FD9AC 0200A8C0
> <4>[60392.953565] I[1:    ksoftirqd/1:   19] 5e08 : EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000 BC8B82B8 FFFFFFC8
> <4>[60392.953580] I[1:    ksoftirqd/1:   19] 5e28 : 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000 FFFF0000 831FD9AC
> <4>[60392.953594] I[1:    ksoftirqd/1:   19] 5e48 : 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000 00004301 00000000
> <4>[60392.953608] I[1:    ksoftirqd/1:   19] 5e68 : 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004 00000003 00000000
> <4>[60392.953622] I[1:    ksoftirqd/1:   19] 5e88 : 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0
> <4>[60392.953636] I[1:    ksoftirqd/1:   19] 5ea8 : 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0 00000000 00000000
> <4>[60392.953650] I[1:    ksoftirqd/1:   19] 5ec8 : 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000 00000000 00000000
> <4>[60392.953664] I[1:    ksoftirqd/1:   19] 5ee8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00123900
> <4>[60392.953678] I[1:    ksoftirqd/1:   19] 5f08 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.953692] I[1:    ksoftirqd/1:   19] 5f28 : 42FCE000 FFFFFFC8 00000000 00000000 00000000 00100000 00000000 00000001
> <4>[60392.953706] I[1:    ksoftirqd/1:   19] 5f48 : 00000000 00000000 00000000 00000000 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0
> <4>[60392.953720] I[1:    ksoftirqd/1:   19] 5f68 : 00000000 00000000 FFFFFFFF 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
> <4>[60392.953736] I[1:    ksoftirqd/1:   19] 
> <4>[60392.953736] I[1:    ksoftirqd/1:   19] X22: 0xffffffffffffff00:
> <4>[60392.953747] I[1:    ksoftirqd/1:   19] ff00 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953764] I[1:    ksoftirqd/1:   19] ff20 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953779] I[1:    ksoftirqd/1:   19] ff40 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953795] I[1:    ksoftirqd/1:   19] ff60 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953813] I[1:    ksoftirqd/1:   19] ff80 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953830] I[1:    ksoftirqd/1:   19] ffa0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953846] I[1:    ksoftirqd/1:   19] ffc0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953862] I[1:    ksoftirqd/1:   19] ffe0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953877] I[1:    ksoftirqd/1:   19] 0000 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953893] I[1:    ksoftirqd/1:   19] 0020 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953909] I[1:    ksoftirqd/1:   19] 0040 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953925] I[1:    ksoftirqd/1:   19] 0060 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953941] I[1:    ksoftirqd/1:   19] 0080 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953957] I[1:    ksoftirqd/1:   19] 00a0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953973] I[1:    ksoftirqd/1:   19] 00c0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.953989] I[1:    ksoftirqd/1:   19] 00e0 : ******** ******** ******** ******** ******** ******** ******** ********
> <4>[60392.954007] I[1:    ksoftirqd/1:   19] 
> <4>[60392.954007] I[1:    ksoftirqd/1:   19] X23: 0xffffffc0051e5e58:
> <4>[60392.954019] I[1:    ksoftirqd/1:   19] 5e58 : 00017031 00000000 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8
> <4>[60392.954034] I[1:    ksoftirqd/1:   19] 5e78 : FFFFFFFF 00000004 00000003 00000000 00790078 00000000 00000000 00000000
> <4>[60392.954048] I[1:    ksoftirqd/1:   19] 5e98 : 051E5E98 FFFFFFC0 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0
> <4>[60392.954062] I[1:    ksoftirqd/1:   19] 5eb8 : 051E5EB0 FFFFFFC0 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0
> <4>[60392.954077] I[1:    ksoftirqd/1:   19] 5ed8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954091] I[1:    ksoftirqd/1:   19] 5ef8 : 00000000 00000000 00000000 00123900 00000000 00000000 00000000 00000000
> <4>[60392.954105] I[1:    ksoftirqd/1:   19] 5f18 : 00000000 00000000 00000000 00000000 42FCE000 FFFFFFC8 00000000 00000000
> <4>[60392.954119] I[1:    ksoftirqd/1:   19] 5f38 : 00000000 00100000 00000000 00000001 00000000 00000000 00000000 00000000
> <4>[60392.954135] I[1:    ksoftirqd/1:   19] 5f58 : 051E5F58 FFFFFFC0 051E5F58 FFFFFFC0 00000000 00000000 FFFFFFFF 00000000
> <4>[60392.954149] I[1:    ksoftirqd/1:   19] 5f78 : 00000000 00000000 FFFFFFFF 7FFFFFFF 00000000 00000000 00000000 00000000
> <4>[60392.954163] I[1:    ksoftirqd/1:   19] 5f98 : 00000000 00000000 090343B0 FFFFFF80 051E5E00 FFFFFFC0 00000001 FFFFFFFF
> <4>[60392.954178] I[1:    ksoftirqd/1:   19] 5fb8 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 000C026F
> <4>[60392.954192] I[1:    ksoftirqd/1:   19] 5fd8 : 0007C913 FFFFFFFF 00000000 00000000 00000000 00000000 00006000 00000000
> <4>[60392.954206] I[1:    ksoftirqd/1:   19] 5ff8 : 00000000 00000000 00000001 00000000 014000C0 43A820B3 00010600 00000001
> <4>[60392.954220] I[1:    ksoftirqd/1:   19] 6018 : 00000000 00000000 0AAF9E10 FFFFFF80 00000000 00000000 00000071 00000000
> <4>[60392.954234] I[1:    ksoftirqd/1:   19] 6038 : 00000000 00002752 00000000 00000000 00000000 00000000 FFFFFFFF 7FFFFFFF
> <4>[60392.954249] I[1:    ksoftirqd/1:   19] 
> <4>[60392.954249] I[1:    ksoftirqd/1:   19] X24: 0xffffff800aaebb80:
> <4>[60392.954261] I[1:    ksoftirqd/1:   19] bb80 : 0AAEEB88 FFFFFF80 0AB08710 FFFFFF80 08F65024 FFFFFF80 08F65060 FFFFFF80
> <4>[60392.954275] I[1:    ksoftirqd/1:   19] bba0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954289] I[1:    ksoftirqd/1:   19] bbc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954303] I[1:    ksoftirqd/1:   19] bbe0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954318] I[1:    ksoftirqd/1:   19] bc00 : 00000000 00000000 00000000 00000000 0AAEBC10 FFFFFF80 0AAEBC10 FFFFFF80
> <4>[60392.954333] I[1:    ksoftirqd/1:   19] bc20 : 0AAEBC98 FFFFFF80 0AAEBC98 FFFFFF80 00000000 00000000 00000000 00000000
> <4>[60392.954347] I[1:    ksoftirqd/1:   19] bc40 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954361] I[1:    ksoftirqd/1:   19] bc60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954376] I[1:    ksoftirqd/1:   19] bc80 : 00000002 000004F2 00020002 00000000 000174E6 00000000 0AAEBC20 FFFFFF80
> <4>[60392.954391] I[1:    ksoftirqd/1:   19] bca0 : 0AAEBC20 FFFFFF80 00000000 00000000 00000000 00000000 0B61BC48 FFFFFF80
> <4>[60392.954405] I[1:    ksoftirqd/1:   19] bcc0 : 0B61BC48 FFFFFF80 0A550D80 FFFFFF80 00000000 00000000 9DB59DB5 00000000
> <4>[60392.954419] I[1:    ksoftirqd/1:   19] bce0 : 0C000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954433] I[1:    ksoftirqd/1:   19] bd00 : 09644DB0 FFFFFF80 F0000091 00000000 EF839800 FFFFFFC8 EF839900 FFFFFFC8
> <4>[60392.954447] I[1:    ksoftirqd/1:   19] bd20 : 0919AB44 FFFFFF80 0A580018 FFFFFF80 00000000 00000001 00000002 00000000
> <4>[60392.954461] I[1:    ksoftirqd/1:   19] bd40 : 00000000 00000000 0A580018 FFFFFF80 0AB08478 FFFFFF80 0AAEBD20 FFFFFF80
> <4>[60392.954475] I[1:    ksoftirqd/1:   19] bd60 : 00000000 00000000 00000000 00000000 00000000 00000000 EF896858 FFFFFFC8
> <4>[60392.954490] I[1:    ksoftirqd/1:   19] 
> <4>[60392.954490] I[1:    ksoftirqd/1:   19] X25: 0xffffffc0051e5d00:
> <4>[60392.954501] I[1:    ksoftirqd/1:   19] 5d00 : 00000000 00000000 00000000 00000000 E745D5E7 496DFBF1 00000000 00000000
> <4>[60392.954516] I[1:    ksoftirqd/1:   19] 5d20 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954530] I[1:    ksoftirqd/1:   19] 5d40 : 00000000 00000000 00000000 00000000 00000000 00000000 FFFF0000 0200A8C0
> <4>[60392.954544] I[1:    ksoftirqd/1:   19] 5d60 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954562] I[1:    ksoftirqd/1:   19] 5d80 : 00000000 00000000 0081FF80 00000000 00000000 00080000 00000000 00000000
> <4>[60392.954579] I[1:    ksoftirqd/1:   19] 5da0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954593] I[1:    ksoftirqd/1:   19] 5dc0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954607] I[1:    ksoftirqd/1:   19] 5de0 : 00000000 00000000 051E0BC0 FFFFFFC0 00000085 00000000 3C0ADD53 FCA9C68D
> <4>[60392.954622] I[1:    ksoftirqd/1:   19] 5e00 : 831FD9AC 0200A8C0 EEE47EBA 808CBB01 4004000A 00000000 00000000 00000000
> <4>[60392.954640] I[1:    ksoftirqd/1:   19] 5e20 : BC8B82B8 FFFFFFC8 0AAF9E10 FFFFFF80 0AAEBC80 FFFFFF80 00000000 00000000
> <4>[60392.954673] I[1:    ksoftirqd/1:   19] 5e40 : FFFF0000 831FD9AC 00000000 00000000 FFFF0000 0200A8C0 00017031 00000000
> <4>[60392.954687] I[1:    ksoftirqd/1:   19] 5e60 : 00004301 00000000 0000FD75 00000000 EE63F5D0 FFFFFFC8 FFFFFFFF 00000004
> <4>[60392.954702] I[1:    ksoftirqd/1:   19] 5e80 : 00000003 00000000 00790078 00000000 00000000 00000000 051E5E98 FFFFFFC0
> <4>[60392.954716] I[1:    ksoftirqd/1:   19] 5ea0 : 051E5E98 FFFFFFC0 00000000 00000001 051E5EB0 FFFFFFC0 051E5EB0 FFFFFFC0
> <4>[60392.954730] I[1:    ksoftirqd/1:   19] 5ec0 : 00000000 00000000 051E5EC8 FFFFFFC0 051E5EC8 FFFFFFC0 00000000 00000000
> <4>[60392.954745] I[1:    ksoftirqd/1:   19] 5ee0 : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954761] I[1:    ksoftirqd/1:   19] 
> <4>[60392.954761] I[1:    ksoftirqd/1:   19] X28: 0xffffffc82fe10b2c:
> <4>[60392.954772] I[1:    ksoftirqd/1:   19] 0b2c : 05F02FA5 F6F39767 535F5E0E 0CC4A1B3 04B50A41 DE8744AE 00000040 00000020
> <4>[60392.954787] I[1:    ksoftirqd/1:   19] 0b4c : 00000000 00000002 00000000 DB9D6F18 FFFFFFC8 00000000 00000000 00000000
> <4>[60392.954801] I[1:    ksoftirqd/1:   19] 0b6c : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954815] I[1:    ksoftirqd/1:   19] 0b8c : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954830] I[1:    ksoftirqd/1:   19] 0bac : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954844] I[1:    ksoftirqd/1:   19] 0bcc : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954858] I[1:    ksoftirqd/1:   19] 0bec : 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> <4>[60392.954872] I[1:    ksoftirqd/1:   19] 0c0c : 00080000 5000C045 000036D4 62240140 0200A8C0 0200A8C0 ABB50103 00000000
> <4>[60392.954886] I[1:    ksoftirqd/1:   19] 0c2c : 34000045 004023E3 99CA0640 0200A8C0 831FD9AC BB018C80 AE46FB9F 537870A5
> <4>[60392.954900] I[1:    ksoftirqd/1:   19] 0c4c : 48031180 00002D8D 0A080101 F2556142 DBE9DC24 736D692E 76726573 00656369
> <4>[60392.954914] I[1:    ksoftirqd/1:   19] 0c6c : 00390100 00000000 4E000000 554C4C55 00644C4C 00000000 00000000 00000000
> <4>[60392.954928] I[1:    ksoftirqd/1:   19] 0c8c : 00000000 00000000 00000000 00000000 00000000 00000000 00000001 00000000
> <4>[60392.954942] I[1:    ksoftirqd/1:   19] 0cac : 00000000 00000000 00000000 00000000 00000000 00000001 00000000 00000000
> <4>[60392.954956] I[1:    ksoftirqd/1:   19] 0ccc : 00000000 00000000 00000000 00000000 00000000 00100000 00000001 2FE10CF8
> <4>[60392.954970] I[1:    ksoftirqd/1:   19] 0cec : FFFFFFC8 00000000 00000000 001B4880 FFFFFFBF 00001000 00000000 00000000
> <4>[60392.954984] I[1:    ksoftirqd/1:   19] 0d0c : 00000002 00000001 00000001 00000000 00000000 00000001 00000001 00000000
> <4>[60392.954999] I[1:    ksoftirqd/1:   19] 
> <4>[60392.954999] I[1:    ksoftirqd/1:   19] X29: 0xffffff800b95b890:
> <4>[60392.955010] I[1:    ksoftirqd/1:   19] b890 : EF89B000 FFFFFFC8 EE595D00 B254C55C 2FE10C24 FFFFFFC8 36B80010 FFFFFFC0
> <4>[60392.955025] I[1:    ksoftirqd/1:   19] b8b0 : 0B2D8880 FFFFFF80 0AAEBC80 FFFFFF80 00000010 00000000 00000000 00000000
> <4>[60392.955039] I[1:    ksoftirqd/1:   19] b8d0 : 0B95B930 FFFFFF80 09015704 FFFFFF80 00000003 00000000 0906F3D0 FFFFFF80
> <4>[60392.955053] I[1:    ksoftirqd/1:   19] b8f0 : 00000001 00000000 EF89B000 FFFFFFC8 0AAEBC80 FFFFFF80 2FE10C24 FFFFFFC8
> <4>[60392.955067] I[1:    ksoftirqd/1:   19] b910 : 00000000 00000000 00000000 FFFFFFC8 00780078 FFFFFFC8 EE595D00 B254C55C
> <4>[60392.955081] I[1:    ksoftirqd/1:   19] b930 : 0B95B990 FFFFFF80 09034E40 FFFFFF80 0A5386B0 FFFFFF80 0A540FD0 FFFFFF80
> <4>[60392.955096] I[1:    ksoftirqd/1:   19] b950 : EF89B0A0 FFFFFFC8 EF89B000 FFFFFFC8 00000008 00000000 2FE10C24 FFFFFFC8
> <4>[60392.955110] I[1:    ksoftirqd/1:   19] b970 : 00000000 00000000 00000006 00000000 00000000 00000000 1EEBFB80 FFFFFFC8
> <4>[60392.955124] I[1:    ksoftirqd/1:   19] b990 : 0B95B9C0 FFFFFF80 0904A990 FFFFFF80 0AAEBC80 FFFFFF80 36B82128 FFFFFFC0
> <4>[60392.955138] I[1:    ksoftirqd/1:   19] b9b0 : 2FE10C2C FFFFFFC8 1EEBFB80 FFFFFFC8 0B95BA00 FFFFFF80 0904A6D0 FFFFFF80
> <4>[60392.955152] I[1:    ksoftirqd/1:   19] b9d0 : 1EEBFC18 FFFFFFC8 EE595D00 B254C55C 0A540578 FFFFFF80 12995900 FFFFFFC0
> <4>[60392.955167] I[1:    ksoftirqd/1:   19] b9f0 : 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8 0B95BA40 FFFFFF80 0904A3AC FFFFFF80
> <4>[60392.955181] I[1:    ksoftirqd/1:   19] ba10 : 00000008 00000000 1EEBFC18 FFFFFFC8 0A540578 FFFFFF80 0964F9E8 FFFFFF80
> <4>[60392.955195] I[1:    ksoftirqd/1:   19] ba30 : 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8 0B95BA70 FFFFFF80 0900B104 FFFFFF80
> <4>[60392.955209] I[1:    ksoftirqd/1:   19] ba50 : 2FE10C10 FFFFFFC8 00000001 00000000 0AAEBC80 FFFFFF80 1EEBFB80 FFFFFFC8
> <4>[60392.955223] I[1:    ksoftirqd/1:   19] ba70 : 0B95BAE0 FFFFFF80 0900AFD4 FFFFFF80 00000004 00000000 00000001 00000002
> <4>[60392.955238] I[1:    ksoftirqd/1:   19] 
> <0>[60392.955246] I[1:    ksoftirqd/1:   19] Process ksoftirqd/1 (pid: 19, stack limit = 0xffffff800b958000)
> <0>[60392.955253] I[1:    ksoftirqd/1:   19] Call trace:
> <4>[60392.955263] I[1:    ksoftirqd/1:   19] Exception stack(0xffffff800b95b700 to 0xffffff800b95b840)
> <4>[60392.955272] I[1:    ksoftirqd/1:   19] b700: 0000000000000040 000000000200a8c0 000000000000808c 00000000831fd9ac
> <4>[60392.955281] I[1:    ksoftirqd/1:   19] b720: 000000000000bb01 000000000000808c 0000000000000010 0000000000000000
> <4>[60392.955290] I[1:    ksoftirqd/1:   19] b740: 0000000000000200 0000000000000020 0000000000007530 ffffffc0051e5f58
> <4>[60392.955298] I[1:    ksoftirqd/1:   19] b760: 0000000000000000 0000000000000002 0000000000000000 ffffffffffffffff
> <4>[60392.955306] I[1:    ksoftirqd/1:   19] b780: 0000000000000000 0000000000000400 0000000000000001 ffffffc0051e5e00
> <4>[60392.955315] I[1:    ksoftirqd/1:   19] b7a0: ffffffc0051e5e88 0000000000000071 0000000000000000 ffffffc0051e5f58
> <4>[60392.955323] I[1:    ksoftirqd/1:   19] b7c0: ffffff800aaebc80 ffffffc0051e5e00 0000000000000001 0000000000000003
> <4>[60392.955332] I[1:    ksoftirqd/1:   19] b7e0: ffffffc82fe10c2c ffffff800b95b990 ffffff80090350cc ffffff800b95b940
> <4>[60392.955341] I[1:    ksoftirqd/1:   19] b800: ffffff80090351d0 0000000060c00145 0000000000000001 b254c55cee595d00
> <4>[60392.955349] I[1:    ksoftirqd/1:   19] b820: ffffffffffffffff ffffff800b2d7db8 ffffff800b95b990 ffffff80090351d0
> <0>[60392.955362] I[1:    ksoftirqd/1:   19] [<ffffff80090351d0>] tcp_v4_err+0x4b0/0x4bc
> <0>[60392.955372] I[1:    ksoftirqd/1:   19] [<ffffff800904a990>] icmp_socket_deliver+0x70/0xc0
> <0>[60392.955380] I[1:    ksoftirqd/1:   19] [<ffffff800904a6d0>] icmp_unreach+0x1b4/0x1f0
> <0>[60392.955389] I[1:    ksoftirqd/1:   19] [<ffffff800904a3ac>] icmp_rcv+0x310/0x348
> <0>[60392.955400] I[1:    ksoftirqd/1:   19] [<ffffff800900b104>] ip_local_deliver_finish+0xf8/0x1f0
> <0>[60392.955409] I[1:    ksoftirqd/1:   19] [<ffffff800900afd4>] ip_local_deliver+0xc8/0x100
> <0>[60392.955418] I[1:    ksoftirqd/1:   19] [<ffffff800900b790>] ip_rcv_finish+0x218/0x2e4
> <0>[60392.955426] I[1:    ksoftirqd/1:   19] [<ffffff800900b4c0>] ip_rcv+0x2c4/0x37c
> <0>[60392.955437] I[1:    ksoftirqd/1:   19] [<ffffff8008f84f98>] __netif_receive_skb_core+0x8f0/0xa68
> <0>[60392.955446] I[1:    ksoftirqd/1:   19] [<ffffff8008f85960>] process_backlog+0x17c/0x290
> <0>[60392.955455] I[1:    ksoftirqd/1:   19] [<ffffff8008f85ef0>] net_rx_action+0x224/0x464
> <0>[60392.955465] I[1:    ksoftirqd/1:   19] [<ffffff80081b9564>] __do_softirq+0x200/0x39c
> <0>[60392.955475] I[1:    ksoftirqd/1:   19] [<ffffff80081f1f04>] run_ksoftirqd+0x34/0x74
> <0>[60392.955485] I[1:    ksoftirqd/1:   19] [<ffffff800821ad7c>] smpboot_thread_fn+0x1d0/0x2f8
> <0>[60392.955495] I[1:    ksoftirqd/1:   19] [<ffffff8008215c48>] kthread+0x11c/0x130
> <0>[60392.955505] I[1:    ksoftirqd/1:   19] [<ffffff80081bdaa0>] ret_from_fork+0x10/0x1c
> <0>[60392.955515] I[1:    ksoftirqd/1:   19] Code: f9402d08 94063788 aa1603e8 17ffff80 (d4210000) 
> <4>[60392.955524] I[1:    ksoftirqd/1:   19] ---[ end trace c1d9b9267f6ef3c2 ]---
> 

^ permalink raw reply


This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox