From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mailgw.kylinos.cn (mailgw.kylinos.cn [124.126.103.232]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id EADBA2E172B for ; Fri, 22 Aug 2025 05:36:55 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=124.126.103.232 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755841020; cv=none; b=eB+tCi8C60P04xj4HiVOH67FpIARRuUVOC2Y3IcYFVfO3uXeHLceNXEbKQ1lYvD9z9f0GF6/hTJE3sp8DMlHPnJ3VSzkYf1AKdl57XAkF7yWzXiEYZ0d5fRu3nOvYPdFixl+cqVcVXXf3rdgvc3X7Z306/EBSxgOj/1v0XGeB/k= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1755841020; c=relaxed/simple; bh=RKG/r/kw6RTssDky/QUrir+3GbPvhMg6vDK9PyzTKRg=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=Wbi7PTLsiCjv+daZR5TrHiEhZCnQsnYq0YJh7mvX7POi19ZRlB4WdkDxuMfYpSmo0F+ZxwKJT/VTXGYFRVWLv8w9Oe1L9LFT1HGPzbvqOrAA3ILFdtgyURoqabYwBDKJguTAGuGFi9vJhB81ucoA26uSHocfAp9HHwCY1w4V/fQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn; spf=pass smtp.mailfrom=kylinos.cn; arc=none smtp.client-ip=124.126.103.232 Authentication-Results: smtp.subspace.kernel.org; dmarc=none (p=none dis=none) header.from=kylinos.cn Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=kylinos.cn X-UUID: ff0e7ca47f1911f0b29709d653e92f7d-20250822 X-CID-P-RULE: Release_Ham X-CID-O-INFO: VERSION:1.1.45,REQID:9ccedf0b-1170-4dc1-96d3-28d000810191,IP:10, URL:25,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACT ION:release,TS:20 X-CID-INFO: VERSION:1.1.45,REQID:9ccedf0b-1170-4dc1-96d3-28d000810191,IP:10,UR L:25,TC:0,Content:0,EDM:0,RT:0,SF:-15,FILE:0,BULK:0,RULE:Release_Ham,ACTIO N:release,TS:20 X-CID-META: VersionHash:6493067,CLOUDID:82c5a7bdba99bf8e5ea47f67cac0396c,BulkI D:2508091148269AKA4902,BulkQuantity:15,Recheck:0,SF:17|19|24|44|64|66|78|8 0|81|82|83|102|841|850,TC:nil,Content:0|50,EDM:-3,IP:-2,URL:11|99|83|1,Fil e:nil,RT:nil,Bulk:40,QS:nil,BEC:nil,COL:0,OSI:0,OSA:0,AV:0,LES:1,SPR:NO,DK R:0,DKP:0,BRR:0,BRE:0,ARC:0 X-CID-BVR: 0,NGT X-CID-BAS: 0,NGT,0,_ X-CID-FACTOR: TF_CID_SPAM_ULN,TF_CID_SPAM_SNR,TF_CID_SPAM_FAS,TF_CID_SPAM_FSD, TF_CID_SPAM_FSI,TF_CID_SPAM_OBB X-UUID: ff0e7ca47f1911f0b29709d653e92f7d-20250822 X-User: duanchenghao@kylinos.cn Received: from localhost [(223.70.159.239)] by mailgw.kylinos.cn (envelope-from ) (Generic MTA with TLSv1.3 TLS_AES_256_GCM_SHA384 256/256) with ESMTP id 2088312430; Fri, 22 Aug 2025 13:36:47 +0800 Date: Fri, 22 Aug 2025 13:36:43 +0800 From: Chenghao Duan To: Vincent Li Cc: Huacai Chen , loongarch@lists.linux.dev, Hengqi Chen , Tiezhu Yang Subject: Re: kernel lockup on bpf selftests module_attach Message-ID: <20250822053643.GA685331@chenghao-pc> References: <20250812083441.GA1119757@chenghao-pc> <20250814120015.GA1626128@chenghao-pc> <20250822031102.GA331509@chenghao-pc> Precedence: bulk X-Mailing-List: loongarch@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: On Thu, Aug 21, 2025 at 10:22:46PM -0700, Vincent Li wrote: > On Thu, Aug 21, 2025 at 10:10 PM Vincent Li wrote: > > > > On Thu, Aug 21, 2025 at 8:11 PM Chenghao Duan wrote: > > > > > > On Thu, Aug 21, 2025 at 08:04:07AM -0700, Vincent Li wrote: > > > > On Thu, Aug 14, 2025 at 5:00 AM Chenghao Duan wrote: > > > > > > > > > > On Tue, Aug 12, 2025 at 06:42:08AM -0700, Vincent Li wrote: > > > > > > On Tue, Aug 12, 2025 at 1:34 AM Chenghao Duan wrote: > > > > > > > > > > > > > > On Sun, Aug 10, 2025 at 10:39:24AM -0700, Vincent Li wrote: > > > > > > > > Hi Chenghao, > > > > > > > > > > > > > > > > On Sat, Aug 9, 2025 at 12:11 PM Vincent Li wrote: > > > > > > > > > > > > > > > > > > On Fri, Aug 8, 2025 at 11:03 PM Huacai Chen wrote: > > > > > > > > > > > > > > > > > > > > Hi, Chenghao, > > > > > > > > > > > > > > > > > > > > Please take a look. > > > > > > > > > > > > > > > > > > > > Huacai > > > > > > > > > > > > > > > > > > > I reverted loongson-next branch tailcall count fix patches, struct > > > > > > > > > ops trampoline patch, keep the rest of trampoline patches, > > > > > > > > > module_attach test experienced the same issue, so definitely > > > > > > > > > trampoline patches issue. > > > > > > > > > > > > > > > > > > > > > > > > > I attempted to isolate which test in module_attach triggers the > > > > > > > > "Unable to handle kernel paging request..." error, it appears to be > > > > > > > > this one in "prog_tests/module_attach.c" > > > > > > > > > > > > > > > > ASSERT_OK(trigger_module_test_read(READ_SZ), "trigger_read"); > > > > > > > > > > > > > > > > you can try to comment out other tests in "prog_tests/module_attach.c" > > > > > > > > and perform the test, it might help isolate the issue. > > > > > > > > > > > > > > > > > > > > > > Hi Vincent, > > > > > > > > > > > > > > The results I tested are different from yours. Could there be other > > > > > > > differences between us? I am using the latest code of the loongarch-next > > > > > > > branch. > > > > > > > > > > > > > > [root@localhost bpf]# ./test_progs -v -t module_attach > > > > > > > bpf_testmod.ko is already unloaded. > > > > > > > Loading bpf_testmod.ko... > > > > > > > Successfully loaded bpf_testmod.ko. > > > > > > > test_module_attach:PASS:skel_open 0 nsec > > > > > > > test_module_attach:PASS:set_attach_target 0 nsec > > > > > > > test_module_attach:PASS:set_attach_target_explicit 0 nsec > > > > > > > WATCHDOG: test case module_attach executes for 10 seconds... > > > > > > > libbpf: prog 'handle_fmod_ret': BPF program load failed: -EINVAL > > > > > > > libbpf: prog 'handle_fmod_ret': -- BEGIN PROG LOAD LOG -- > > > > > > > bpf_testmod_test_read() is not modifiable > > > > > > > processed 0 insns (limit 1000000) max_states_per_insn 0 total_states 0 peak_states 0 mark_read 0 > > > > > > > -- END PROG LOAD LOG -- > > > > > > > libbpf: prog 'handle_fmod_ret': failed to load: -EINVAL > > > > > > > libbpf: failed to load object 'test_module_attach' > > > > > > > libbpf: failed to load BPF skeleton 'test_module_attach': -EINVAL > > > > > > > test_module_attach:FAIL:skel_load failed to load skeleton > > > > > > > #205 module_attach:FAIL > > > > > > > Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED > > > > > > > Successfully unloaded bpf_testmod.ko. > > > > > > > > > > > > > > > > > > > I build and run the most recent loongarch-next kernel too, can you try > > > > > > my config https://www.bpfire.net/download/loongfire/config.txt? I am > > > > > > on fedora, here are the steps I build, run the kernel, and run the > > > > > > test > > > > > > > > > > > > 1, check branch > > > > > > [root@fedora linux-loongson]# git branch > > > > > > * loongarch-next > > > > > > master > > > > > > no-tailcall > > > > > > no-trampoline > > > > > > > > > > > > 2, build kernel and reboot > > > > > > cp config.txt .config; make clean; make -j6; make modules_install; > > > > > > make install; grub2-mkconfig -o /boot/grub2/grub.cfg; reboot > > > > > > > > > > > > 3, after reboot and login, build bpf selftests, run module_attach > > > > > > test, dmesg to check kernel log > > > > > > cd tools/testing/selftests/bpf; make -j6; ./test_progs -t module_attach > > > > > > > > > > > > > > > > Hi Vincent, > > > > > > > > > > I tried to refer to the config you provided, but the test results I > > > > > obtained are as follows. I also specifically tested "modify" to verify > > > > > the effectiveness of the patch, and the test of module_attach returns -EOPNOTSUPP. > > > > > > > > > > [root@localhost bpf]# ./test_progs -v -t modify_return > > > > > bpf_testmod.ko is already unloaded. > > > > > Loading bpf_testmod.ko... > > > > > Successfully loaded bpf_testmod.ko. > > > > > run_test:PASS:skel_load 0 nsec > > > > > run_test:PASS:modify_return__attach failed 0 nsec > > > > > run_test:PASS:test_run 0 nsec > > > > > run_test:PASS:test_run ret 0 nsec > > > > > run_test:PASS:modify_return side_effect 0 nsec > > > > > run_test:PASS:modify_return fentry_result 0 nsec > > > > > run_test:PASS:modify_return fexit_result 0 nsec > > > > > run_test:PASS:modify_return fmod_ret_result 0 nsec > > > > > run_test:PASS:modify_return fentry_result2 0 nsec > > > > > run_test:PASS:modify_return fexit_result2 0 nsec > > > > > run_test:PASS:modify_return fmod_ret_result2 0 nsec > > > > > run_test:PASS:skel_load 0 nsec > > > > > run_test:PASS:modify_return__attach failed 0 nsec > > > > > run_test:PASS:test_run 0 nsec > > > > > run_test:PASS:test_run ret 0 nsec > > > > > run_test:PASS:modify_return side_effect 0 nsec > > > > > run_test:PASS:modify_return fentry_result 0 nsec > > > > > run_test:PASS:modify_return fexit_result 0 nsec > > > > > run_test:PASS:modify_return fmod_ret_result 0 nsec > > > > > run_test:PASS:modify_return fentry_result2 0 nsec > > > > > run_test:PASS:modify_return fexit_result2 0 nsec > > > > > run_test:PASS:modify_return fmod_ret_result2 0 nsec > > > > > #200 modify_return:OK > > > > > Summary: 1/0 PASSED, 0 SKIPPED, 0 FAILED > > > > > Successfully unloaded bpf_testmod.ko. > > > > > [root@localhost bpf]# ./test_progs -v -t module_attach > > > > > bpf_testmod.ko is already unloaded. > > > > > Loading bpf_testmod.ko... > > > > > Successfully loaded bpf_testmod.ko. > > > > > test_module_attach:PASS:skel_open 0 nsec > > > > > test_module_attach:PASS:set_attach_target 0 nsec > > > > > test_module_attach:PASS:set_attach_target_explicit 0 nsec > > > > > test_module_attach:PASS:skel_load 0 nsec > > > > > libbpf: prog 'kprobe_multi': failed to attach: -EOPNOTSUPP > > > > > libbpf: prog 'kprobe_multi': failed to auto-attach: -EOPNOTSUPP > > > > > > > > the -EOPNOTSUPP comes from libbpf, but I am not sure if it is error in > > > > kernel leads to libbpf error or libbpf itself, you can do strace -f > > > > -s1024 -o /tmp/module_attatch.txt ./test_progs -v -t module_attach. > > > > The strace should have bpf syscall and I think it can tell you if the > > > > -EOPNOTSUPP is the result of kernel error or libbpf, you can share the > > > > strace if you want. > > > > > > > 2037 read(16, "", 8192) = 0 > > > 2037 close(16) = 0 > > > 2037 bpf(BPF_LINK_CREATE, {link_create={prog_fd=61, target_fd=0, attach_type=BPF_TRACE_KPROBE_MULTI, flags=0, kprobe_multi={flags=0, cnt=1, syms=NULL, addrs=[0xffff8000035717d0], cookies=NULL}}}, 64) = -1 EOPNOTSUPP (不支持的操作) > > > > so bpf syscall cmd BPF_LINK_CREATE returns '-1 EOPNOTSUPP' exactly? I > > could not tell because I thought the return value is '-1' > > > > > 2037 write(1, "libbpf: prog 'kprobe_multi': failed to attach: -EOPNOTSUPP\n", 59) = 59 > > > 2037 write(1, "libbpf: prog 'kprobe_multi': failed to auto-attach: -EOPNOTSUPP\n", 64) = 64 > > > 2037 write(1, "test_module_attach:FAIL:skel_attach skeleton attach failed: -95\n", 64) = 64 > > > > > > not support attach_type BPF_TRACE_KPROBE_MULTI > > > > > > > Could you share your kernel config (.config used for kernel compiling > > or running kernel /boot/config-*) ? I wonder if you have the FPROBE > > really configured, since include/linux/fprobe.h has: > > > > #ifdef CONFIG_FPROBE > > int register_fprobe(struct fprobe *fp, const char *filter, const char > > *notfilter); > > int register_fprobe_ips(struct fprobe *fp, unsigned long *addrs, int num); > > int register_fprobe_syms(struct fprobe *fp, const char **syms, int num); > > int unregister_fprobe(struct fprobe *fp); > > bool fprobe_is_registered(struct fprobe *fp); > > int fprobe_count_ips_from_filter(const char *filter, const char *notfilter); > > #else > > static inline int register_fprobe(struct fprobe *fp, const char > > *filter, const char *notfilter) > > { > > return -EOPNOTSUPP; > > } > > static inline int register_fprobe_ips(struct fprobe *fp, unsigned long > > *addrs, int num) > > { > > return -EOPNOTSUPP; > > } > > static inline int register_fprobe_syms(struct fprobe *fp, const char > > **syms, int num) > > { > > return -EOPNOTSUPP; > > } > > static inline int unregister_fprobe(struct fprobe *fp) > > { > > return -EOPNOTSUPP; > > } > > static inline bool fprobe_is_registered(struct fprobe *fp) > > { > > return false; > > } > > static inline int fprobe_count_ips_from_filter(const char *filter, > > const char *notfilter) > > { > > return -EOPNOTSUPP; > > } > > #endif > > > > and check CONFIG_BPF_EVENTS since linux/trace_events.h has: > > #ifdef CONFIG_BPF_EVENTS > unsigned int trace_call_bpf(struct trace_event_call *call, void *ctx); > int perf_event_attach_bpf_prog(struct perf_event *event, struct > bpf_prog *prog, u64 bpf_cookie); > void perf_event_detach_bpf_prog(struct perf_event *event); > int perf_event_query_prog_array(struct perf_event *event, void __user *info); > > struct bpf_raw_tp_link; > int bpf_probe_register(struct bpf_raw_event_map *btp, struct > bpf_raw_tp_link *link); > int bpf_probe_unregister(struct bpf_raw_event_map *btp, struct > bpf_raw_tp_link *link); > > struct bpf_raw_event_map *bpf_get_raw_tracepoint(const char *name); > void bpf_put_raw_tracepoint(struct bpf_raw_event_map *btp); > int bpf_get_perf_event_info(const struct perf_event *event, u32 *prog_id, > u32 *fd_type, const char **buf, > u64 *probe_offset, u64 *probe_addr, > unsigned long *missed); > int bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct > bpf_prog *prog); > int bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct > bpf_prog *prog); > #else > static inline unsigned int trace_call_bpf(struct trace_event_call > *call, void *ctx) > { > return 1; > } > > static inline int > perf_event_attach_bpf_prog(struct perf_event *event, struct bpf_prog > *prog, u64 bpf_cookie) > { > return -EOPNOTSUPP; > } > > static inline void perf_event_detach_bpf_prog(struct perf_event *event) { } > > static inline int > perf_event_query_prog_array(struct perf_event *event, void __user *info) > { > return -EOPNOTSUPP; > } > struct bpf_raw_tp_link; > static inline int bpf_probe_register(struct bpf_raw_event_map *btp, > struct bpf_raw_tp_link *link) > { > return -EOPNOTSUPP; > } > static inline int bpf_probe_unregister(struct bpf_raw_event_map *btp, > struct bpf_raw_tp_link *link) > { > return -EOPNOTSUPP; > } > static inline struct bpf_raw_event_map *bpf_get_raw_tracepoint(const char *name) > { > return NULL; > } > static inline void bpf_put_raw_tracepoint(struct bpf_raw_event_map *btp) > { > } > static inline int bpf_get_perf_event_info(const struct perf_event *event, > u32 *prog_id, u32 *fd_type, > const char **buf, u64 *probe_offset, > u64 *probe_addr, unsigned > long *missed) > { > return -EOPNOTSUPP; > } > static inline int > bpf_kprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) > { > return -EOPNOTSUPP; > } > static inline int > bpf_uprobe_multi_link_attach(const union bpf_attr *attr, struct bpf_prog *prog) > { > return -EOPNOTSUPP; > } > #endif I checked the config and used the most straightforward method: intentionally adding errors in the macro's if-else to distinguish. > > > > Chenghao > > > > > > > > > > > > > > > test_module_attach:FAIL:skel_attach skeleton attach failed: -95 > > > > > #201 module_attach:FAIL > > > > > Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED > > > > > Successfully unloaded bpf_testmod.ko. > > > > > > > > > > > > > > > Chenghao > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Chenghao > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Aug 9, 2025 at 1:03 PM Vincent Li wrote: > > > > > > > > > > > > > > > > > > > > > > On Fri, Aug 8, 2025 at 8:48 PM Vincent Li wrote: > > > > > > > > > > > > > > > > > > > > > > > > On Fri, Aug 8, 2025 at 8:03 PM Huacai Chen wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > Hi, Vincent, > > > > > > > > > > > > > > > > > > > > > > > > > > On Sat, Aug 9, 2025 at 12:53 AM Vincent Li wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Folks, > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hengqi mentioned offline that the loongarch kernel locked up when > > > > > > > > > > > > > > running full bpf selftests, so I went ahead and ran make run_tests to > > > > > > > > > > > > > > perform full bpf selftest, I observed lockup too. It appears the > > > > > > > > > > > > > > lockup happens when running module_attach test which includes testing > > > > > > > > > > > > > > on fentry so this could be related to the trampoline patch series. for > > > > > > > > > > > > > > example, if I just run ./test_progs -t module_attach, the kernel > > > > > > > > > > > > > > lockup immediately. > > > > > > > > > > > > > Is this a regression caused by the latest trampoline patches? Or in > > > > > > > > > > > > > another word, Does vanilla 6.16 has this problem? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I suspect this is caused by the latest trampoline patches because the > > > > > > > > > > > > module_attach is to test the fentry feature for kernel module > > > > > > > > > > > > functions, I believe Changhao and I only tested the fentry feature for > > > > > > > > > > > > non-module kernel functions. I can try kernel without the trampoline > > > > > > > > > > > > patches and will let you know the result. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I reverted trampoline patches from loongarch-next branch and run > > > > > > > > > > > ./test_progs -t module_attach simply just errors out with the fentry > > > > > > > > > > > feature not supported > > > > > > > > > > > > > > > > > > > > > > [root@fedora bpf]# ./test_progs -t module_attach > > > > > > > > > > > test_module_attach:PASS:skel_open 0 nsec > > > > > > > > > > > test_module_attach:PASS:set_attach_target 0 nsec > > > > > > > > > > > test_module_attach:PASS:set_attach_target_explicit 0 nsec > > > > > > > > > > > test_module_attach:PASS:skel_load 0 nsec > > > > > > > > > > > libbpf: prog 'handle_fentry': failed to attach: -ENOTSUPP > > > > > > > > > > > libbpf: prog 'handle_fentry': failed to auto-attach: -ENOTSUPP > > > > > > > > > > > test_module_attach:FAIL:skel_attach skeleton attach failed: -524 > > > > > > > > > > > #205 module_attach:FAIL > > > > > > > > > > > > > > > > > > > > > > All error logs: > > > > > > > > > > > test_module_attach:PASS:skel_open 0 nsec > > > > > > > > > > > test_module_attach:PASS:set_attach_target 0 nsec > > > > > > > > > > > test_module_attach:PASS:set_attach_target_explicit 0 nsec > > > > > > > > > > > test_module_attach:PASS:skel_load 0 nsec > > > > > > > > > > > libbpf: prog 'handle_fentry': failed to attach: -ENOTSUPP > > > > > > > > > > > libbpf: prog 'handle_fentry': failed to auto-attach: -ENOTSUPP > > > > > > > > > > > test_module_attach:FAIL:skel_attach skeleton attach failed: -524 > > > > > > > > > > > #205 module_attach:FAIL > > > > > > > > > > > Summary: 0/0 PASSED, 0 SKIPPED, 1 FAILED > > > > > > > > > > > > > > > > > > > > > > I also tested loongarch-next branch with the trampoline patch series > > > > > > > > > > > with no lockup kernel config so I can run dmesg to check kernel error > > > > > > > > > > > log, ./test_progs -t module_attach result in below kernel log: > > > > > > > > > > > > > > > > > > > > > > [ 417.429954] bpf_testmod: loading out-of-tree module taints kernel. > > > > > > > > > > > [ 419.728620] CPU 70475748 Unable to handle kernel paging request at > > > > > > > > > > > virtual address 0000000800000024, era == 90000000041d5854, ra == > > > > > > > > > > > 90000000041d5848 > > > > > > > > > > > [ 419.728629] Oops[#1]: > > > > > > > > > > > [ 419.728632] CPU 70475748 Unable to handle kernel paging request at > > > > > > > > > > > virtual address 0000000000000018, era == 9000000005750268, ra == > > > > > > > > > > > 9000000004163938 > > > > > > > > > > > [ 441.305370] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks: > > > > > > > > > > > [ 441.305380] rcu: 5-...0: (29 ticks this GP) > > > > > > > > > > > idle=eb74/1/0x4000000000000000 softirq=72377/72379 fqs=2599 > > > > > > > > > > > [ 441.305386] rcu: (detected by 4, t=5252 jiffies, g=60333, q=186 ncpus=8) > > > > > > > > > > > [ 441.305390] Sending NMI from CPU 4 to CPUs 5: > > > > > > > > > > > [ 451.305494] rcu: rcu_preempt kthread starved for 2499 jiffies! > > > > > > > > > > > g60333 f0x0 RCU_GP_DOING_FQS(6) ->state=0x0 ->cpu=1 > > > > > > > > > > > [ 451.305500] rcu: Unless rcu_preempt kthread gets sufficient CPU > > > > > > > > > > > time, OOM is now expected behavior. > > > > > > > > > > > [ 451.305502] rcu: RCU grace-period kthread stack dump: > > > > > > > > > > > [ 451.305504] task:rcu_preempt state:R stack:0 pid:15 > > > > > > > > > > > tgid:15 ppid:2 task_flags:0x208040 flags:0x00000800 > > > > > > > > > > > [ 451.305510] Stack : 9000000100467e80 0000000000000402 > > > > > > > > > > > 0000000000000010 90000001003b0680 > > > > > > > > > > > [ 451.305519] 90000000058e0000 0000000000000000 > > > > > > > > > > > 0000000000000040 9000000006c2dfd0 > > > > > > > > > > > [ 451.305526] 900000000578c9b0 0000000000000001 > > > > > > > > > > > 9000000006b21000 0000000000000005 > > > > > > > > > > > [ 451.305533] 00000001000093a8 00000001000093a8 > > > > > > > > > > > 0000000000000000 0000000000000004 > > > > > > > > > > > [ 451.305540] 90000000058f04e0 0000000000000000 > > > > > > > > > > > 0000000000000002 b793724be1dfb2b8 > > > > > > > > > > > [ 451.305547] 00000001000093a9 b793724be1dfb2b8 > > > > > > > > > > > 000000000000003f 9000000006c2dfd0 > > > > > > > > > > > [ 451.305554] 9000000006c30c18 0000000000000005 > > > > > > > > > > > 9000000006b0e000 9000000006b21000 > > > > > > > > > > > [ 451.305560] 9000000100453c98 90000001003aff80 > > > > > > > > > > > 9000000006c31140 900000000578c9b0 > > > > > > > > > > > [ 451.305567] 00000001000093a8 9000000005794d3c > > > > > > > > > > > 00000000000000b4 0000000000000000 > > > > > > > > > > > [ 451.305574] 90000000024021b8 00000001000093a8 > > > > > > > > > > > 9000000004284f20 000000000a400001 > > > > > > > > > > > [ 451.305581] ... > > > > > > > > > > > [ 451.305584] Call Trace: > > > > > > > > > > > [ 451.305586] [<900000000578b868>] __schedule+0x410/0x1520 > > > > > > > > > > > [ 451.305595] [<900000000578c9ac>] schedule+0x34/0x190 > > > > > > > > > > > [ 451.305599] [<9000000005794d38>] schedule_timeout+0x98/0x140 > > > > > > > > > > > [ 451.305604] [<9000000004258f40>] rcu_gp_fqs_loop+0x5f8/0x868 > > > > > > > > > > > [ 451.305609] [<900000000425d358>] rcu_gp_kthread+0x260/0x2e0 > > > > > > > > > > > [ 451.305614] [<90000000041be704>] kthread+0x144/0x238 > > > > > > > > > > > [ 451.305619] [<9000000005787b60>] ret_from_kernel_thread+0x28/0xc8 > > > > > > > > > > > [ 451.305624] [<90000000041620e4>] ret_from_kernel_thread_asm+0xc/0x88 > > > > > > > > > > > > > > > > > > > > > > [ 451.305630] rcu: Stack dump where RCU GP kthread last ran: > > > > > > > > > > > [ 451.305633] Sending NMI from CPU 4 to CPUs 1: > > > > > > > > > > > [ 451.305636] NMI backtrace for cpu 1 skipped: idling at idle_exit+0x0/0x4 > > > > > > > > > > > [ 451.306655] rcu: INFO: rcu_preempt detected expedited stalls on > > > > > > > > > > > CPUs/tasks: { 5-...D } 7298 jiffies s: 853 root: 0x20/. > > > > > > > > > > > [ 451.306665] rcu: blocking rcu_node structures (internal RCU debug): > > > > > > > > > > > [ 451.306669] Sending NMI from CPU 6 to CPUs 5: > > > > > > > > > > > [ 451.306672] Unable to send backtrace IPI to CPU5 - perhaps it hung? > > > > > > > > > > > > > > > > > > > > > > So related to trampoline patches for sure unless I am missing something. > > > > > > > > > > > > > > > > > > > > > > > > Huacai > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > A side note, if I put the module_attach test in > > > > > > > > > > > > > > tools/testing/selftests/bpf/DENYLIST to skip the module_attach test, > > > > > > > > > > > > > > the module_attach test is not skipped. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > > > Vincent