From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Yuriy N. Shkandybin" Date: Mon, 05 Mar 2007 20:17:29 +0000 Subject: PROBLEM: 2.6.20 pptp server lookup Message-Id: <001b01c75f63$5f79b040$0701010a@note> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: linux-ppp@vger.kernel.org Hello. Already several kernel releases i've expirienced different lockups of vpn (pptp) server. There is aroud 200 ppp connections at moment. With kernel debug i was able to retrive next information: First: Showing all locks held in the system: 1 lock held by agetty/4486: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 1 lock held by agetty/4487: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 1 lock held by agetty/4488: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 2 locks held by pptpctrl/4500: #0: (&tty->atomic_write_lock){--..}, at: [] tty_write+0x83/0x1d0 #1: (&ap->recv_lock){....}, at: [] ppp_asynctty_receive+0x2e/0x710 ======================BUG: spinlock lockup on CPU#1, pppd/4504, df5048c4 [] _raw_spin_lock+0x100/0x134 [] ppp_async_ioctl+0xa7/0x1d0 [] ppp_ioctl+0xa5/0xbff [] down_read+0x29/0x3a [] ppp_async_ioctl+0x0/0x1d0 [] ppp_ioctl+0xce/0xbff [] _spin_unlock+0x14/0x1c [] do_wp_page+0x256/0x4ba [] __handle_mm_fault+0x74e/0xa22 [] do_ioctl+0x64/0x6d [] vfs_ioctl+0x50/0x273 [] sys_ioctl+0x34/0x50 [] sysenter_past_esp+0x5f/0x99 ===========BUG: soft lockup detected on CPU#0! [] softlockup_tick+0x8d/0xbc [] update_process_times+0x28/0x5e [] smp_apic_timer_interrupt+0x80/0x9c [] apic_timer_interrupt+0x33/0x38 [] delay_tsc+0x9/0x13 [] __delay+0x6/0x7 [] _raw_spin_lock+0xa9/0x134 [] tty_write+0x83/0x1d0 [] tty_ldisc_try+0x2f/0x33 [] lock_kernel+0x19/0x24 [] tty_write+0x10b/0x1d0 [] write_chan+0x0/0x320 [] vfs_write+0x87/0xf0 [] tty_write+0x0/0x1d0 [] sys_write+0x41/0x6a [] sysenter_past_esp+0x5f/0x99 =========== Second) <0>BUG: spinlock lockup on CPU#0, pppd/5209, de3e2884 [] _raw_spin_lock+0x100/0x134 BUG: spinlock lockup on CPU#1, ip-down/7524, c0353300 [] _raw_spin_lock+0x100/0x134 [] lock_kernel+0x19/0x24 [] chrdev_open+0x8a/0x16e [] chrdev_open+0x0/0x16e [] __dentry_open+0xaf/0x1a0 [] nameidata_to_filp+0x31/0x3a [] do_filp_open+0x39/0x40 [] _spin_unlock+0x14/0x1c [] get_unused_fd+0xaa/0xbb [] do_sys_open+0x3a/0x6d [] sys_open+0x1c/0x20 [] sysenter_past_esp+0x5f/0x99 =========== [] ppp_async_ioctl+0xa7/0x1d0 [] ppp_ioctl+0xa5/0xbff [] down_read+0x29/0x3a [] ppp_async_ioctl+0x0/0x1d0 [] ppp_ioctl+0xce/0xbff [] _spin_unlock+0x14/0x1c [] do_wp_page+0x256/0x4ba [] __handle_mm_fault+0x74e/0xa22 [] do_ioctl+0x64/0x6d [] vfs_ioctl+0x50/0x273 [] sys_ioctl+0x34/0x50 [] sysenter_past_esp+0x5f/0x99 =========== Third) BUG: soft lockup detected on CPU#0! [] softlockup_tick+0x8d/0xbc [] update_process_times+0x28/0x5e [] smp_apic_timer_interrupt+0x80/0x9c [] apic_timer_interrupt+0x33/0x38 [] delay_tsc+0x9/0x13 [] __delay+0x6/0x7 [] _raw_spin_lock+0xa9/0x134 [] tty_ldisc_try+0x2f/0x33 [] lock_kernel+0x19/0x24 [] tty_read+0x5a/0xbe [] vfs_read+0x85/0xee [] tty_read+0x0/0xbe [] sys_read+0x41/0x6a [] sysenter_past_esp+0x5f/0x99 ===========BUG: soft lockup detected on CPU#0! [] softlockup_tick+0x8d/0xbc [] update_process_times+0x28/0x5e [] smp_apic_timer_interrupt+0x80/0x9c [] apic_timer_interrupt+0x33/0x38 [] prio_tree_insert+0xe8/0x23b [] _raw_spin_lock+0xaf/0x134 [] tty_ldisc_try+0x2f/0x33 [] lock_kernel+0x19/0x24 [] tty_read+0x5a/0xbe [] vfs_read+0x85/0xee [] tty_read+0x0/0xbe [] sys_read+0x41/0x6a [] sysenter_past_esp+0x5f/0x99 Next via SysRq: Showing all locks held in the system: 1 lock held by agetty/5057: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 1 lock held by agetty/5058: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 1 lock held by agetty/5059: #0: (&tty->atomic_read_lock){--..}, at: [] read_chan+0x41a/0x60b 2 locks held by pptpctrl/5071: #0: (&tty->atomic_write_lock){--..}, at: [] tty_write+0x83/0x1d0 #1: (&ap->recv_lock){....}, at: [] ppp_asynctty_receive+0x2e/0x710 ~#SysRq : Show Blocked State free sibling task PC stack pid father child younger older pptpctrl D C02A18E0 0 5071 4646 5074 5094 5064 (L-TLB) df3a3bd0 00000082 0029b837 c02a18e0 00000246 00000000 dd4f131c dd563cac def86030 c140864c 00000000 00000000 00000009 def86030 2ccaa8e5 0000017d 3b9aca00 def8613c c1407cc0 00000000 006a9953 00000000 de8f503c 00000046 Call Trace: [] __down+0x8c/0xf4 [] default_wake_function+0x0/0xc [] __down_failed+0x7/0xc [] ppp_asynctty_close+0x28/0x98 [] ppp_asynctty_close+0x93/0x98 [] ppp_asynctty_hangup+0x5/0x8 [] do_tty_hangup+0x119/0x391 [] release_dev+0x154/0x69b [] poison_obj+0x20/0x3d [] poison_obj+0x20/0x3d [] cache_free_debugcheck+0xc7/0x1dd [] destroy_inode+0x20/0x33 [] dput+0xd9/0x145 [] tty_release+0xf/0x18 [] __fput+0x3c/0xe7 [] filp_close+0x3e/0x62 [] put_files_struct+0xa5/0xb8 [] do_exit+0x124/0x7e2 [] vgacon_set_cursor_size+0x2d/0xeb [] do_trap+0x0/0xa9 [] do_invalid_op+0x0/0xab [] do_invalid_op+0xa2/0xab [] skb_under_panic+0x59/0x5d [] __wake_up+0x32/0x43 [] release_console_sem+0x22d/0x239 [] vprintk+0x1fb/0x37c [] error_code+0x7c/0x84 [] skb_under_panic+0x59/0x5d [] ppp_asynctty_receive+0x6d2/0x710 [] ppp_asynctty_receive+0x6e0/0x710 [] pty_write+0x39/0x41 [] write_chan+0x212/0x320 [] default_wake_function+0x0/0xc [] tty_write+0x11c/0x1d0 [] write_chan+0x0/0x320 [] vfs_write+0x87/0xf0 [] tty_write+0x0/0x1d0 [] sys_write+0x41/0x6a [] sysenter_past_esp+0x5f/0x99 =========== Dump registers Pid: 13874, comm: pptpctrl EIP: 0060:[] CPU: 0 EIP is at delay_tsc+0x0/0x13 EFLAGS: 00000246 Not tainted (2.6.20-gentoo #10) EAX: 00000001 EBX: c0353300 ECX: f52a298e EDX: c799bc55 ESI: 492d5b81 EDI: 00000000 EBP: 8eb4e7d4 DS: 007b ES: 007b GS: 00d8 CR0: 8005003b CR2: b7e598b8 CR3: 0e6f3000 CR4: 000006d0 [] __delay+0x6/0x7 [] _raw_spin_lock+0xa9/0x134 [] tty_ldisc_try+0x2f/0x33 [] lock_kernel+0x19/0x24 [] tty_read+0x5a/0xbe [] vfs_read+0x85/0xee [] tty_read+0x0/0xbe [] sys_read+0x41/0x6a [] sysenter_past_esp+0x5f/0x99 =========== Information about system: sh scripts/ver_linux If some fields are empty or look unusual you may have an old version. Compare to the current minimal requirements in Documentation/Changes. Linux vpn2 2.6.20-gentoo #10 SMP Mon Mar 5 11:07:33 MSK 2007 i686 Intel(R) Pentium(R) 4 CPU 2.40GHz GenuineIntel GNU/Linux Gnu C 4.1.2 Gnu make 3.81 binutils 2.17 util-linux 2.12r mount 2.12r module-init-tools 3.2.2 e2fsprogs 1.39 reiserfsprogs 3.6.19 PPP 2.4.4 Linux C Library > libc.2.5 Dynamic linker (ldd) 2.5 Procps 3.2.7 Net-tools 1.60 Kbd 1.12 Sh-utils 6.7 udev 104 Modules Loaded 8021q ipt_TCPMSS xt_tcpudp xt_pkttype iptable_filter ip_tables x_tables i2c_i801 i2c_core Jura