* Soft lockup with 2.6.17-rc1 on amd64
@ 2006-04-20 2:41 Steinar H. Gunderson
2006-04-20 4:07 ` Neil Brown
0 siblings, 1 reply; 5+ messages in thread
From: Steinar H. Gunderson @ 2006-04-20 2:41 UTC (permalink / raw)
To: linux-raid; +Cc: neilb
(Please Cc me on any replies, I'm not on the list.)
Hi,
2.6.17-rc1 on amd64 (dual-core, with SMP enabled) gives me:
[ 36.829054] md: md driver 0.90.3 MAX_MD_DEVS=256, MD_SB_DISKS=27
[ 36.835091] md: bitmap version 4.39
[ 36.839010] md: linear personality registered for level -1
[ 36.847169] md: raid0 personality registered for level 0
[ 36.855434] md: raid1 personality registered for level 1
[ 36.863676] md: raid10 personality registered for level 10
[ 36.871507] raid5: automatically using best checksumming function: generic_sse
[ 36.898420] generic_sse: 5938.000 MB/sec
[ 36.902697] raid5: using function: generic_sse (5938.000 MB/sec)
[ 36.909737] md: raid5 personality registered for level 5
[ 36.915052] md: raid4 personality registered for level 4
[ 36.990294] raid6: int64x1 1833 MB/s
[ 37.058190] raid6: int64x2 2497 MB/s
[ 37.126118] raid6: int64x4 2154 MB/s
[ 37.194059] raid6: int64x8 1619 MB/s
[ 37.261962] raid6: sse2x1 2359 MB/s
[ 37.329866] raid6: sse2x2 3183 MB/s
[ 37.397781] raid6: sse2x4 3405 MB/s
[ 37.401564] raid6: using algorithm sse2x4 (3405 MB/s)
[ 37.406668] md: raid6 personality registered for level 6
[ 37.726168] md: bind<dm-0>
[ 37.729096] md: bind<dm-1>
[ 37.742155] md: bind<dm-2>
[ 37.745266] md: bind<dm-3>
[ 47.736121] BUG: soft lockup detected on CPU#1!
[ 47.740655]
[ 47.740655] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 47.748483] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 47.758681] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 47.769352] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 47.778304] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 47.787981] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 47.797294] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 47.808192] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 47.817030] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 47.824421] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 47.833605] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 47.844313] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 47.854032] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 47.863675] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 47.871171] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 47.879082] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 47.887012] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 47.896239] <ffffffff8021817e>{ia32_sysret+0}
[ 57.727463] BUG: soft lockup detected on CPU#1!
[ 57.732020]
[ 57.732020] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 57.739860] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 57.750049] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 57.760667] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 57.769647] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 57.779342] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 57.788688] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 57.799570] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 57.808355] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 57.815791] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 57.824940] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 57.835724] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 57.845367] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 57.855034] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 57.862530] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 57.870450] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 57.878397] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 57.887633] <ffffffff8021817e>{ia32_sysret+0}
[ 67.718805] BUG: soft lockup detected on CPU#1!
[ 67.723353]
[ 67.723354] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 67.731193] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 67.741452] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 67.752089] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 67.761041] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 67.770711] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 67.780031] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 67.790930] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 67.799731] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 67.807151] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 67.816317] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 67.827007] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 67.836725] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 67.846378] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 67.853908] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 67.861871] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 67.869817] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 67.879080] <ffffffff8021817e>{ia32_sysret+0}
[ 77.710148] BUG: soft lockup detected on CPU#1!
[ 77.714685]
[ 77.714686] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 77.722545] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 77.732778] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 77.743447] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 77.752409] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 77.762113] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 77.771425] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 77.782349] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 77.791127] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 77.798527] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 77.807676] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 77.818384] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 77.828059] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 77.837701] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 77.845190] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 77.853135] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 77.861083] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 77.870336] <ffffffff8021817e>{ia32_sysret+0}
[ 87.701492] BUG: soft lockup detected on CPU#1!
[ 87.706010]
[ 87.706011] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 87.713853] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 87.724015] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 87.734626] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 87.743595] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 87.753291] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 87.762586] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 87.773502] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 87.782303] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 87.789723] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 87.798862] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 87.809579] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 87.819264] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 87.828959] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 87.836438] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 87.844373] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 87.852294] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 87.861548] <ffffffff8021817e>{ia32_sysret+0}
[ 97.696829] BUG: soft lockup detected on CPU#1!
[ 97.701351]
[ 97.701352] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 97.709160] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 97.719321] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 97.729977] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 97.738962] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 97.748614] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 97.757979] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 97.768843] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 97.777646] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 97.785045] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 97.794204] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 97.804953] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 97.814597] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 97.824248] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 97.831779] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 97.839715] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 97.847644] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 97.856854] <ffffffff8021817e>{ia32_sysret+0}
[ 107.688172] BUG: soft lockup detected on CPU#1!
[ 107.692746]
[ 107.692747] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 107.700624] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 107.710812] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 107.721491] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 107.730504] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 107.740147] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 107.749485] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 107.760350] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 107.769153] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 107.776561] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 107.785719] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 107.796460] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 107.806129] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 107.815806] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 107.823337] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 107.831266] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 107.839238] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 107.848493] <ffffffff8021817e>{ia32_sysret+0}
[ 117.679515] BUG: soft lockup detected on CPU#1!
[ 117.684064]
[ 117.684064] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
[ 117.691923] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
[ 117.702137] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
[ 117.712834] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
[ 117.721821] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
[ 117.731498] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
[ 117.740853] <ffffffff880c5923>{:md_mod:md_ioctl+353} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 117.751710] <ffffffff880c68c0>{:md_mod:md_ioctl+4350} <ffffffff80280120>{__d_lookup+173}
[ 117.760546] <ffffffff80277555>{do_lookup+99} <ffffffff8027ef84>{dput+33}
[ 117.767946] <ffffffff802c8786>{blkdev_driver_ioctl+91} <ffffffff802c8e94>{blkdev_ioctl+1690}
[ 117.777088] <ffffffff880c4fad>{:md_mod:md_open+44} <ffffffff8039e667>{__mutex_lock_interruptible_slowpath+816}
[ 117.787829] <ffffffff8039ea47>{__mutex_lock_slowpath+751} <ffffffff80270944>{check_disk_change+29}
[ 117.797506] <ffffffff880c4fdb>{:md_mod:md_open+90} <ffffffff8039e2e6>{__mutex_unlock_slowpath+403}
[ 117.807253] <ffffffff8027170f>{do_open+653} <ffffffff80270cfd>{bdget+274}
[ 117.814740] <ffffffff802709f8>{block_ioctl+25} <ffffffff8027ae59>{do_ioctl+33}
[ 117.822668] <ffffffff8027b0f8>{vfs_ioctl+597} <ffffffff8027b16a>{sys_ioctl+89}
[ 117.830668] <ffffffff80292346>{do_ioctl32_pointer+9} <ffffffff80291ed4>{compat_sys_ioctl+627}
[ 117.839904] <ffffffff8021817e>{ia32_sysret+0}
[ 123.802707] raid5: device dm-3 operational as raid disk 3
[ 123.808102] raid5: device dm-2 operational as raid disk 2
[ 123.813515] raid5: device dm-1 operational as raid disk 1
[ 123.818932] raid5: device dm-0 operational as raid disk 0
[ 123.824632] raid5: allocated 4262kB for md1
[ 123.828873] raid5: raid level 5 set md1 active with 4 out of 4 devices, algorithm 2
[ 123.836541] RAID5 conf printout:
[ 123.839818] --- rd:4 wd:4 fd:0
[ 123.842973] disk 0, o:1, dev:dm-0
[ 123.846376] disk 1, o:1, dev:dm-1
[ 123.849777] disk 2, o:1, dev:dm-2
[ 123.853187] disk 3, o:1, dev:dm-3
[ 123.888070] md: bind<dm-10>
[ 123.913148] md: bind<dm-11>
[ 123.927897] raid1: raid set md0 active with 2 out of 2 mirrors
After that, everything keeps working as normal. Any ideas?
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Soft lockup with 2.6.17-rc1 on amd64
2006-04-20 2:41 Soft lockup with 2.6.17-rc1 on amd64 Steinar H. Gunderson
@ 2006-04-20 4:07 ` Neil Brown
2006-04-20 4:53 ` Neil Brown
2006-04-20 6:35 ` Neil Brown
0 siblings, 2 replies; 5+ messages in thread
From: Neil Brown @ 2006-04-20 4:07 UTC (permalink / raw)
To: Steinar H. Gunderson; +Cc: linux-raid
On Thursday April 20, sgunderson@bigfoot.com wrote:
> (Please Cc me on any replies, I'm not on the list.)
Always!
> [ 37.406668] md: raid6 personality registered for level 6
> [ 37.726168] md: bind<dm-0>
> [ 37.729096] md: bind<dm-1>
> [ 37.742155] md: bind<dm-2>
> [ 37.745266] md: bind<dm-3>
> [ 47.736121] BUG: soft lockup detected on CPU#1!
> [ 47.740655]
> [ 47.740655] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
> [ 47.748483] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
> [ 47.758681] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
> [ 47.769352] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
> [ 47.778304] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
> [ 47.787981] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
....
> [ 123.802707] raid5: device dm-3 operational as raid disk 3
> [ 123.808102] raid5: device dm-2 operational as raid disk 2
Wow! 76 second to set up a raid5 array - all of that invalidating an
inode which would not have had any valid data in it!
Can you try this patch please?
Thanks,
NeilBrown
-------------------
Signed-off-by: Neil Brown <neilb@suse.de>
### Diffstat output
./fs/block_dev.c | 1 +
1 file changed, 1 insertion(+)
diff ./fs/block_dev.c~current~ ./fs/block_dev.c
--- ./fs/block_dev.c~current~ 2006-04-20 14:04:12.000000000 +1000
+++ ./fs/block_dev.c 2006-04-20 14:04:39.000000000 +1000
@@ -244,6 +244,7 @@ static struct inode *bdev_alloc_inode(st
struct bdev_inode *ei = kmem_cache_alloc(bdev_cachep, SLAB_KERNEL);
if (!ei)
return NULL;
+ inode_init_once(&ei->vfs_inode);
return &ei->vfs_inode;
}
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Soft lockup with 2.6.17-rc1 on amd64
2006-04-20 4:07 ` Neil Brown
@ 2006-04-20 4:53 ` Neil Brown
2006-04-20 6:35 ` Neil Brown
1 sibling, 0 replies; 5+ messages in thread
From: Neil Brown @ 2006-04-20 4:53 UTC (permalink / raw)
To: Steinar H. Gunderson, linux-raid
On Thursday April 20, neilb@suse.de wrote:
> On Thursday April 20, sgunderson@bigfoot.com wrote:
> > (Please Cc me on any replies, I'm not on the list.)
> Always!
>
>
> > [ 37.406668] md: raid6 personality registered for level 6
> > [ 37.726168] md: bind<dm-0>
> > [ 37.729096] md: bind<dm-1>
> > [ 37.742155] md: bind<dm-2>
> > [ 37.745266] md: bind<dm-3>
> > [ 47.736121] BUG: soft lockup detected on CPU#1!
> > [ 47.740655]
> > [ 47.740655] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
> > [ 47.748483] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
> > [ 47.758681] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
> > [ 47.769352] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
> > [ 47.778304] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
> > [ 47.787981] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
> ....
> > [ 123.802707] raid5: device dm-3 operational as raid disk 3
> > [ 123.808102] raid5: device dm-2 operational as raid disk 2
>
> Wow! 76 second to set up a raid5 array - all of that invalidating an
> inode which would not have had any valid data in it!
>
> Can you try this patch please?
Actually, I wouldn't bother. I've read further and missed the fact
that inode_init_once was being called already.
Is this delay reproducible at all? If so I'll get you to put some
tracing in.
However, I somehow doubt it is reproducible, as it is very similar to
https://bugzilla.novell.com/show_bug.cgi?id=152099
and that wasn't reproducible.
But if it is... great!
NeilBrown
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Soft lockup with 2.6.17-rc1 on amd64
2006-04-20 4:07 ` Neil Brown
2006-04-20 4:53 ` Neil Brown
@ 2006-04-20 6:35 ` Neil Brown
2006-04-20 11:05 ` Steinar H. Gunderson
1 sibling, 1 reply; 5+ messages in thread
From: Neil Brown @ 2006-04-20 6:35 UTC (permalink / raw)
To: Steinar H. Gunderson, linux-raid
On Thursday April 20, neilb@suse.de wrote:
> On Thursday April 20, sgunderson@bigfoot.com wrote:
> > (Please Cc me on any replies, I'm not on the list.)
> Always!
>
>
> > [ 37.406668] md: raid6 personality registered for level 6
> > [ 37.726168] md: bind<dm-0>
> > [ 37.729096] md: bind<dm-1>
> > [ 37.742155] md: bind<dm-2>
> > [ 37.745266] md: bind<dm-3>
> > [ 47.736121] BUG: soft lockup detected on CPU#1!
> > [ 47.740655]
> > [ 47.740655] Call Trace: <IRQ> <ffffffff80247e68>{softlockup_tick+218}
> > [ 47.748483] <ffffffff8022f59f>{update_process_times+66} <ffffffff80212ffa>{smp_local_timer_interrupt+35}
> > [ 47.758681] <ffffffff80213614>{smp_apic_timer_interrupt+65} <ffffffff8020a106>{apic_timer_interrupt+98} <EOI>
> > [ 47.769352] <ffffffff8028ad41>{mpage_writepages+730} <ffffffff802495bd>{find_get_pages+91}
> > [ 47.778304] <ffffffff8024fd99>{pagevec_lookup+23} <ffffffff8025096d>{invalidate_mapping_pages+183}
> > [ 47.787981] <ffffffff80212c8d>{smp_call_function+48} <ffffffff880c44e3>{:md_mod:do_md_run+664}
> ....
> > [ 123.802707] raid5: device dm-3 operational as raid disk 3
> > [ 123.808102] raid5: device dm-2 operational as raid disk 2
>
> Wow! 76 second to set up a raid5 array - all of that invalidating an
> inode which would not have had any valid data in it!
>
> Can you try this patch please?
Yeh, that one was completely wrong. I think this one will fix it, but
it probably isn't reproducible, so you probably cannot test it.
However: could you please give me details of dm-[0-3]? Particularly
how big they are, but also what sort of dm target and what underlying
devices.
Also, what is the clock speed of your processor - I assume it is not
just hyper-threading but actually has two independent core?
Thanks,
NeilBrown
---------------------------------------
Remove softlockup from invalidate_mapping_pages.
If invalidate_mapping_pages is called to invalidate a very large
mapping (e.g. a very large block device) and if the only active page
in that device is near the end (or at least, at a very large index),
such as, say, the superblock of an md array, and if that page
happens to be locked when invalidate_mapping_pages is called,
then
pagevec_lookup will return this page and
as it is locked, 'next' will be incremented and pagevec_lookup
will be called again. and again. and again.
while we count from 0 upto a very large number.
We should really always set 'next' to 'page->index+1' before going
around the loop again, not just if the page isn't locked.
Cc: "Steinar H. Gunderson" <sgunderson@bigfoot.com>
Signed-off-by: Neil Brown <neilb@suse.de>
### Diffstat output
./mm/truncate.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff ./mm/truncate.c~current~ ./mm/truncate.c
--- ./mm/truncate.c~current~ 2006-04-20 15:27:22.000000000 +1000
+++ ./mm/truncate.c 2006-04-20 15:38:20.000000000 +1000
@@ -238,13 +238,11 @@ unsigned long invalidate_mapping_pages(s
for (i = 0; i < pagevec_count(&pvec); i++) {
struct page *page = pvec.pages[i];
- if (TestSetPageLocked(page)) {
- next++;
+ next = page->index+1;
+
+ if (TestSetPageLocked(page))
continue;
- }
- if (page->index > next)
- next = page->index;
- next++;
+
if (PageDirty(page) || PageWriteback(page))
goto unlock;
if (page_mapped(page))
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Soft lockup with 2.6.17-rc1 on amd64
2006-04-20 6:35 ` Neil Brown
@ 2006-04-20 11:05 ` Steinar H. Gunderson
0 siblings, 0 replies; 5+ messages in thread
From: Steinar H. Gunderson @ 2006-04-20 11:05 UTC (permalink / raw)
To: Neil Brown; +Cc: linux-raid
On Thu, Apr 20, 2006 at 04:35:51PM +1000, Neil Brown wrote:
> However: could you please give me details of dm-[0-3]? Particularly
> how big they are, but also what sort of dm target and what underlying
> devices.
They're set up by EVMS, and point to partitions on a normal disk. They're
about 250GB each.
> Also, what is the clock speed of your processor - I assume it is not
> just hyper-threading but actually has two independent core?
pannekake:~# grep MHz /proc/cpuinfo
cpu MHz : 2009.274
cpu MHz : 2009.274
And yes, it's a real dual-core; I don't know any amd64 machines with
hyperthreading.
Note that I'm on serial console; that might be oddly related somehow?
/* Steinar */
--
Homepage: http://www.sesse.net/
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2006-04-20 11:05 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-04-20 2:41 Soft lockup with 2.6.17-rc1 on amd64 Steinar H. Gunderson
2006-04-20 4:07 ` Neil Brown
2006-04-20 4:53 ` Neil Brown
2006-04-20 6:35 ` Neil Brown
2006-04-20 11:05 ` Steinar H. Gunderson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).