futex warning tracback on 2.6.29.1-rt6

futex warning tracback on 2.6.29.1-rt6

[Clark Williams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Thomas,

Got the below warning/traceback on 2.6.29.1-rt6.  This is running on my
T60, 64-bit kernel.  Haven't been able to figure out how to trigger it
yet. 

$ uname -a
Linux torg 2.6.29.1-rt6 #20 SMP PREEMPT RT Thu Apr 9 15:30:06 CDT 2009
x86_64 x86_64 x86_64 GNU/Linux

Clark

- ------------[ cut here ]------------
WARNING: at kernel/futex.c:184 drop_futex_key_refs+0x7e/0x111()
Hardware name:        
Modules linked in: tun fuse i915 drm i2c_algo_bit sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table loop dm_multipath scsi_dh uinput snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq arc4 ecb snd_seq_device iwl3945 snd_pcm_oss mac80211 btusb snd_mixer_oss snd_pcm snd_timer sr_mod iTCO_wdt snd cdrom lib80211 pl2303 iTCO_vendor_support video bluetooth yenta_socket ata_generic joydev pcspkr usbserial ata_piix soundcore i2c_i801 i2c_core cfg80211 rsrc_nonstatic e1000e sg ati_remote thinkpad_acpi rfkill hwmon output snd_page_alloc dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ahci libata sd_mod scsi_mod uhci_hcd ohci_hcd ssb [last unloaded: microcode]
Pid: 12898, comm: sbcl Not tainted 2.6.29.1-rt6 #20
Call Trace:
 [<ffffffff81065411>] warn_slowpath+0xe2/0x116
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff8152078b>] ? rt_mutex_unlock+0x47/0x67
 [<ffffffff810a7414>] ? rt_up_read+0x2b/0x2f
 [<ffffffff81044bbb>] ? get_user_pages_fast+0x1e4/0x235
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a023d>] drop_futex_key_refs+0x7e/0x111
 [<ffffffff810a04b9>] futex_wait_setup+0x1e9/0x1fb
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a08ee>] futex_wait+0x13f/0x2ce
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff81086896>] ? __wake_up_bit+0xd/0x51
 [<ffffffff811052c1>] ? page_waitqueue+0x9/0x68
 [<ffffffff811072f2>] ? unlock_page+0x27/0x2b
 [<ffffffff8109c92e>] ? __lock_acquire+0x659/0x744
 [<ffffffff81527701>] ? do_page_fault+0x7e2/0x80d
 [<ffffffff810157cd>] ? native_sched_clock+0x8c/0xba
 [<ffffffff8101573b>] ? sched_clock+0x42/0x48
 [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
 [<ffffffff8105b130>] ? default_wake_function+0x0/0x16
 [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
 [<ffffffff810a4937>] do_futex+0xaa/0x1d1
 [<ffffffff810a4c2b>] sys_futex+0x1cd/0x1dc
 [<ffffffff8100d1b2>] system_call_fastpath+0x16/0x1b
- ---[ end trace 384450cb13d7d800 ]---

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknetR0ACgkQHyuj/+TTEp2XkACgqc+Ko+v6cmDsbaVOg9tWaNUF
pO0AnRHVIqZYUo9PrxeKc4ws+BXZMyvN
=mJTq
-----END PGP SIGNATURE-----

Re: futex warning tracback on 2.6.29.1-rt6

[Darren Hart]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset=windows-1252; format=flowed, Size: 4382 bytes --]

Clark Williams wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Thomas,
> 
> Got the below warning/traceback on 2.6.29.1-rt6.  This is running on my
> T60, 64-bit kernel.  Haven't been able to figure out how to trigger it
> yet. 
> 
> $ uname -a
> Linux torg 2.6.29.1-rt6 #20 SMP PREEMPT RT Thu Apr 9 15:30:06 CDT 2009
> x86_64 x86_64 x86_64 GNU/Linux
> 
> Clark
> 
> - ------------[ cut here ]------------
> WARNING: at kernel/futex.c:184 drop_futex_key_refs+0x7e/0x111()

static void drop_futex_key_refs(union futex_key *key)
{
        if (!key->both.ptr) {
                /* If we're here then we tried to put a key we failed to get */
                WARN_ON_ONCE(1);
                return;
        }

OK, looks like we have some key management issues to sort out.  Clark, I presume you weren't running
the glibc patch to enable requeue_pi?  From the trace below this appears to have happened during a
normal futex_wait() call.  Looking at futex_wait_setup() I see a rather obvious bug:

        ret = get_futex_key(uaddr, fshared, &q->key);
        if (unlikely(ret != 0))
                goto out;

...

out:
        if (ret)
                put_futex_key(fshared, &q->key);

So if we fail to get the key, we goto out... and then put the key.  Oops.  I'll prepare a patch and send
out after some testing.


> Hardware name:        
> Modules linked in: tun fuse i915 drm i2c_algo_bit sunrpc ip6t_REJECT nf_conntrack_ipv6 ip6table_filter ip6_tables ipv6 cpufreq_ondemand acpi_cpufreq freq_table loop dm_multipath scsi_dh uinput snd_hda_codec_analog snd_hda_intel snd_hda_codec snd_hwdep snd_seq_dummy snd_seq_oss snd_seq_midi_event snd_seq arc4 ecb snd_seq_device iwl3945 snd_pcm_oss mac80211 btusb snd_mixer_oss snd_pcm snd_timer sr_mod iTCO_wdt snd cdrom lib80211 pl2303 iTCO_vendor_support video bluetooth yenta_socket ata_generic joydev pcspkr usbserial ata_piix soundcore i2c_i801 i2c_core cfg80211 rsrc_nonstatic e1000e sg ati_remote thinkpad_acpi rfkill hwmon output snd_page_alloc dm_snapshot dm_zero dm_mirror dm_region_hash dm_log dm_mod ahci libata sd_mod scsi_mod uhci_hcd ohci_hcd ssb [last unloaded: microcode]
> Pid: 12898, comm: sbcl Not tainted 2.6.29.1-rt6 #20
> Call Trace:
>  [<ffffffff81065411>] warn_slowpath+0xe2/0x116
>  [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
>  [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
>  [<ffffffff8152078b>] ? rt_mutex_unlock+0x47/0x67
>  [<ffffffff810a7414>] ? rt_up_read+0x2b/0x2f
>  [<ffffffff81044bbb>] ? get_user_pages_fast+0x1e4/0x235
>  [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
>  [<ffffffff810a023d>] drop_futex_key_refs+0x7e/0x111
>  [<ffffffff810a04b9>] futex_wait_setup+0x1e9/0x1fb
>  [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
>  [<ffffffff810a08ee>] futex_wait+0x13f/0x2ce
>  [<ffffffff8101573b>] ? sched_clock+0x42/0x48
>  [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
>  [<ffffffff8109e00a>] ? __lock_release+0x10c/0x125
>  [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
>  [<ffffffff81086896>] ? __wake_up_bit+0xd/0x51
>  [<ffffffff811052c1>] ? page_waitqueue+0x9/0x68
>  [<ffffffff811072f2>] ? unlock_page+0x27/0x2b
>  [<ffffffff8109c92e>] ? __lock_acquire+0x659/0x744
>  [<ffffffff81527701>] ? do_page_fault+0x7e2/0x80d
>  [<ffffffff810157cd>] ? native_sched_clock+0x8c/0xba
>  [<ffffffff8101573b>] ? sched_clock+0x42/0x48
>  [<ffffffff8109b621>] ? lock_release_holdtime+0x18d/0x192
>  [<ffffffff8105b130>] ? default_wake_function+0x0/0x16
>  [<ffffffff8100ce96>] ? ftrace_call+0x5/0x2b
>  [<ffffffff810a4937>] do_futex+0xaa/0x1d1
>  [<ffffffff810a4c2b>] sys_futex+0x1cd/0x1dc
>  [<ffffffff8100d1b2>] system_call_fastpath+0x16/0x1b
> - ---[ end trace 384450cb13d7d800 ]---
> 
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v2.0.10 (GNU/Linux)
> 
> iEYEARECAAYFAknetR0ACgkQHyuj/+TTEp2XkACgqc+Ko+v6cmDsbaVOg9tWaNUF
> pO0AnRHVIqZYUo9PrxeKc4ws+BXZMyvN
> =mJTq
> -----END PGP SIGNATURE-----
> N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±®Û¬z»"žØ^n‡r¡ö¦zË\x1aëh™¨è­Ú&¢ø\x1e®G«éh®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmml==


-- 
Darren Hart
IBM Linux Technology Center
Real-Time Linux Team
--
To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: futex warning tracback on 2.6.29.1-rt6

[Clark Williams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Fri, 10 Apr 2009 08:50:26 -0700
Darren Hart <dvhltc@us.ibm.com> wrote:
> 
> OK, looks like we have some key management issues to sort out.  Clark, I presume you weren't running
> the glibc patch to enable requeue_pi?  From the trace below this appears to have happened during a
> normal futex_wait() call.  Looking at futex_wait_setup() I see a rather obvious bug:
> 

That's correct, I was just running the stock F10 glibc.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.10 (GNU/Linux)

iEYEARECAAYFAknffZ0ACgkQHyuj/+TTEp2yXACeJB0p/GlgzjuMXU5MHGdOniNT
J2oAmwREpNfq83JOP+OuxTqPTaXkzLDQ
=tZu+
-----END PGP SIGNATURE-----