From: Shuah Khan <shuahkh@osg.samsung.com>
To: Mauro Carvalho Chehab <mchehab@osg.samsung.com>
Cc: Javier Martinez Canillas <javier@osg.samsung.com>,
Linux Media Mailing List <linux-media@vger.kernel.org>,
Shuah Khan <shuahkh@osg.samsung.com>
Subject: Re: Slow path and cpu lock warnings - MC Next Gen
Date: Wed, 11 Nov 2015 08:49:16 -0700 [thread overview]
Message-ID: <5643637C.3010205@osg.samsung.com> (raw)
In-Reply-To: <20151111133610.0d2f0923@recife.lan>
On 11/11/2015 08:36 AM, Mauro Carvalho Chehab wrote:
> Em Wed, 11 Nov 2015 07:22:47 -0700
> Shuah Khan <shuahkh@osg.samsung.com> escreveu:
>
>> On 11/11/2015 05:30 AM, Mauro Carvalho Chehab wrote:
>>> Em Mon, 09 Nov 2015 08:55:06 -0700
>>> Shuah Khan <shuahkh@osg.samsung.com> escreveu:
>>>
>>>> On 11/09/2015 08:51 AM, Shuah Khan wrote:
>>>>> As I mentioned on the IRC, here is the log for the problems I am seeing.
>>>>> I have to do eject HVR 950Q TV stick to see the problem.
>>>>>
>>>>> mc_next_gen.v8.4 branch with no changes.
>>>>>
>>>>> I can test and debug this week.
>>>>>
>>>>> thanks,
>>>>> -- Shuah
>>>>>
>>>>
>>>> Forgot to cc linux-media, just in case others are interested
>>>> and have ideas on debugging.
>>>>
>>>> thanks,
>>>> -- Shuah
>>>
>>>> [ 8.558049] ======================================================
>>>> [ 8.558056] [ INFO: possible circular locking dependency detected ]
>>>> [ 8.558063] 4.2.0-rc2+ #21 Not tainted
>>>> [ 8.558070] -------------------------------------------------------
>>>> [ 8.558077] systemd-udevd/143 is trying to acquire lock:
>>>> [ 8.558084] (init_mutex){+.+.+.}, at: [<ffffffffa0001f36>] acpi_video_get_backlight_type+0x17/0x163 [video]
>>>> [ 8.558104]
>>>> but task is already holding lock:
>>>> [ 8.558114] (&(&backlight_notifier)->rwsem){++++..}, at: [<ffffffff8107fc49>] __blocking_notifier_call_chain+0x39/0x70
>>>> [ 8.558133]
>>>> which lock already depends on the new lock.
>>>>
>>>> [ 8.558147]
>>>> the existing dependency chain (in reverse order) is:
>>>> [ 8.558158]
>>>> -> #1 (&(&backlight_notifier)->rwsem){++++..}:
>>>> [ 8.558170] [<ffffffff810b0c61>] lock_acquire+0xb1/0x130
>>>> [ 8.558180] [<ffffffff817fb006>] down_write+0x36/0x70
>>>> [ 8.558189] [<ffffffff8107fdc1>] blocking_notifier_chain_register+0x21/0xb0
>>>> [ 8.558202] [<ffffffff81424c98>] backlight_register_notifier+0x18/0x20
>>>> [ 8.558212] [<ffffffffa0002019>] acpi_video_get_backlight_type+0xfa/0x163 [video]
>>>> [ 8.558225] [<ffffffffa000167e>] acpi_video_bus_add+0x686/0xdfa [video]
>>>> [ 8.558237] [<ffffffff814371b0>] acpi_device_probe+0x50/0xf7
>>>> [ 8.558247] [<ffffffff814db62d>] driver_probe_device+0x14d/0x470
>>>> [ 8.558257] [<ffffffff814db9e4>] __driver_attach+0x94/0xa0
>>>> [ 8.558266] [<ffffffff814d9366>] bus_for_each_dev+0x66/0xa0
>>>> [ 8.558275] [<ffffffff814daece>] driver_attach+0x1e/0x20
>>>> [ 8.558287] [<ffffffff814daa4e>] bus_add_driver+0x1ee/0x280
>>>> [ 8.558296] [<ffffffff814dc280>] driver_register+0x60/0xe0
>>>> [ 8.558304] [<ffffffff81437936>] acpi_bus_register_driver+0x3b/0x43
>>>> [ 8.558313] [<ffffffffa0000328>] acpi_video_register+0x6e/0x90 [video]
>>>> [ 8.558322] [<ffffffffa000b087>] hid_generic_exit+0x87/0x1000 [hid_generic]
>>>> [ 8.558334] [<ffffffff810002eb>] do_one_initcall+0xab/0x1d0
>>>> [ 8.558344] [<ffffffff817f317e>] do_init_module+0x60/0x1e9
>>>> [ 8.558353] [<ffffffff810f1900>] load_module+0x2170/0x2780
>>>> [ 8.558363] [<ffffffff810f212a>] SyS_finit_module+0x9a/0xc0
>>>> [ 8.558372] [<ffffffff817fd5d7>] entry_SYSCALL_64_fastpath+0x12/0x6f
>>>> [ 8.558381]
>>>> -> #0 (init_mutex){+.+.+.}:
>>>> [ 8.558392] [<ffffffff810aff59>] __lock_acquire+0x1f29/0x1f90
>>>> [ 8.558401] [<ffffffff810b0c61>] lock_acquire+0xb1/0x130
>>>> [ 8.558409] [<ffffffff817f9ceb>] mutex_lock_nested+0x4b/0x340
>>>> [ 8.558417] [<ffffffffa0001f36>] acpi_video_get_backlight_type+0x17/0x163 [video]
>>>> [ 8.558430] [<ffffffffa00020ba>] acpi_video_backlight_notify+0x19/0x2f [video]
>>>> [ 8.558442] [<ffffffff8107faad>] notifier_call_chain+0x5d/0x80
>>>> [ 8.558451] [<ffffffff8107fc61>] __blocking_notifier_call_chain+0x51/0x70
>>>> [ 8.558462] [<ffffffff8107fc96>] blocking_notifier_call_chain+0x16/0x20
>>>> [ 8.558474] [<ffffffff81424ff2>] backlight_device_register+0x1a2/0x260
>>>> [ 8.558483] [<ffffffffa02af70a>] radeon_atom_backlight_init+0xda/0x1d0 [radeon]
>>>> [ 8.558543] [<ffffffffa0254713>] radeon_link_encoder_connector+0xc3/0x130 [radeon]
>>>> [ 8.558585] [<ffffffffa0232b06>] radeon_get_atom_connector_info_from_object_table+0x3a6/0x950 [radeon]
>>>> [ 8.558625] [<ffffffffa0257e4c>] radeon_modeset_init+0x5dc/0xa40 [radeon]
>>>> [ 8.558667] [<ffffffffa0230b0b>] radeon_driver_load_kms+0x12b/0x220 [radeon]
>>>> [ 8.558706] [<ffffffffa0033281>] drm_dev_register+0xb1/0x100 [drm]
>>>> [ 8.558728] [<ffffffffa003605d>] drm_get_pci_dev+0x8d/0x1e0 [drm]
>>>> [ 8.558747] [<ffffffffa022c404>] radeon_pci_probe+0xa4/0xc0 [radeon]
>>>> [ 8.558783] [<ffffffff813f3715>] local_pci_probe+0x45/0xa0
>>>> [ 8.558792] [<ffffffff813f4911>] pci_device_probe+0xd1/0x120
>>>> [ 8.558800] [<ffffffff814db62d>] driver_probe_device+0x14d/0x470
>>>> [ 8.558809] [<ffffffff814db9e4>] __driver_attach+0x94/0xa0
>>>> [ 8.558818] [<ffffffff814d9366>] bus_for_each_dev+0x66/0xa0
>>>> [ 8.558827] [<ffffffff814daece>] driver_attach+0x1e/0x20
>>>> [ 8.558835] [<ffffffff814daa4e>] bus_add_driver+0x1ee/0x280
>>>> [ 8.558844] [<ffffffff814dc280>] driver_register+0x60/0xe0
>>>> [ 8.558852] [<ffffffff813f2fd4>] __pci_register_driver+0x64/0x70
>>>> [ 8.558860] [<ffffffffa0036290>] drm_pci_init+0xe0/0x110 [drm]
>>>> [ 8.558879] [<ffffffffa039d09d>] radeon_init+0x9d/0xb2 [radeon]
>>>> [ 8.558911] [<ffffffff810002eb>] do_one_initcall+0xab/0x1d0
>>>> [ 8.558920] [<ffffffff817f317e>] do_init_module+0x60/0x1e9
>>>> [ 8.558928] [<ffffffff810f1900>] load_module+0x2170/0x2780
>>>> [ 8.558937] [<ffffffff810f212a>] SyS_finit_module+0x9a/0xc0
>>>> [ 8.558945] [<ffffffff817fd5d7>] entry_SYSCALL_64_fastpath+0x12/0x6f
>>>> [ 8.558954]
>>>> other info that might help us debug this:
>>>>
>>>> [ 8.558968] Possible unsafe locking scenario:
>>>>
>>>> [ 8.558978] CPU0 CPU1
>>>> [ 8.558984] ---- ----
>>>> [ 8.558989] lock(&(&backlight_notifier)->rwsem);
>>>> [ 8.558997] lock(init_mutex);
>>>> [ 8.559004] lock(&(&backlight_notifier)->rwsem);
>>>> [ 8.559015] lock(init_mutex);
>>>> [ 8.559021]
>>>> *** DEADLOCK ***
>>>>
>>>> [ 8.559035] 4 locks held by systemd-udevd/143:
>>>> [ 8.559041] #0: (&dev->mutex){......}, at: [<ffffffff814db99b>] __driver_attach+0x4b/0xa0
>>>> [ 8.559056] #1: (&dev->mutex){......}, at: [<ffffffff814db9a9>] __driver_attach+0x59/0xa0
>>>> [ 8.559072] #2: (drm_global_mutex){+.+.+.}, at: [<ffffffffa00331f6>] drm_dev_register+0x26/0x100 [drm]
>>>> [ 8.559097] #3: (&(&backlight_notifier)->rwsem){++++..}, at: [<ffffffff8107fc49>] __blocking_notifier_call_chain+0x39/0x70
>>>> [ 8.559113]
>>>> stack backtrace:
>>>> [ 8.559124] CPU: 2 PID: 143 Comm: systemd-udevd Not tainted 4.2.0-rc2+ #21
>>>> [ 8.559132] Hardware name: Hewlett-Packard HP ProBook 6475b/180F, BIOS 68TTU Ver. F.04 08/03/2012
>>>> [ 8.559143] ffffffff8285ace0 ffff88002b09f378 ffffffff817f4254 ffffffff810c155a
>>>> [ 8.559156] ffffffff8285ace0 ffff88002b09f3c8 ffffffff810ac6a3 0000000000000003
>>>> [ 8.559169] ffff88002b09f438 ffff88002b09f3c8 ffff88002b06b398 ffff88002b06a600
>>>> [ 8.559181] Call Trace:
>>>> [ 8.559190] [<ffffffff817f4254>] dump_stack+0x45/0x57
>>>> [ 8.559208] [<ffffffff810c155a>] ? console_unlock+0x1da/0x580
>>>> [ 8.559216] [<ffffffff810ac6a3>] print_circular_bug+0x1e3/0x250
>>>> [ 8.559225] [<ffffffff810aff59>] __lock_acquire+0x1f29/0x1f90
>>>> [ 8.559234] [<ffffffff810b0c61>] lock_acquire+0xb1/0x130
>>>> [ 8.559243] [<ffffffffa0001f36>] ? acpi_video_get_backlight_type+0x17/0x163 [video]
>>>> [ 8.559255] [<ffffffff817f9ceb>] mutex_lock_nested+0x4b/0x340
>>>> [ 8.559264] [<ffffffffa0001f36>] ? acpi_video_get_backlight_type+0x17/0x163 [video]
>>>> [ 8.559277] [<ffffffffa0001f36>] acpi_video_get_backlight_type+0x17/0x163 [video]
>>>> [ 8.559289] [<ffffffffa00020ba>] acpi_video_backlight_notify+0x19/0x2f [video]
>>>> [ 8.559301] [<ffffffff8107faad>] notifier_call_chain+0x5d/0x80
>>>> [ 8.559310] [<ffffffff8107fc61>] __blocking_notifier_call_chain+0x51/0x70
>>>> [ 8.559319] [<ffffffff8107fc96>] blocking_notifier_call_chain+0x16/0x20
>>>> [ 8.559327] [<ffffffff81424ff2>] backlight_device_register+0x1a2/0x260
>>>> [ 8.559374] [<ffffffffa02af70a>] radeon_atom_backlight_init+0xda/0x1d0 [radeon]
>>>> [ 8.559416] [<ffffffffa0254713>] radeon_link_encoder_connector+0xc3/0x130 [radeon]
>>>> [ 8.559455] [<ffffffffa0232b06>] radeon_get_atom_connector_info_from_object_table+0x3a6/0x950 [radeon]
>>>> [ 8.559481] [<ffffffffa0038c1b>] ? drm_mode_crtc_set_gamma_size+0x5b/0x70 [drm]
>>>> [ 8.559523] [<ffffffffa0257e4c>] radeon_modeset_init+0x5dc/0xa40 [radeon]
>>>> [ 8.559566] [<ffffffffa03071d8>] ? radeon_ib_ring_tests+0x58/0xc0 [radeon]
>>>> [ 8.559601] [<ffffffffa0230b0b>] radeon_driver_load_kms+0x12b/0x220 [radeon]
>>>> [ 8.559620] [<ffffffffa0033281>] drm_dev_register+0xb1/0x100 [drm]
>>>> [ 8.559639] [<ffffffffa003605d>] drm_get_pci_dev+0x8d/0x1e0 [drm]
>>>> [ 8.559674] [<ffffffffa022c404>] radeon_pci_probe+0xa4/0xc0 [radeon]
>>>> [ 8.559683] [<ffffffff813f3715>] local_pci_probe+0x45/0xa0
>>>> [ 8.559691] [<ffffffff813f47d0>] ? pci_match_device+0xe0/0x110
>>>> [ 8.559699] [<ffffffff813f4911>] pci_device_probe+0xd1/0x120
>>>> [ 8.559708] [<ffffffff814db62d>] driver_probe_device+0x14d/0x470
>>>> [ 8.559717] [<ffffffff814db9e4>] __driver_attach+0x94/0xa0
>>>> [ 8.559726] [<ffffffff814db950>] ? driver_probe_device+0x470/0x470
>>>> [ 8.559734] [<ffffffff814d9366>] bus_for_each_dev+0x66/0xa0
>>>> [ 8.559743] [<ffffffff814daece>] driver_attach+0x1e/0x20
>>>> [ 8.559752] [<ffffffff814daa4e>] bus_add_driver+0x1ee/0x280
>>>> [ 8.559760] [<ffffffff814dc280>] driver_register+0x60/0xe0
>>>> [ 8.559767] [<ffffffff813f2fd4>] __pci_register_driver+0x64/0x70
>>>> [ 8.559786] [<ffffffffa0036290>] drm_pci_init+0xe0/0x110 [drm]
>>>> [ 8.559794] [<ffffffffa039d000>] ? 0xffffffffa039d000
>>>> [ 8.559825] [<ffffffffa039d09d>] radeon_init+0x9d/0xb2 [radeon]
>>>> [ 8.559834] [<ffffffff810002eb>] do_one_initcall+0xab/0x1d0
>>>> [ 8.559843] [<ffffffff817f3146>] ? do_init_module+0x28/0x1e9
>>>> [ 8.559852] [<ffffffff811c5c3b>] ? kmem_cache_alloc_trace+0xbb/0x160
>>>> [ 8.559862] [<ffffffff817f317e>] do_init_module+0x60/0x1e9
>>>> [ 8.559870] [<ffffffff810f1900>] load_module+0x2170/0x2780
>>>> [ 8.559878] [<ffffffff810edd30>] ? __symbol_put+0x40/0x40
>>>> [ 8.559888] [<ffffffff810f212a>] SyS_finit_module+0x9a/0xc0
>>>> [ 8.559897] [<ffffffff817fd5d7>] entry_SYSCALL_64_fastpath+0x12/0x6f
>>>
>>> Sorry, but I fail to see how this is related to the V4L2 subsystem.
>>>
>>> At least on my eyes, it seems that the bug is somewhere at the Radeon
>>> driver.
>>>
>>
>> Mauro,
>>
>> I think you didn't look down the dmesg far enough. The following is the
>> problem I am talking about and you will see media_device_unregister()
>> on the stack. This occurs as soon as the device is removed.
>
> Shuah,
>
> I saw that, but it is clear, from the above log, that the Radeon
> driver is broken and it has some bad lock dependencies with the
> driver_attach locks. Any other bad lock report related to the
> Radeon driver or driver binding/unbiding code are very likely
> related to the above bug.
>
> You should either fix the bad lock at the Radeon driver or not
> load it at all, in order to be able to get any reliable results
> about possible locking troubles with the MC drivers with the Kernel
> lock tests.
>
Yeah Radeon driver bug could be making things worse. Did you see
any problems with device removal during your testing?
ok found the following commit that fixes the problem:
7231ed1a813e0a9d249bbbe58e66ca058aee83e1
This went into 4.2-rc4 or rc5. I will test applying just this
one patch to mc_next_gen.v8.4 branch and see if device removal
problem also goes away.
thanks,
-- Shuah
--
Shuah Khan
Sr. Linux Kernel Developer
Open Source Innovation Group
Samsung Research America (Silicon Valley)
shuahkh@osg.samsung.com | (970) 217-8978
next prev parent reply other threads:[~2015-11-11 15:49 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
[not found] <5640C0F2.9070000@osg.samsung.com>
[not found] ` <5640C1DA.7040100@osg.samsung.com>
2015-11-11 12:30 ` Slow path and cpu lock warnings - MC Next Gen Mauro Carvalho Chehab
2015-11-11 14:22 ` Shuah Khan
2015-11-11 15:36 ` Mauro Carvalho Chehab
2015-11-11 15:49 ` Shuah Khan [this message]
2015-11-12 0:25 ` Shuah Khan
2015-11-12 3:25 ` Shuah Khan
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5643637C.3010205@osg.samsung.com \
--to=shuahkh@osg.samsung.com \
--cc=javier@osg.samsung.com \
--cc=linux-media@vger.kernel.org \
--cc=mchehab@osg.samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.