Linux GPIO subsystem development
 help / color / mirror / Atom feed
From: Mario Limonciello <mario.limonciello@amd.com>
To: Hardik Prakash <hardikprakash.official@gmail.com>,
	Thorsten Leemhuis <regressions@leemhuis.info>
Cc: Andy Shevchenko <andriy.shevchenko@intel.com>,
	Andi Shyti <andi.shyti@kernel.org>,
	linux-i2c@vger.kernel.org, linux-gpio@vger.kernel.org,
	wsa@kernel.org, brgl@bgdev.pl, basavaraj.natikar@amd.com,
	linusw@kernel.org, nathan@kernel.org,
	chaitanya.kumar.borah@intel.com,
	Linux kernel regressions list <regressions@lists.linux.dev>
Subject: Re: i2c designware change broke touchpad of a thinkpad
Date: Wed, 12 Aug 2026 07:34:58 -0500	[thread overview]
Message-ID: <444caa3c-01ae-4516-b63a-576e095fef94@amd.com> (raw)
In-Reply-To: <CANTFpSWhqkw4735EFyQ4XoSEiqsgCKt4jTxH4SDO-m3=ybwccw@mail.gmail.com>



On 8/12/26 00:45, Hardik Prakash wrote:
> On Wed, 12 Aug 2026, Thorsten Leemhuis wrote:
>> While we wait for Mario a quick question: Should we revert
>> the culprit for 7.2 and reapply it later once we sorted this out? Of
>> course I'll continue to help debugging this.
>>
>> If it was just me I'd say "don't bother", but these kind of Thinkpads
>> are what some Linux users frequently buy, so I fear that other people
>> will run into this if the culprit makes it into 7.2.
> 
> I agreed, I'd rather have this reverted than ship a known regression.
> 
> Andi, WSA -- given 7.2 is close, could this go out as a revert now? I'll
> keep working the AMD bus-timing angle with Thorsten and Mario, and
> we can resubmit a corrected fix for a later release once it's actually
> understood.

Let's please revert for 7.2 final and keep working at this.

I've got a /hunch/ what's going on.

For both Dell and Lenovo I know a lot of designs have these I2C buses 
that have both the EC and the APU connected to them.  "Both" the EC and 
APU "can" operate as master.

The reason that this exists is so that when the system is in pre-boot 
the EC acts as master.  This lets you use the touchpad in the BIOS menus 
for example.  When the OS boots up there is some ASL called which sends 
an EC command that tells the EC to stand down.  At this point the APU 
takes over and acts as master.

So my theory here is that by changing this ordering the _DSM is not 
called at the right timing anymore.

This is a fairly similar issue as was seen that led to this commit:

https://git.kernel.org/torvalds/c/7d62beb102d6f
> 
> Thanks,
> Hardik
> 
> On Wed, 12 Aug 2026 at 10:51, Thorsten Leemhuis
> <regressions@leemhuis.info> wrote:
>>
>> On 8/12/26 06:59, Hardik Prakash wrote:
>>> On Mon, 10 Aug 2026, Thorsten Leemhuis wrote:
>>>> Did a fresh boot and uploaded the dmesg again as well just in case:
>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg2
>>>> https://www.leemhuis.info/files/misc/lsmod
>>>> https://www.leemhuis.info/files/misc/interrupts
>>>
>>> Thanks. Second boot reproduces the identical pattern: device_is_bound()
>>> =true at 1.132s, lost arbitration at 2.139s -- same ~1s gap as the first
>>> capture, so this is consistent and reproducible, not a one-off.
>>>
>>> interrupts rules out simple IRQ-line sharing: AMDI0010:01 has its own
>>> dedicated line (IRQ 11), separate from AMDI0010:00 (IRQ 10) and every
>>> amd_gpio ACPI:Event line. psp-1 exists (IR-PCI-MSIX-0000:c3:00.2) but
>>> shows zero recorded activity on any CPU.
>>>
>>> Since lost arbitration means an actual SDA/SCL-level collision (not an
>>> IRQ conflict), and there's no shared IRQ or visible PSP activity, if
>>> something else is touching this bus it's happening below what Linux's
>>> interrupt accounting can see -- which is why I wanted your take, Mario.
>>> Does this match a known pattern of firmware (PSP/EC) briefly taking
>>> this bus during boot on AMD platforms?
>>
>> Thx again. While we wait for Mario a quick question: Should we revert
>> the culprit for 7.2 and reapply it later once we sorted this out? Of
>> course I'll continue to help debugging this.
>>
>> If it was just me I'd say "don't bother", but these kind of Thinkpads
>> are what some Linux users frequently buy, so I fear that other people
>> will run into this if the culprit makes it into 7.2.
>>
>> Ciao, Thorsten
>>
>>> On Mon, 10 Aug 2026 at 12:18, Thorsten Leemhuis <linux@leemhuis.info> wrote:
>>>>
>>>> On 8/10/26 07:54, Hardik Prakash wrote:
>>>>> On Sun, 9 Aug 2026, Thorsten Leemhuis wrote:
>>>>>> Sorry, picked the wrong file while preparing the mail, it's here:
>>>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg
>>>>>
>>>>> Thanks, that's the one. Full timeline from that boot:
>>>>>
>>>>>    0.619s  AMDI0010:00 and AMDI0010:01 defer (AMDI0030:00 not registered)
>>>>>    1.136s  Retry succeeds: device_is_bound()=true for AMDI0030:00,
>>>>>            confirmed for all three children (XXXX0000:00, SYNA8018:00,
>>>>>            NXP1001:00)
>>>>>    2.222s  lost arbitration on AMDI0010:01 (x3)
>>>>>    3.260s  controller timed out, HID descriptor fetch fails
>>>>>
>>>>> The dependency check itself is behaving correctly here -- it defers,
>>>>> then correctly confirms AMDI0030:00 is bound before letting probe
>>>>> continue. The arbitration failure happens over a second *after* that,
>>>>> when the touchpad attempts its actual first transaction.
>>>>>
>>>>> So this doesn't look like a logic bug in the check itself. My read: by
>>>>> delaying dw_i2c_plat_probe() by ~500ms (correctly, for GPIO's sake),
>>>>> something else ends up contending for the AMDI0010:01 bus by the time
>>>>> the touchpad's first transaction actually happens -- a window that
>>>>> the original (undeferred, earlier) probe attempt apparently avoided.
>>>>>
>>>>> Mario -- does this look like a known AMD I2C/PSP semaphore contention
>>>>> pattern to you? Wondering if there's a shared bus/firmware consumer
>>>>> that could explain a ~1s-later collision like this, independent of
>>>>> the GPIO controller itself being ready.
>>>>>
>>>>> Thorsten, if you have it handy: /proc/interrupts and lsmod from this
>>>>> boot would help narrow down what else might be touching that bus
>>>>> around the 2.2s mark.
>>>>
>>>> Did a fresh boot and uploaded the dmesg again as well just in case:
>>>>
>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg2
>>>> https://www.leemhuis.info/files/misc/lsmod
>>>> https://www.leemhuis.info/files/misc/interrupts
>>>>
>>>> Ciao, Thorsten
>>>>> On Mon, 10 Aug 2026 at 11:06, Thorsten Leemhuis <linux@leemhuis.info> wrote:
>>>>>>
>>>>>> On 8/10/26 07:23, Hardik Prakash wrote:
>>>>>>> On Sat, 8 Aug 2026, Thorsten Leemhuis wrote:
>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert
>>>>>>>>
>>>>>>>> [    0.619018] i2c-dw-gpio-dbg: i2c_dw_check_gpio_dependencies(AMDI0010:00)
>>>>>>>> [...]
>>>>>>>> [    1.136010] i2c-dw-gpio-dbg: device_is_bound()=true for AMDI0030:00
>>>>>>>
>>>>>>> Thanks, this trace looks correct on its face (defer at 0.619s, retry
>>>>>>> succeeds at 1.122-1.136s), but the link above points to the same file
>>>>>>> you sent last round (the master+revert baseline, no debug patch, no
>>>>>>> i2c-dw-gpio-dbg lines in it at all) rather than this new capture.
>>>>>>>> Could you share the full, unfiltered dmesg from this exact boot (the
>>>>>>> one with the debug patch applied)? I need to see where
>>>>>>> "lost arbitration" falls relative to the trace above.
>>>>>>
>>>>>> Sorry, picked the wrong file while preparing the mail, it's here:
>>>>>> https://www.leemhuis.info/files/misc/dmesg_patch_dyndbg
>>>>>>
>>>>>> Ciao, Thorsten
>>>>>>
>>>>>>> Separately, from your DSDT: NFC1 (NXP1001), which shows up as a child
>>>>>>> of AMDI0010:01 in your trace, declares its _CRS as a raw resource
>>>>>>> buffer containing three GPIO resources referencing \_SB.GPIO -- one
>>>>>>> GpioIo and two separate GpioInt entries. Not sure yet if that's
>>>>>>> relevant, but it's a pattern I haven't seen on my hardware
>>>>>>> which I've tested against.
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Hardik
>>>>>>>
>>>>>>> On Sun, 9 Aug 2026 at 21:49, Thorsten Leemhuis <linux@leemhuis.info> wrote:
>>>>>>>>
>>>>>>>> On 8/8/26 22:25, Andy Shevchenko wrote:
>>>>>>>>> On Sat, Aug 08, 2026 at 10:31:28PM +0530, Hardik Prakash wrote:
>>>>>>>>>> On Fri, 7 Aug 2026, Thorsten Leemhuis wrote:
>>>>>>>>>>> This became 0a4bb2abc3e56d ("i2c: designware: defer probe if child
>>>>>>>>>>> GpioInt controllers are not bound") and causes a regression for me: the
>>>>>>>>>>> Touchpad in my ThinkPad T14s Gen 4 (AMD) stopped working. From the
>>>>>>>>>>> kernel log:
>>>>>>>>>>>
>>>>>>>>>>> i2c_designware AMDI0010:01: i2c_dw_handle_tx_abort: lost arbitration
>>>>>>>>>>> i2c_designware AMDI0010:01: controller timed out
>>>>>>>>>>> i2c_hid_acpi i2c-SYNA8018:00: failed to fetch HID descriptor: -110
>>>>>>>>>>> i2c_hid_acpi i2c-SYNA8018:00: Failed to fetch the HID Descriptor
>>>>>>>>>>>
>>>>>>>>>>> Reverting this change in current mainline/master (f9a2394a23482) fixed
>>>>>>>>>>> things for me. Full dmesg of current master and master+revert can be
>>>>>>>>>>> found here:
>>>>>>>>>>>
>>>>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master
>>>>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert
>>>>>>>>>> [...]
>>>>>>>>>> Two things that would help narrow this down, whenever you have a moment:
>>>>>>>>
>>>>>>>> Thx for looking into this. Here we go:
>>>>>>>>
>>>>>>>>>> 1. A DSDT dump
>>>>>>>>
>>>>>>>> https://www.leemhuis.info/files/misc/dsdt.dsl
>>>>>>>>
>>>>>>>>>> 2. A dmesg with dynamic debug from a kernel built with the attached
>>>>>>>>>>     patch on top of current mainline/master
>>>>>>>>
>>>>>>>> https://www.leemhuis.info/files/misc/dmesg-master-plus_revert
>>>>>>>>
>>>>>>>> [    0.619018] i2c-dw-gpio-dbg: i2c_dw_check_gpio_dependencies(AMDI0010:00)
>>>>>>>> [    0.619018] i2c-dw-gpio-dbg: check_child_gpioint(adev=XXXX0000:00)
>>>>>>>> [    0.619076] i2c-dw-gpio-dbg: GpioInt resource_source=\_SB.GPIO (len=10)
>>>>>>>> [    0.619079] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded,
>>>>>>>> device=AMDI0030:00
>>>>>>>> [    0.619080] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() returned
>>>>>>>> NULL, deferring
>>>>>>>> [    0.619082] i2c-dw-gpio-dbg: check_child_gpioint ret=-517 (defer)
>>>>>>>> [    0.619102] i2c-dw-gpio-dbg: i2c_dw_check_gpio_dependencies(AMDI0010:01)
>>>>>>>> [    0.619104] i2c-dw-gpio-dbg: check_child_gpioint(adev=SYNA8018:00)
>>>>>>>> [    0.619159] i2c-dw-gpio-dbg: GpioInt resource_source=\_SB.GPIO (len=10)
>>>>>>>> [    0.619162] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded,
>>>>>>>> device=AMDI0030:00
>>>>>>>> [    0.619162] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() returned
>>>>>>>> NULL, deferring
>>>>>>>> [    0.619164] i2c-dw-gpio-dbg: check_child_gpioint ret=-517 (defer)
>>>>>>>> [    1.122517] i2c-dw-gpio-dbg: i2c_dw_check_gpio_dependencies(AMDI0010:00)
>>>>>>>> [    1.122523] i2c-dw-gpio-dbg: check_child_gpioint(adev=XXXX0000:00)
>>>>>>>> [    1.122625] i2c-dw-gpio-dbg: GpioInt resource_source=\_SB.GPIO (len=10)
>>>>>>>> [    1.122636] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded,
>>>>>>>> device=AMDI0030:00
>>>>>>>> [    1.122639] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() succeeded
>>>>>>>> [    1.122642] i2c-dw-gpio-dbg: device_is_bound()=true for AMDI0030:00
>>>>>>>> [    1.135894] i2c-dw-gpio-dbg: i2c_dw_check_gpio_dependencies(AMDI0010:01)
>>>>>>>> [    1.135898] i2c-dw-gpio-dbg: check_child_gpioint(adev=SYNA8018:00)
>>>>>>>> [    1.135976] i2c-dw-gpio-dbg: GpioInt resource_source=\_SB.GPIO (len=10)
>>>>>>>> [    1.135980] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded,
>>>>>>>> device=AMDI0030:00
>>>>>>>> [    1.135983] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() succeeded
>>>>>>>> [    1.135985] i2c-dw-gpio-dbg: device_is_bound()=true for AMDI0030:00
>>>>>>>> [    1.135987] i2c-dw-gpio-dbg: check_child_gpioint(adev=NXP1001:00)
>>>>>>>> [    1.136005] i2c-dw-gpio-dbg: GpioInt resource_source=\_SB.GPIO (len=10)
>>>>>>>> [    1.136008] i2c-dw-gpio-dbg: acpi_fetch_acpi_dev() succeeded,
>>>>>>>> device=AMDI0030:00
>>>>>>>> [    1.136009] i2c-dw-gpio-dbg: gpio_device_find_by_fwnode() succeeded
>>>>>>>> [    1.136010] i2c-dw-gpio-dbg: device_is_bound()=true for AMDI0030:00
>>>>>>>>
>>>>>>>> HTH, Ciao, Thorsten
>>>>>>>>
>>>>>>>>>> On Fri, 7 Aug 2026 at 19:37, Thorsten Leemhuis <linux@leemhuis.info> wrote:
>>>>>>>>>>> On 7/28/26 10:30, Andi Shyti wrote:
>>>>>>>>>>>> On Sat, Jul 18, 2026 at 11:13:31AM +0530, Hardik Prakash wrote:
>>>>>>>>>>>>> I2C controllers may have child devices with GpioInt resources that
>>>>>>>>>>>>> depend on GPIO controllers being fully initialized. If the I2C
>>>>>>>>>>>>> controller probes and enumerates children before the referenced GPIO
>>>>>>>>>>>>> controller has completed probe, GPIO interrupts may not be properly
>>>>>>>>>>>>> configured, leading to device failures.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Lenovo Yoga 7 14AGP11, the WACF2200 touchscreen (child of
>>>>>>>>>>>>> AMDI0010:02) has a GpioInt resource pointing to GPIO 157 on the
>>>>>>>>>>>>> pinctrl-amd controller (AMDI0030:00). When i2c-designware probes
>>>>>>>>>>>>> AMDI0010:02 before pinctrl-amd finishes initializing, I2C transactions
>>>>>>>>>>>>> fail with lost arbitration errors:
>>>>>>>>>>>>>
>>>>>>>>>>>>>    0.285952  amd_gpio_probe: registering gpiochip  <- GPIO chip visible
>>>>>>>>>>>>>    0.287121  amd_gpio_probe: requesting parent IRQ <- probe still running
>>>>>>>>>>>>>    0.301454  AMDI0010:02 dw_i2c_plat_probe: start  <- races here
>>>>>>>>>>>>>    2.348157  lost arbitration
>>>>>>>>>>>>>
>>>>>>>>>>>>> Add a dependency check that walks ACPI child devices and defers probe
>>>>>>>>>>>>> until any referenced GPIO controller is bound.
>>>>>>>>>
>>>>>>>>> [...]
>>>>>>>>>
>>>>>>>>>>> #regzbot introduced: 0a4bb2abc3e56d
>>>>>>>>>>> #regzbot title i2c: designware: ThinkPad touchpad broke
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>


      reply	other threads:[~2026-08-12 12:35 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-07-18  5:43 [PATCH v13] i2c: designware: defer probe if child GpioInt controllers are not bound Hardik Prakash
2026-07-18  8:50 ` Andy Shevchenko
2026-07-20  8:07 ` Bartosz Golaszewski
2026-07-28  6:04   ` Hardik Prakash
2026-07-28  8:30 ` Andi Shyti
2026-08-07 14:07   ` i2c designware change broke touchpad of a thinkpad (was: i2c: designware: defer probe if child GpioInt controllers are not bound) Thorsten Leemhuis
2026-08-08 17:01     ` Hardik Prakash
     [not found]       ` <aneQvMN_Zet2b62n@ashevche-desk.local>
2026-08-09 16:19         ` i2c designware change broke touchpad of a thinkpad Thorsten Leemhuis
2026-08-10  5:23           ` Hardik Prakash
2026-08-10  5:36             ` Thorsten Leemhuis
2026-08-10  5:54               ` Hardik Prakash
2026-08-10  6:48                 ` Thorsten Leemhuis
2026-08-12  4:59                   ` Hardik Prakash
2026-08-12  5:21                     ` Thorsten Leemhuis
2026-08-12  5:45                       ` Hardik Prakash
2026-08-12 12:34                         ` Mario Limonciello [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=444caa3c-01ae-4516-b63a-576e095fef94@amd.com \
    --to=mario.limonciello@amd.com \
    --cc=andi.shyti@kernel.org \
    --cc=andriy.shevchenko@intel.com \
    --cc=basavaraj.natikar@amd.com \
    --cc=brgl@bgdev.pl \
    --cc=chaitanya.kumar.borah@intel.com \
    --cc=hardikprakash.official@gmail.com \
    --cc=linusw@kernel.org \
    --cc=linux-gpio@vger.kernel.org \
    --cc=linux-i2c@vger.kernel.org \
    --cc=nathan@kernel.org \
    --cc=regressions@leemhuis.info \
    --cc=regressions@lists.linux.dev \
    --cc=wsa@kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox