linux-arm-kernel.lists.infradead.org archive mirror
 help / color / mirror / Atom feed
* [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
@ 2025-06-05  3:02 程凌飞
  2025-06-05 10:25 ` Sven Peter
  0 siblings, 1 reply; 6+ messages in thread
From: 程凌飞 @ 2025-06-05  3:02 UTC (permalink / raw)
  To: sven, j, alyssa, neal
  Cc: zhangzhenwei22b, wangzhe12, chenglingfei22s, maddy, mpe, npiggin,
	christophe.leroy, naveen, andi.shyti, asahi, linux-arm-kernel,
	linuxppc-dev, linux-i2c, linux-kernel

Hi, all!

We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux. 

The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services. 
At that time, the i2c-pasemi-platform was i2c-apple.

We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched 
the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and 
tested again with Linux v6.14.6, but the crash remains.

Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware 
design constraints, we can not record the panic logs through kdump.

Thus we tried to find the root cause of the issue manually. When we perform rmmod, the kernel performs device releasing on 
the i2c bus, then calls the remove function in snd-soc-cs42l83-i2c, which calls the cs42l42_common_remove in cs42l42, 
because cs42l42->init_done is true, it performs regmap_write, and finally calls into pasemi_smb_waitready in i2c-pasemi
-core.c. We noticed that smbus->use_irq is true, and after it calls into wait_for_completion_timeout, the system crashs!

We found that wait_for_completion_timeout is one of the core scheduler APIs used by tens of thousands of other drivers,
it is unlikely causing the crash. So we tried to remove the call to wait_for_completion_timeout, then the system seems to
run well.

However, because we have little knowledge about i2c devices and specifications, we are not sure whether this change will 
cause other potential harms for the system and device. Is this call to wait necesary here? Or can you give a more 
sophisticated fix?

We’re happy to provide additional logs, configs, or testing assistance. Any guidance would be greatly appreciated!



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
  2025-06-05  3:02 [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1 程凌飞
@ 2025-06-05 10:25 ` Sven Peter
  2025-06-05 11:55   ` chenglingfei
  0 siblings, 1 reply; 6+ messages in thread
From: Sven Peter @ 2025-06-05 10:25 UTC (permalink / raw)
  To: 程凌飞, j, alyssa, neal
  Cc: zhangzhenwei22b, wangzhe12, maddy, mpe, npiggin, christophe.leroy,
	naveen, andi.shyti, asahi, linux-arm-kernel, linuxppc-dev,
	linux-i2c, linux-kernel

Hi,

On 05.06.25 05:02, 程凌飞 wrote:
> Hi, all!
> 
> We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux.
> 
> The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services.
> At that time, the i2c-pasemi-platform was i2c-apple.
> 
> We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
> Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched
> the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and
> tested again with Linux v6.14.6, but the crash remains.
> 
> Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware
> design constraints, we can not record the panic logs through kdump.

Do you have UART connected to a device under test which you could use to 
grab the panic log from the kernel? Alternatively you can also run the 
kernel under m1n1's hypervisor and grab the log that way. It'll emulate 
the serial port and redirect its output via USB.

> 
> Thus we tried to find the root cause of the issue manually. When we perform rmmod, the kernel performs device releasing on
> the i2c bus, then calls the remove function in snd-soc-cs42l83-i2c, which calls the cs42l42_common_remove in cs42l42,
> because cs42l42->init_done is true, it performs regmap_write, and finally calls into pasemi_smb_waitready in i2c-pasemi
> -core.c. We noticed that smbus->use_irq is true, and after it calls into wait_for_completion_timeout, the system crashs!>
> We found that wait_for_completion_timeout is one of the core scheduler APIs used by tens of thousands of other drivers,
> it is unlikely causing the crash. So we tried to remove the call to wait_for_completion_timeout, then the system seems to
> run well.
> 
> However, because we have little knowledge about i2c devices and specifications, we are not sure whether this change will
> cause other potential harms for the system and device. Is this call to wait necesary here? Or can you give a more
> sophisticated fix?

Yes, that call is necessary. It waits for the "transfer completed" 
interrupt from the hardware. Without it the driver will try to read data 
before it's available and you'll see corruption. I'm surprised hardware 
attached to i2c (usb pd controller and audio I think) works at all with 
that change.


Sven



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
  2025-06-05 10:25 ` Sven Peter
@ 2025-06-05 11:55   ` chenglingfei
  2025-06-05 14:02     ` Sven Peter
  0 siblings, 1 reply; 6+ messages in thread
From: chenglingfei @ 2025-06-05 11:55 UTC (permalink / raw)
  To: Sven Peter
  Cc: j, alyssa, neal, zhangzhenwei22b, wangzhe12, maddy, mpe, npiggin,
	christophe.leroy, naveen, andi.shyti, asahi, linux-arm-kernel,
	linuxppc-dev, linux-i2c, linux-kernel




> -----原始邮件-----
&gt; 发件人: "Sven Peter" <sven@kernel.org>
&gt; 发送时间: 2025-06-05 18:25:09 (星期四)
&gt; 收件人: 程凌飞 <chenglingfei22s@ict.ac.cn>, j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev
&gt; 抄送: zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
&gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
&gt; 
&gt; Hi,
&gt; 
&gt; On 05.06.25 05:02, 程凌飞 wrote:
&gt; &gt; Hi, all!
&gt; &gt; 
&gt; &gt; We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux.
&gt; &gt; 
&gt; &gt; The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services.
&gt; &gt; At that time, the i2c-pasemi-platform was i2c-apple.
&gt; &gt; 
&gt; &gt; We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
&gt; &gt; Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched
&gt; &gt; the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and
&gt; &gt; tested again with Linux v6.14.6, but the crash remains.
&gt; &gt; 
&gt; &gt; Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware
&gt; &gt; design constraints, we can not record the panic logs through kdump.
&gt; 
&gt; Do you have UART connected to a device under test which you could use to 
&gt; grab the panic log from the kernel? Alternatively you can also run the 
&gt; kernel under m1n1's hypervisor and grab the log that way. It'll emulate 
&gt; the serial port and redirect its output via USB.
&gt; 

I don't have UART, but I have tried to run the kernel under m1n1's hypervisor. However, it does not trigger the release of cs42l83. 
Given that m1n1 provides full peripheral device emulation capability, the most plausible explanation would be an incorrect 
firmware loading sequence. But the documentation of Asahi provides little details about how to generate an initramfs with 
firmware (I think), can you give more guidance about it?

&gt; &gt; 
&gt; &gt; Thus we tried to find the root cause of the issue manually. When we perform rmmod, the kernel performs device releasing on
&gt; &gt; the i2c bus, then calls the remove function in snd-soc-cs42l83-i2c, which calls the cs42l42_common_remove in cs42l42,
&gt; &gt; because cs42l42-&gt;init_done is true, it performs regmap_write, and finally calls into pasemi_smb_waitready in i2c-pasemi
&gt; &gt; -core.c. We noticed that smbus-&gt;use_irq is true, and after it calls into wait_for_completion_timeout, the system crashs!&gt;
&gt; &gt; We found that wait_for_completion_timeout is one of the core scheduler APIs used by tens of thousands of other drivers,
&gt; &gt; it is unlikely causing the crash. So we tried to remove the call to wait_for_completion_timeout, then the system seems to
&gt; &gt; run well.
&gt; &gt; 
&gt; &gt; However, because we have little knowledge about i2c devices and specifications, we are not sure whether this change will
&gt; &gt; cause other potential harms for the system and device. Is this call to wait necesary here? Or can you give a more
&gt; &gt; sophisticated fix?
&gt; 
&gt; Yes, that call is necessary. It waits for the "transfer completed" 
&gt; interrupt from the hardware. Without it the driver will try to read data 
&gt; before it's available and you'll see corruption. I'm surprised hardware 
&gt; attached to i2c (usb pd controller and audio I think) works at all with 
&gt; that change.
&gt; 
&gt; 
&gt; Sven

Are there any methods or tools to systematically verify its functionality? I am not sure whether the devices attached to i2c
should work well even after the i2c-pasemi-platform has been removed.
</chenglingfei22s@ict.ac.cn></sven@kernel.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
  2025-06-05 11:55   ` chenglingfei
@ 2025-06-05 14:02     ` Sven Peter
  2025-06-05 15:13       ` chenglingfei
  0 siblings, 1 reply; 6+ messages in thread
From: Sven Peter @ 2025-06-05 14:02 UTC (permalink / raw)
  To: chenglingfei
  Cc: j, alyssa, neal, zhangzhenwei22b, wangzhe12, maddy, mpe, npiggin,
	christophe.leroy, naveen, andi.shyti, asahi, linux-arm-kernel,
	linuxppc-dev, linux-i2c, linux-kernel

Hi,

On 05.06.25 13:55, chenglingfei wrote:
> 
> 
> 
> &gt; -----原始邮件-----
> &gt; 发件人: "Sven Peter" <sven@kernel.org>
> &gt; 发送时间: 2025-06-05 18:25:09 (星期四)
> &gt; 收件人: 程凌飞 <chenglingfei22s@ict.ac.cn>, j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev
> &gt; 抄送: zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
> &gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
> &gt;
> &gt; Hi,
> &gt;
> &gt; On 05.06.25 05:02, 程凌飞 wrote:
> &gt; &gt; Hi, all!
> &gt; &gt;
> &gt; &gt; We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux.
> &gt; &gt;
> &gt; &gt; The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services.
> &gt; &gt; At that time, the i2c-pasemi-platform was i2c-apple.
> &gt; &gt;
> &gt; &gt; We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
> &gt; &gt; Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched
> &gt; &gt; the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and
> &gt; &gt; tested again with Linux v6.14.6, but the crash remains.
> &gt; &gt;
> &gt; &gt; Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware
> &gt; &gt; design constraints, we can not record the panic logs through kdump.
> &gt;
> &gt; Do you have UART connected to a device under test which you could use to
> &gt; grab the panic log from the kernel? Alternatively you can also run the
> &gt; kernel under m1n1's hypervisor and grab the log that way. It'll emulate
> &gt; the serial port and redirect its output via USB.
> &gt;
> 
> I don't have UART, but I have tried to run the kernel under m1n1's hypervisor. However, it does not trigger the release of cs42l83.
> Given that m1n1 provides full peripheral device emulation capability, the most plausible explanation would be an incorrect
> firmware loading sequence. But the documentation of Asahi provides little details about how to generate an initramfs with
> firmware (I think), can you give more guidance about it?

I'm not sure why you are even trying to create a special initramfs. Just
load your usual kernel using the usual boot flow as a guest. There's 
also no firmware involved in i2c and I'm not sure what you mean with 
"full peripheral device emulation" either or how that's related to firmware.
You also mention that the crash happens when you run rmmod so I again 
don't understand what "it does not trigger the release of cs42l83" means 
here.

> 
> &gt; &gt;
> &gt; &gt; Thus we tried to find the root cause of the issue manually. When we perform rmmod, the kernel performs device releasing on
> &gt; &gt; the i2c bus, then calls the remove function in snd-soc-cs42l83-i2c, which calls the cs42l42_common_remove in cs42l42,
> &gt; &gt; because cs42l42-&gt;init_done is true, it performs regmap_write, and finally calls into pasemi_smb_waitready in i2c-pasemi
> &gt; &gt; -core.c. We noticed that smbus-&gt;use_irq is true, and after it calls into wait_for_completion_timeout, the system crashs!&gt;
> &gt; &gt; We found that wait_for_completion_timeout is one of the core scheduler APIs used by tens of thousands of other drivers,
> &gt; &gt; it is unlikely causing the crash. So we tried to remove the call to wait_for_completion_timeout, then the system seems to
> &gt; &gt; run well.
> &gt; &gt;
> &gt; &gt; However, because we have little knowledge about i2c devices and specifications, we are not sure whether this change will
> &gt; &gt; cause other potential harms for the system and device. Is this call to wait necesary here? Or can you give a more
> &gt; &gt; sophisticated fix?
> &gt;
> &gt; Yes, that call is necessary. It waits for the "transfer completed"
> &gt; interrupt from the hardware. Without it the driver will try to read data
> &gt; before it's available and you'll see corruption. I'm surprised hardware
> &gt; attached to i2c (usb pd controller and audio I think) works at all with
> &gt; that change.
> &gt;
> &gt;
> &gt; Sven
> 
> Are there any methods or tools to systematically verify its functionality? I am not sure whether the devices attached to i2c
> should work well even after the i2c-pasemi-platform has been removed.

I don't understand. You say you saw a crash inside pasemi_smb_waitready 
when calling wait_for_completion_timeout and decided to remove that 
method. When you remove the call you break the entire driver because it 
will now try to read data long before the i2c transaction has been 
completed.
Obviously, no i2c device will work when the driver isn't loaded but 
without waiting for the completion they also won't work when the driver 
is loaded.


Sven



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
  2025-06-05 14:02     ` Sven Peter
@ 2025-06-05 15:13       ` chenglingfei
  2025-06-05 19:58         ` Sven Peter
  0 siblings, 1 reply; 6+ messages in thread
From: chenglingfei @ 2025-06-05 15:13 UTC (permalink / raw)
  To: Sven Peter
  Cc: j, alyssa, neal, zhangzhenwei22b, chenglingfei22s, wangzhe12,
	maddy, mpe, npiggin, christophe.leroy, naveen, andi.shyti, asahi,
	linux-arm-kernel, linuxppc-dev, linux-i2c, linux-kernel




&gt; -----原始邮件-----
&gt; 发件人: "Sven Peter" <sven@kernel.org>
&gt; 发送时间: 2025-06-05 22:02:35 (星期四)
&gt; 收件人: chenglingfei <chenglingfei22s@ict.ac.cn>
&gt; 抄送: j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev, zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
&gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
&gt; 
&gt; Hi,
&gt; 
&gt; On 05.06.25 13:55, chenglingfei wrote:
&gt; &gt; 
&gt; &gt; 
&gt; &gt; 
&gt; &gt; &gt; -----原始邮件-----
&gt; &gt; &gt; 发件人: "Sven Peter" <sven@kernel.org>
&gt; &gt; &gt; 发送时间: 2025-06-05 18:25:09 (星期四)
&gt; &gt; &gt; 收件人: 程凌飞 <chenglingfei22s@ict.ac.cn>, j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev
&gt; &gt; &gt; 抄送: zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
&gt; &gt; &gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
&gt; &gt; &gt;
&gt; &gt; &gt; Hi,
&gt; &gt; &gt;
&gt; &gt; &gt; On 05.06.25 05:02, 程凌飞 wrote:
&gt; &gt; &gt; &gt; Hi, all!
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux.
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services.
&gt; &gt; &gt; &gt; At that time, the i2c-pasemi-platform was i2c-apple.
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
&gt; &gt; &gt; &gt; Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched
&gt; &gt; &gt; &gt; the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and
&gt; &gt; &gt; &gt; tested again with Linux v6.14.6, but the crash remains.
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware
&gt; &gt; &gt; &gt; design constraints, we can not record the panic logs through kdump.
&gt; &gt; &gt;
&gt; &gt; &gt; Do you have UART connected to a device under test which you could use to
&gt; &gt; &gt; grab the panic log from the kernel? Alternatively you can also run the
&gt; &gt; &gt; kernel under m1n1's hypervisor and grab the log that way. It'll emulate
&gt; &gt; &gt; the serial port and redirect its output via USB.
&gt; &gt; &gt;
&gt; &gt; 
&gt; &gt; I don't have UART, but I have tried to run the kernel under m1n1's hypervisor. However, it does not trigger the release of cs42l83.
&gt; &gt; Given that m1n1 provides full peripheral device emulation capability, the most plausible explanation would be an incorrect
&gt; &gt; firmware loading sequence. But the documentation of Asahi provides little details about how to generate an initramfs with
&gt; &gt; firmware (I think), can you give more guidance about it?
&gt; 
&gt; I'm not sure why you are even trying to create a special initramfs. Just
&gt; load your usual kernel using the usual boot flow as a guest. There's 
&gt; also no firmware involved in i2c and I'm not sure what you mean with 
&gt; "full peripheral device emulation" either or how that's related to firmware.
&gt; You also mention that the crash happens when you run rmmod so I again 
&gt; don't understand what "it does not trigger the release of cs42l83" means 
&gt; here.
&gt; 

Well, simply running rmmod i2c-pasemi-platform doesn't directly cause a crash. 
The crash occurs when the module removal triggers device_remove for cs42l83, 
which ultimately calls pasemi_smb_waitready in i2c-pasemi-platform. You may refer
to the brief analysis provided in my first email for more details.

When booting the kernel without m1n1, cs42l83 is automatically probed after 
i2c-pasemi-platform loads and subsequently removed when executing rmmod 
i2c-pasemi-platform, resulting in a kernel crash. However, when booting under m1n1,
cs42l83 isn't probed or removed -- the device appears to be non-existent. This 
observation led me to mention "full peripheral device emulation."

Furthermore, since cs42l83 remains untouched under m1n1, the chain of operations 
involving device_remove and the subsequent call to pasemi_smb_waitready never occurs.
This inherently prevents the crash scenario, which explains why I'm unable to reproduce
the crash when running under m1n1.

I can try again by 'loading your usual kernel using the usual boot flow as a guest,',
but I don't think it'll make much difference.

&gt; &gt; 
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; Thus we tried to find the root cause of the issue manually. When we perform rmmod, the kernel performs device releasing on
&gt; &gt; &gt; &gt; the i2c bus, then calls the remove function in snd-soc-cs42l83-i2c, which calls the cs42l42_common_remove in cs42l42,
&gt; &gt; &gt; &gt; because cs42l42-&gt;init_done is true, it performs regmap_write, and finally calls into pasemi_smb_waitready in i2c-pasemi
&gt; &gt; &gt; &gt; -core.c. We noticed that smbus-&gt;use_irq is true, and after it calls into wait_for_completion_timeout, the system crashs!&gt;
&gt; &gt; &gt; &gt; We found that wait_for_completion_timeout is one of the core scheduler APIs used by tens of thousands of other drivers,
&gt; &gt; &gt; &gt; it is unlikely causing the crash. So we tried to remove the call to wait_for_completion_timeout, then the system seems to
&gt; &gt; &gt; &gt; run well.
&gt; &gt; &gt; &gt;
&gt; &gt; &gt; &gt; However, because we have little knowledge about i2c devices and specifications, we are not sure whether this change will
&gt; &gt; &gt; &gt; cause other potential harms for the system and device. Is this call to wait necesary here? Or can you give a more
&gt; &gt; &gt; &gt; sophisticated fix?
&gt; &gt; &gt;
&gt; &gt; &gt; Yes, that call is necessary. It waits for the "transfer completed"
&gt; &gt; &gt; interrupt from the hardware. Without it the driver will try to read data
&gt; &gt; &gt; before it's available and you'll see corruption. I'm surprised hardware
&gt; &gt; &gt; attached to i2c (usb pd controller and audio I think) works at all with
&gt; &gt; &gt; that change.
&gt; &gt; &gt;
&gt; &gt; &gt;
&gt; &gt; &gt; Sven
&gt; &gt; 
&gt; &gt; Are there any methods or tools to systematically verify its functionality? I am not sure whether the devices attached to i2c
&gt; &gt; should work well even after the i2c-pasemi-platform has been removed.
&gt; 
&gt; I don't understand. You say you saw a crash inside pasemi_smb_waitready 
&gt; when calling wait_for_completion_timeout and decided to remove that 
&gt; method. When you remove the call you break the entire driver because it 
&gt; will now try to read data long before the i2c transaction has been 
&gt; completed.
&gt; Obviously, no i2c device will work when the driver isn't loaded but 
&gt; without waiting for the completion they also won't work when the driver 
&gt; is loaded.
&gt; 
&gt; 
&gt; Sven


</chenglingfei22s@ict.ac.cn></sven@kernel.org></chenglingfei22s@ict.ac.cn></sven@kernel.org>

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
  2025-06-05 15:13       ` chenglingfei
@ 2025-06-05 19:58         ` Sven Peter
  0 siblings, 0 replies; 6+ messages in thread
From: Sven Peter @ 2025-06-05 19:58 UTC (permalink / raw)
  To: chenglingfei
  Cc: j, alyssa, neal, zhangzhenwei22b, wangzhe12, maddy, mpe, npiggin,
	christophe.leroy, naveen, andi.shyti, asahi, linux-arm-kernel,
	linuxppc-dev, linux-i2c, linux-kernel

Hi,

On 05.06.25 17:13, chenglingfei wrote:
> 
> 
> 
> &gt; -----原始邮件-----
> &gt; 发件人: "Sven Peter" <sven@kernel.org>
> &gt; 发送时间: 2025-06-05 22:02:35 (星期四)
> &gt; 收件人: chenglingfei <chenglingfei22s@ict.ac.cn>
> &gt; 抄送: j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev, zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
> &gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
> &gt;
> &gt; Hi,
> &gt;
> &gt; On 05.06.25 13:55, chenglingfei wrote:
> &gt; &gt;
> &gt; &gt;
> &gt; &gt;
> &gt; &gt; &gt; -----原始邮件-----
> &gt; &gt; &gt; 发件人: "Sven Peter" <sven@kernel.org>
> &gt; &gt; &gt; 发送时间: 2025-06-05 18:25:09 (星期四)
> &gt; &gt; &gt; 收件人: 程凌飞 <chenglingfei22s@ict.ac.cn>, j@jannau.net, alyssa@rosenzweig.io, neal@gompa.dev
> &gt; &gt; &gt; 抄送: zhangzhenwei22b@ict.ac.cn, wangzhe12@ict.ac.cn, maddy@linux.ibm.com, mpe@ellerman.id.au, npiggin@gmail.com, christophe.leroy@csgroup.eu, naveen@kernel.org, andi.shyti@kernel.org, asahi@lists.linux.dev, linux-arm-kernel@lists.infradead.org, linuxppc-dev@lists.ozlabs.org, linux-i2c@vger.kernel.org, linux-kernel@vger.kernel.org
> &gt; &gt; &gt; 主题: Re: [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Hi,
> &gt; &gt; &gt;
> &gt; &gt; &gt; On 05.06.25 05:02, 程凌飞 wrote:
> &gt; &gt; &gt; &gt; Hi, all!
> &gt; &gt; &gt; &gt;
> &gt; &gt; &gt; &gt; We’ve encountered a kernel crash when running rmmod i2c-pasemi-platform on a Mac Mini M1 (T8103) running Asahi Arch Linux.
> &gt; &gt; &gt; &gt;
> &gt; &gt; &gt; &gt; The bug was first found on the Linux v6.6, which is built manually with the Asahi given config to run our services.
> &gt; &gt; &gt; &gt; At that time, the i2c-pasemi-platform was i2c-apple.
> &gt; &gt; &gt; &gt;
> &gt; &gt; &gt; &gt; We noticed in the Linux v6.7, the pasemi is splitted into two separate modules, one of which is i2c-pasemi-platform.
> &gt; &gt; &gt; &gt; Therefore, we built Linux v6.14.6 and tried to rmmod i2c-pasemi-platform again, the crash still exists. Moreover, we fetched
> &gt; &gt; &gt; &gt; the latest i2c-pasemi-platform on linux-next(911483b25612c8bc32a706ba940738cc43299496) and asahi, built them and
> &gt; &gt; &gt; &gt; tested again with Linux v6.14.6, but the crash remains.
> &gt; &gt; &gt; &gt;
> &gt; &gt; &gt; &gt; Because kexec is not supported and will never be fully supported on Apple Silicon platforms due to hardware and firmware
> &gt; &gt; &gt; &gt; design constraints, we can not record the panic logs through kdump.
> &gt; &gt; &gt;
> &gt; &gt; &gt; Do you have UART connected to a device under test which you could use to
> &gt; &gt; &gt; grab the panic log from the kernel? Alternatively you can also run the
> &gt; &gt; &gt; kernel under m1n1's hypervisor and grab the log that way. It'll emulate
> &gt; &gt; &gt; the serial port and redirect its output via USB.
> &gt; &gt; &gt;
> &gt; &gt;
> &gt; &gt; I don't have UART, but I have tried to run the kernel under m1n1's hypervisor. However, it does not trigger the release of cs42l83.
> &gt; &gt; Given that m1n1 provides full peripheral device emulation capability, the most plausible explanation would be an incorrect
> &gt; &gt; firmware loading sequence. But the documentation of Asahi provides little details about how to generate an initramfs with
> &gt; &gt; firmware (I think), can you give more guidance about it?
> &gt;
> &gt; I'm not sure why you are even trying to create a special initramfs. Just
> &gt; load your usual kernel using the usual boot flow as a guest. There's
> &gt; also no firmware involved in i2c and I'm not sure what you mean with
> &gt; "full peripheral device emulation" either or how that's related to firmware.
> &gt; You also mention that the crash happens when you run rmmod so I again
> &gt; don't understand what "it does not trigger the release of cs42l83" means
> &gt; here.
> &gt;
> 
> Well, simply running rmmod i2c-pasemi-platform doesn't directly cause a crash.
> The crash occurs when the module removal triggers device_remove for cs42l83,
> which ultimately calls pasemi_smb_waitready in i2c-pasemi-platform. You may refer
> to the brief analysis provided in my first email for more details.
> 
> When booting the kernel without m1n1, cs42l83 is automatically probed after
> i2c-pasemi-platform loads and subsequently removed when executing rmmod
> i2c-pasemi-platform, resulting in a kernel crash. However, when booting under m1n1,
> cs42l83 isn't probed or removed -- the device appears to be non-existent. This
> observation led me to mention "full peripheral device emulation."

I'm still not sure what "full peripheral device emulation" means in that 
context. If cs42l83 isn't probed that's most likely an issue with your 
kernel or your device tree or your boot device. Hence my suggestion to 
just the regular kernel and boot device.


Unrelated, there's something wrong with your email setup, see e.g. 
https://lore.kernel.org/asahi/6064d018.2b279.19740a7eb1c.Coremail.chenglingfei22s@ict.ac.cn/ 
how your mail arrive.

Sven




^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2025-06-05 20:01 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-06-05  3:02 [BUG] rmmod i2c-pasemi-platform causing kernel crash on Apple M1 程凌飞
2025-06-05 10:25 ` Sven Peter
2025-06-05 11:55   ` chenglingfei
2025-06-05 14:02     ` Sven Peter
2025-06-05 15:13       ` chenglingfei
2025-06-05 19:58         ` Sven Peter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).