[REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.

Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ

Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs

Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.

Some relevant bits imo:

    • Sep 19 21:24:36 RudyUbuMbp kernel: hci_bcm4377 0000:73:00.1: can't disable ASPM; OS doesn't have ASPM control

    • Nov 14 01:29:28 RudyUbuMbp kernel: Call Trace:
    • Nov 14 01:29:28 RudyUbuMbp kernel: <TASK>
    • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack_lvl+0x48/0x70
    • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack+0x10/0x20
    • Nov 14 01:29:28 RudyUbuMbp kernel: __ubsan_handle_shift_out_of_bounds+0x156/0x310
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? ttwu_do_activate+0x80/0x290
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? raw_spin_rq_unlock+0x10/0x40
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? try_to_wake_up+0x292/0x6c0
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? sched_slice+0x76/0x140
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? reweight_entity+0x15c/0x170
    • Nov 14 01:29:28 RudyUbuMbp kernel: __reg_op.cold+0x14/0x38
    • Nov 14 01:29:28 RudyUbuMbp kernel: bitmap_release_region+0xe/0x20
    • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_handle_ack+0x8c/0x130 [hci_bcm4377]
    • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_poll_completion_ring+0x196/0x330 [hci_bcm4377]
    • Nov 14 01:29:28 RudyUbuMbp kernel: ? rcu_gp_kthread_wake+0x57/0x90
    • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_irq+0x77/0x140 [hci_bcm4377]
    • Nov 14 01:29:28 RudyUbuMbp kernel: __handle_irq_event_percpu+0x4c/0x1b0
    • Nov 14 01:29:28 RudyUbuMbp kernel: handle_irq_event+0x39/0x80
    • Nov 14 01:29:28 RudyUbuMbp kernel: handle_edge_irq+0x8c/0x250
    • Nov 14 01:29:28 RudyUbuMbp kernel: __common_interrupt+0x4f/0x110
    • Nov 14 01:29:28 RudyUbuMbp kernel: common_interrupt+0x45/0xb0
    • Nov 14 01:29:28 RudyUbuMbp kernel: asm_common_interrupt+0x27/0x40
    • Nov 14 01:29:28 RudyUbuMbp kernel: RIP: 0033:0x7f17b5fd8bca

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Bagas Sanjaya]

[-- Attachment #1: Type: text/plain, Size: 2588 bytes --]

On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
> 
> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
> 
> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
> 
> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.

Can you bisect this regression please?

> 
> Some relevant bits imo:
> 
>     • Sep 19 21:24:36 RudyUbuMbp kernel: hci_bcm4377 0000:73:00.1: can't disable ASPM; OS doesn't have ASPM control
> 
>     • Nov 14 01:29:28 RudyUbuMbp kernel: Call Trace:
>     • Nov 14 01:29:28 RudyUbuMbp kernel: <TASK>
>     • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack_lvl+0x48/0x70
>     • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack+0x10/0x20
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __ubsan_handle_shift_out_of_bounds+0x156/0x310
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? ttwu_do_activate+0x80/0x290
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? raw_spin_rq_unlock+0x10/0x40
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? try_to_wake_up+0x292/0x6c0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? sched_slice+0x76/0x140
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? reweight_entity+0x15c/0x170
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __reg_op.cold+0x14/0x38
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bitmap_release_region+0xe/0x20
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_handle_ack+0x8c/0x130 [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_poll_completion_ring+0x196/0x330 [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? rcu_gp_kthread_wake+0x57/0x90
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_irq+0x77/0x140 [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __handle_irq_event_percpu+0x4c/0x1b0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: handle_irq_event+0x39/0x80
>     • Nov 14 01:29:28 RudyUbuMbp kernel: handle_edge_irq+0x8c/0x250
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __common_interrupt+0x4f/0x110
>     • Nov 14 01:29:28 RudyUbuMbp kernel: common_interrupt+0x45/0xb0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: asm_common_interrupt+0x27/0x40
>     • Nov 14 01:29:28 RudyUbuMbp kernel: RIP: 0033:0x7f17b5fd8bca

Anyway, thanks for the regression report. I'm adding it to regzbot:

#regzbot ^introduced: v6.4..v6.5

-- 
An old man doll... just what I always wanted! - Clara

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 228 bytes --]

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
> 
> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>> 
>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>> 
>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>> 
>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
> 
> Can you bisect this regression please?

Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Hector Martin]

On 14/11/2023 15.23, Aditya Garg wrote:
> 
> 
>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>
>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>
>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>
>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>
>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>
>> Can you bisect this regression please?
> 
> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.

It's not a regression, it was always broken. I'm sending a patch.

- Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Hector Martin]

On 14/11/2023 15.59, Hector Martin wrote:
> On 14/11/2023 15.23, Aditya Garg wrote:
>>
>>
>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>
>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>
>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>
>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>
>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>
>>> Can you bisect this regression please?
>>
>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
> 
> It's not a regression, it was always broken. I'm sending a patch.
> 
> - Hector

You are quite likely conflating two problems. The ubsan issue you quoted
was always there and the patch I just sent fixes it, but it almost
certainly always worked fine in practice without ubsan.

The Bluetooth problem you are referring to is likely *specific to
Bluetooth LE devices* and the regression was introduced by 288c90224e
and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.

If Bluetooth is broken in *some other way* in 6.6.1 then we need a
proper report or a bisect. Your logs don't show any issues other than
the ubsan noise, which is not a regression.

- Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
> 
> On 14/11/2023 15.59, Hector Martin wrote:
>> On 14/11/2023 15.23, Aditya Garg wrote:
>>> 
>>> 
>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>> 
>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>> 
>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>> 
>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>> 
>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>> 
>>>> Can you bisect this regression please?
>>> 
>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>> 
>> It's not a regression, it was always broken. I'm sending a patch.
>> 
>> - Hector
> 
> You are quite likely conflating two problems. The ubsan issue you quoted
> was always there and the patch I just sent fixes it, but it almost
> certainly always worked fine in practice without ubsan.
> 
> The Bluetooth problem you are referring to is likely *specific to
> Bluetooth LE devices* and the regression was introduced by 288c90224e
> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
> 
> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
> proper report or a bisect. Your logs don't show any issues other than
> the ubsan noise, which is not a regression.
> 
> - Hector
I’ll request users with the hardware to test the patch you sent. Talking about
LE devices, I guess I can go ahead testing with the quirk if your patch doesn't
Work.
> 

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
> 
> On 14/11/2023 15.59, Hector Martin wrote:
>> On 14/11/2023 15.23, Aditya Garg wrote:
>>> 
>>> 
>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>> 
>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>> 
>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>> 
>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>> 
>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>> 
>>>> Can you bisect this regression please?
>>> 
>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>> 
>> It's not a regression, it was always broken. I'm sending a patch.
>> 
>> - Hector
> 
> You are quite likely conflating two problems. The ubsan issue you quoted
> was always there and the patch I just sent fixes it, but it almost
> certainly always worked fine in practice without ubsan.
> 
> The Bluetooth problem you are referring to is likely *specific to
> Bluetooth LE devices* and the regression was introduced by 288c90224e
> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
> 
> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
> proper report or a bisect. Your logs don't show any issues other than
> the ubsan noise, which is not a regression.
> 
> - Hector
> 

UBSAN noise seems to be fixed, Bluetooth not working though

https://pastebin.com/HeVvMVk4

Ill try setting .broken_le_coded = true,

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Hector Martin]

On 14/11/2023 18.03, Aditya Garg wrote:
> 
> 
>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>
>> On 14/11/2023 15.59, Hector Martin wrote:
>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>
>>>>
>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>
>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>
>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>
>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>
>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>
>>>>> Can you bisect this regression please?
>>>>
>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>
>>> It's not a regression, it was always broken. I'm sending a patch.
>>>
>>> - Hector
>>
>> You are quite likely conflating two problems. The ubsan issue you quoted
>> was always there and the patch I just sent fixes it, but it almost
>> certainly always worked fine in practice without ubsan.
>>
>> The Bluetooth problem you are referring to is likely *specific to
>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>
>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>> proper report or a bisect. Your logs don't show any issues other than
>> the ubsan noise, which is not a regression.
>>
>> - Hector
>>
> 
> UBSAN noise seems to be fixed, Bluetooth not working though
> 
> https://pastebin.com/HeVvMVk4
> 
> Ill try setting .broken_le_coded = true,

Now you have a probe timeout, which you didn't have before. That's a
different problem.

Please try this commit and see if it helps:

https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686

If it's this then it's still not a regression, it's probably just random
chance since I think the old timeout value was borderline for the older
chips.

- Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 14-Nov-2023, at 3:15 PM, Hector Martin <marcan@marcan.st> wrote:
> 
> On 14/11/2023 18.03, Aditya Garg wrote:
>> 
>> 
>>>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>> 
>>> On 14/11/2023 15.59, Hector Martin wrote:
>>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>> 
>>>>> 
>>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>> 
>>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>> 
>>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>> 
>>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>> 
>>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>> 
>>>>>> Can you bisect this regression please?
>>>>> 
>>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>> 
>>>> It's not a regression, it was always broken. I'm sending a patch.
>>>> 
>>>> - Hector
>>> 
>>> You are quite likely conflating two problems. The ubsan issue you quoted
>>> was always there and the patch I just sent fixes it, but it almost
>>> certainly always worked fine in practice without ubsan.
>>> 
>>> The Bluetooth problem you are referring to is likely *specific to
>>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>> 
>>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>>> proper report or a bisect. Your logs don't show any issues other than
>>> the ubsan noise, which is not a regression.
>>> 
>>> - Hector
>>> 
>> 
>> UBSAN noise seems to be fixed, Bluetooth not working though
>> 
>> https://pastebin.com/HeVvMVk4
>> 
>> Ill try setting .broken_le_coded = true,
> 
> Now you have a probe timeout, which you didn't have before. That's a
> different problem.
> 
> Please try this commit and see if it helps:
> 
> https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686
> 
> If it's this then it's still not a regression, it's probably just random
> chance since I think the old timeout value was borderline for the older
> chips.
> 
> - Hector
> 

Alright, I'll get it tested.

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 14-Nov-2023, at 3:14 PM, Hector Martin <marcan@marcan.st> wrote:
> 
> On 14/11/2023 18.03, Aditya Garg wrote:
>> 
>> 
>>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>> 
>>> On 14/11/2023 15.59, Hector Martin wrote:
>>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>> 
>>>>> 
>>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>> 
>>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>> 
>>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>> 
>>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>> 
>>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>> 
>>>>>> Can you bisect this regression please?
>>>>> 
>>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>> 
>>>> It's not a regression, it was always broken. I'm sending a patch.
>>>> 
>>>> - Hector
>>> 
>>> You are quite likely conflating two problems. The ubsan issue you quoted
>>> was always there and the patch I just sent fixes it, but it almost
>>> certainly always worked fine in practice without ubsan.
>>> 
>>> The Bluetooth problem you are referring to is likely *specific to
>>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>> 
>>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>>> proper report or a bisect. Your logs don't show any issues other than
>>> the ubsan noise, which is not a regression.
>>> 
>>> - Hector
>>> 
>> 
>> UBSAN noise seems to be fixed, Bluetooth not working though
>> 
>> https://pastebin.com/HeVvMVk4
>> 
>> Ill try setting .broken_le_coded = true,
> 
> Now you have a probe timeout, which you didn't have before. That's a
> different problem.
> 
> Please try this commit and see if it helps:
> 
> https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686
> 
> If it's this then it's still not a regression, it's probably just random
> chance since I think the old timeout value was borderline for the older
> chips.
> 
> - Hector
> 


Hi

I recently got a kernel tested with this patch as well as with .broken_le_coded = true,
Here are the logs: https://pastebin.com/BpfJuJKY

Also, without .broken_le_coded = true, the bluetooth doesn't work, as specified in my previous email.

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Hector Martin]

On 2023/11/19 4:31, Aditya Garg wrote:
> 
> 
>> On 14-Nov-2023, at 3:14 PM, Hector Martin <marcan@marcan.st> wrote:
>>
>> On 14/11/2023 18.03, Aditya Garg wrote:
>>>
>>>
>>>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>>>
>>>> On 14/11/2023 15.59, Hector Martin wrote:
>>>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>>>
>>>>>>
>>>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>>>
>>>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>>>
>>>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>>>
>>>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>>>
>>>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>>>
>>>>>>> Can you bisect this regression please?
>>>>>>
>>>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>>>
>>>>> It's not a regression, it was always broken. I'm sending a patch.
>>>>>
>>>>> - Hector
>>>>
>>>> You are quite likely conflating two problems. The ubsan issue you quoted
>>>> was always there and the patch I just sent fixes it, but it almost
>>>> certainly always worked fine in practice without ubsan.
>>>>
>>>> The Bluetooth problem you are referring to is likely *specific to
>>>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>>>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>>>
>>>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>>>> proper report or a bisect. Your logs don't show any issues other than
>>>> the ubsan noise, which is not a regression.
>>>>
>>>> - Hector
>>>>
>>>
>>> UBSAN noise seems to be fixed, Bluetooth not working though
>>>
>>> https://pastebin.com/HeVvMVk4
>>>
>>> Ill try setting .broken_le_coded = true,
>>
>> Now you have a probe timeout, which you didn't have before. That's a
>> different problem.
>>
>> Please try this commit and see if it helps:
>>
>> https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686
>>
>> If it's this then it's still not a regression, it's probably just random
>> chance since I think the old timeout value was borderline for the older
>> chips.
>>
>> - Hector
>>
> 
> 
> Hi
> 
> I recently got a kernel tested with this patch as well as with .broken_le_coded = true,
> Here are the logs: https://pastebin.com/BpfJuJKY
> 
> Also, without .broken_le_coded = true, the bluetooth doesn't work, as specified in my previous email.

So are you saying everything works now? If not, what doesn't work?
"Bluetooth doesn't work" isn't useful information, especially in the
absence of any useful error messages. You can't just dump dmesg logs at
us, you have to *describe* what the problem is.

If broken_le_coded = true "fixed" it then "bluetooth doesn't work" was a
terrible bug report. What that quirk does is make *connecting/pairing to
Bluetooth LE devices* work. Non-BLE devices already worked, the
controller worked, scanning worked, etc. All that is useful information
if you want to get support for issues. We can't magically divine what's
wrong if you just send us a dmesg and say "it's broken". We need
detailed information about exactly what works and what doesn't (e.g. the
controller not showing up at all is VERY different from it showing up
but not finding your device). The only reason we guessed this here is
that this was a known issue that affected other chips. If we ever run
into a 4377-specific issue that only you can reproduce, "bluetooth
doesn't work" and no error logs really isn't going to get it fixed.

- Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Aditya Garg]

> On 20-Nov-2023, at 4:37 PM, Hector Martin <marcan@marcan.st> wrote:
> 
> 
> 
>> On 2023/11/19 4:31, Aditya Garg wrote:
>> 
>> 
>>>> On 14-Nov-2023, at 3:14 PM, Hector Martin <marcan@marcan.st> wrote:
>>> 
>>> On 14/11/2023 18.03, Aditya Garg wrote:
>>>> 
>>>> 
>>>>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>>>> 
>>>>> On 14/11/2023 15.59, Hector Martin wrote:
>>>>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>>>> 
>>>>>>> 
>>>>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>>>> 
>>>>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>>>> 
>>>>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>>>> 
>>>>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>>>> 
>>>>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>>>> 
>>>>>>>> Can you bisect this regression please?
>>>>>>> 
>>>>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>>>> 
>>>>>> It's not a regression, it was always broken. I'm sending a patch.
>>>>>> 
>>>>>> - Hector
>>>>> 
>>>>> You are quite likely conflating two problems. The ubsan issue you quoted
>>>>> was always there and the patch I just sent fixes it, but it almost
>>>>> certainly always worked fine in practice without ubsan.
>>>>> 
>>>>> The Bluetooth problem you are referring to is likely *specific to
>>>>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>>>>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>>>> 
>>>>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>>>>> proper report or a bisect. Your logs don't show any issues other than
>>>>> the ubsan noise, which is not a regression.
>>>>> 
>>>>> - Hector
>>>>> 
>>>> 
>>>> UBSAN noise seems to be fixed, Bluetooth not working though
>>>> 
>>>> https://pastebin.com/HeVvMVk4
>>>> 
>>>> Ill try setting .broken_le_coded = true,
>>> 
>>> Now you have a probe timeout, which you didn't have before. That's a
>>> different problem.
>>> 
>>> Please try this commit and see if it helps:
>>> 
>>> https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686
>>> 
>>> If it's this then it's still not a regression, it's probably just random
>>> chance since I think the old timeout value was borderline for the older
>>> chips.
>>> 
>>> - Hector
>>> 
>> 
>> 
>> Hi
>> 
>> I recently got a kernel tested with this patch as well as with .broken_le_coded = true,
>> Here are the logs: https://pastebin.com/BpfJuJKY
>> 
>> Also, without .broken_le_coded = true, the bluetooth doesn't work, as specified in my previous email.
> 
> So are you saying everything works now? If not, what doesn't work?
> "Bluetooth doesn't work" isn't useful information, especially in the
> absence of any useful error messages. You can't just dump dmesg logs at
> us, you have to *describe* what the problem is.
> 
My bad for not specifying that. The user reports that the Bluetooth device is not recognised at all.

Also, broken_le_coded = true did not "fix" it.

Talking about dmesg, well not getting any logs regarding this message indeed is a frustrating thing for me, and bisecting seems to be the only option in my mind rn.

> If broken_le_coded = true "fixed" it then "bluetooth doesn't work" was a
> terrible bug report. What that quirk does is make *connecting/pairing to
> Bluetooth LE devices* work. Non-BLE devices already worked, the
> controller worked, scanning worked, etc. All that is useful information
> if you want to get support for issues. We can't magically divine what's
> wrong if you just send us a dmesg and say "it's broken". We need
> detailed information about exactly what works and what doesn't (e.g. the
> controller not showing up at all is VERY different from it showing up
> but not finding your device). The only reason we guessed this here is
> that this was a known issue that affected other chips. If we ever run
> into a 4377-specific issue that only you can reproduce, "bluetooth
> doesn't work" and no error logs really isn't going to get it fixed.
> 
> - Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Hector Martin]

On 2023/11/21 20:42, Aditya Garg wrote:
> 
> 
>> On 20-Nov-2023, at 4:37 PM, Hector Martin <marcan@marcan.st> wrote:
>>
>> 
>>
>>> On 2023/11/19 4:31, Aditya Garg wrote:
>>>
>>>
>>>>> On 14-Nov-2023, at 3:14 PM, Hector Martin <marcan@marcan.st> wrote:
>>>>
>>>> On 14/11/2023 18.03, Aditya Garg wrote:
>>>>>
>>>>>
>>>>>> On 14-Nov-2023, at 1:28 PM, Hector Martin <marcan@marcan.st> wrote:
>>>>>>
>>>>>> On 14/11/2023 15.59, Hector Martin wrote:
>>>>>>> On 14/11/2023 15.23, Aditya Garg wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>> On 14-Nov-2023, at 5:01 AM, Bagas Sanjaya <bagasdotme@gmail.com> wrote:
>>>>>>>>>
>>>>>>>>> On Mon, Nov 13, 2023 at 08:57:35PM +0000, Aditya Garg wrote:
>>>>>>>>>> Starting from kernel 6.5, a regression in the kernel is causing Bluetooth to not work on T2 Macs with BCM4377 chip.
>>>>>>>>>>
>>>>>>>>>> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: https://pastebin.com/u9U3kbFJ
>>>>>>>>>>
>>>>>>>>>> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: https://pastebin.com/aVHNFMRs
>>>>>>>>>>
>>>>>>>>>> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
>>>>>>>>>
>>>>>>>>> Can you bisect this regression please?
>>>>>>>>
>>>>>>>> Since I don't have access to this hardware, it's not possible for me to bisect this regression. Let's hope someone is able to do so though.
>>>>>>>
>>>>>>> It's not a regression, it was always broken. I'm sending a patch.
>>>>>>>
>>>>>>> - Hector
>>>>>>
>>>>>> You are quite likely conflating two problems. The ubsan issue you quoted
>>>>>> was always there and the patch I just sent fixes it, but it almost
>>>>>> certainly always worked fine in practice without ubsan.
>>>>>>
>>>>>> The Bluetooth problem you are referring to is likely *specific to
>>>>>> Bluetooth LE devices* and the regression was introduced by 288c90224e
>>>>>> and fixed by 41e9cdea9c, which is also in 6.5.11 and 6.6.1.
>>>>>>
>>>>>> If Bluetooth is broken in *some other way* in 6.6.1 then we need a
>>>>>> proper report or a bisect. Your logs don't show any issues other than
>>>>>> the ubsan noise, which is not a regression.
>>>>>>
>>>>>> - Hector
>>>>>>
>>>>>
>>>>> UBSAN noise seems to be fixed, Bluetooth not working though
>>>>>
>>>>> https://pastebin.com/HeVvMVk4
>>>>>
>>>>> Ill try setting .broken_le_coded = true,
>>>>
>>>> Now you have a probe timeout, which you didn't have before. That's a
>>>> different problem.
>>>>
>>>> Please try this commit and see if it helps:
>>>>
>>>> https://github.com/AsahiLinux/linux/commit/8ec770b4f78fc14629705206e2db54d9d6439686
>>>>
>>>> If it's this then it's still not a regression, it's probably just random
>>>> chance since I think the old timeout value was borderline for the older
>>>> chips.
>>>>
>>>> - Hector
>>>>
>>>
>>>
>>> Hi
>>>
>>> I recently got a kernel tested with this patch as well as with .broken_le_coded = true,
>>> Here are the logs: https://pastebin.com/BpfJuJKY
>>>
>>> Also, without .broken_le_coded = true, the bluetooth doesn't work, as specified in my previous email.
>>
>> So are you saying everything works now? If not, what doesn't work?
>> "Bluetooth doesn't work" isn't useful information, especially in the
>> absence of any useful error messages. You can't just dump dmesg logs at
>> us, you have to *describe* what the problem is.
>>
> My bad for not specifying that. The user reports that the Bluetooth device is not recognised at all.
> 
> Also, broken_le_coded = true did not "fix" it.
> 
> Talking about dmesg, well not getting any logs regarding this message indeed is a frustrating thing for me, and bisecting seems to be the only option in my mind rn.
> 

What does that mean? The controller or the device? Can you initiate a
scan? Does it find any devices? Is pairing broken or just connecting?
Was the device paired before or after the breaking change? *What* device
are you trying to use/pair?

What do these commands say?

$ bluetoothctl show
$ bluetoothctl devices

Since the kernel isn't logging any errors, have you looked at syslog to
see what the whole userspace side has to say? If an operation is failing
*something* should be logging an error.

*Please*, we need proper detailed bug reports if you want this to get
fixed. I'm sure you're aware of what a reasonable bug report is. Please
teach your user how to provide useful information. Otherwise yes, it's
going to be on you to bisect it since there's nothing we can do with
zero details and useful logs.

- Hector

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Janne Grunau]

Hej,

On Mon, Nov 13, 2023, at 21:57, Aditya Garg wrote:
> Starting from kernel 6.5, a regression in the kernel is causing 
> Bluetooth to not work on T2 Macs with BCM4377 chip.
>
> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: 
> https://pastebin.com/u9U3kbFJ
>
> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: 
> https://pastebin.com/aVHNFMRs
>
> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.

There could be a possible regression in kernel 6.4 and later affecting Bluetooth low energy on bcm4377 devices specifically.  288c90224eec ("Bluetooth: Enable all supported LE PHY by default") broke BLE connections on bcm4378 and bcm4387. I did not find any reports about broken BLE on bcm4377 so I marked only bcm4378 and bcm4387 as broken with Coded Phy in 41e9cdea9c4a ("Bluetooth: hci_bcm4377: Mark bcm4378/bcm4387 as BROKEN_LE_CODED")

If  BLE devices like Logitech keyboards/mice are affected (mostly during pairing) it would be worth testing the change below (against v6.7-rc or stable releases):

diff --git a/drivers/bluetooth/hci_bcm4377.c b/drivers/bluetooth/hci_bcm4377.c
index a61757835695..bc8398cde3c3 100644
--- a/drivers/bluetooth/hci_bcm4377.c
+++ b/drivers/bluetooth/hci_bcm4377.c
@@ -2453,6 +2453,7 @@ static const struct bcm4377_hw bcm4377_hw_variants[] = {
                .bar0_window2 = 0x1810c000,
                .disable_aspm = true,
                .broken_ext_scan = true,
+               .broken_le_coded = true,
                .send_ptb = bcm4377_send_ptb,
        },
 
This  should affect 6.4.8 as well so might be unrelated to the reported issue.

Janne

Re: [REGRESSION] Bluetooth is not working on Macs with BCM4377 chip starting from kernel 6.5

[Felix Zhang]

On 2023-11-13 15:57, Aditya Garg wrote:
> Starting from kernel 6.5, a regression in the kernel is causing 
> Bluetooth to not work on T2 Macs with BCM4377 chip.
> 
> Journalctl of kernel 6.4.8 which has Bluetooth working is given here: 
> https://pastebin.com/u9U3kbFJ
> 
> Journalctl of kernel 6.5.2, which has Bluetooth broken is given here: 
> https://pastebin.com/aVHNFMRs
> 
> Also, the bug hasn’t been fixed even in 6.6.1, as reported by users.
> 
> Some relevant bits imo:
> 
>     • Sep 19 21:24:36 RudyUbuMbp kernel: hci_bcm4377 0000:73:00.1: 
> can't disable ASPM; OS doesn't have ASPM control
> 
>     • Nov 14 01:29:28 RudyUbuMbp kernel: Call Trace:
>     • Nov 14 01:29:28 RudyUbuMbp kernel: <TASK>
>     • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack_lvl+0x48/0x70
>     • Nov 14 01:29:28 RudyUbuMbp kernel: dump_stack+0x10/0x20
>     • Nov 14 01:29:28 RudyUbuMbp kernel: 
> __ubsan_handle_shift_out_of_bounds+0x156/0x310
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? ttwu_do_activate+0x80/0x290
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? raw_spin_rq_unlock+0x10/0x40
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? try_to_wake_up+0x292/0x6c0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? sched_slice+0x76/0x140
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? reweight_entity+0x15c/0x170
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __reg_op.cold+0x14/0x38
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bitmap_release_region+0xe/0x20
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_handle_ack+0x8c/0x130 
> [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: 
> bcm4377_poll_completion_ring+0x196/0x330 [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: ? 
> rcu_gp_kthread_wake+0x57/0x90
>     • Nov 14 01:29:28 RudyUbuMbp kernel: bcm4377_irq+0x77/0x140 
> [hci_bcm4377]
>     • Nov 14 01:29:28 RudyUbuMbp kernel: 
> __handle_irq_event_percpu+0x4c/0x1b0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: handle_irq_event+0x39/0x80
>     • Nov 14 01:29:28 RudyUbuMbp kernel: handle_edge_irq+0x8c/0x250
>     • Nov 14 01:29:28 RudyUbuMbp kernel: __common_interrupt+0x4f/0x110
>     • Nov 14 01:29:28 RudyUbuMbp kernel: common_interrupt+0x45/0xb0
>     • Nov 14 01:29:28 RudyUbuMbp kernel: asm_common_interrupt+0x27/0x40
>     • Nov 14 01:29:28 RudyUbuMbp kernel: RIP: 0033:0x7f17b5fd8bca

It seems that the BCM4377 chip stopped working on my MacBookAir9,1 (and 
possibly other T2 devices with the BCM4377) after the logic in the 
Bluetooth core was changed in this commit: 
https://github.com/torvalds/linux/commit/6945795bc81ab7be22750ecfb365056688f2fada

Due to the HCI_QUIRK_USE_BDADDR_PROPERTY being always set in 
drivers/bluetooth/hci_bcm4377.c, the chip would be left unconfigured, as 
per the commit mentioned prior.  On the M1 Macs, it seems that the 
device would be configured in the devicetree.  However, that is not the 
case on T2 Macs.  In order to circumvent this issue, a flag can be added 
to prevent the bit from being set on the BCM4377, while setting it on 
the other devices.  I have not submitted a patch yet, but you can view 
and use the patch with the latest kernel source here: 
https://github.com/MrMan314/linux-t2-patches/blob/main/9004-bcm4377-fix-bdaddr-property.patch

I would also like to thank kekrby (https://github.com/kekrby) for 
helping me with the patch.