(no subject)

[Matthew Garrett]

Bcc: 
Subject: brcmsmac hangs machine if loaded after busmastering disabled
Reply-To: 

I'm dealing with a platform with some buggy system firmware that enables 
DMA on the wireless card and leaves it running even after OS handover, 
triggering memory corruption. I'm working around that by disabling bus 
mastering on the device in the bootloader, but that results in the 
system hanging during brcmsmac init. pci_set_master() is being called in 
the bcma code, so it's nothing that straghtforward. Any idea what might 
be going wrong here?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Arend van Spriel]

On 03/19/2012 10:51 PM, Matthew Garrett wrote:
>
> I'm dealing with a platform with some buggy system firmware that enables
> DMA on the wireless card and leaves it running even after OS handover,

 From what you showed me we are dealing with a MacBook Air using UEFI. 
As our meeting was brief I may have misunderstood, but am I correct to 
say the wireless card is active with UEFI mode to allow wireless network 
selection for netboot or something like that?

I also assume the proper fix should be to have the system firmware 
shutdown the wireless device properly before OS handover. My guess is 
that the device is still doing DMA transfers to some physical address 
space which is OK in UEFI context but the OS has assigned it for 
something else.

> triggering memory corruption. I'm working around that by disabling bus
> mastering on the device in the bootloader, but that results in the
> system hanging during brcmsmac init. pci_set_master() is being called in
> the bcma code, so it's nothing that straghtforward. Any idea what might
> be going wrong here?
>

You showed me a boot menu. So after selecting Fedora the system is 
moving to the grub bootloader first?

Are you disabling bus master or the entire wireless device? Could you 
try disabling the device in the bootloader, ie. pci_disable_device() or 
something equivalent.

Gr. AvS

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Matthew Garrett]

On Wed, Apr 11, 2012 at 01:45:45PM +0200, Arend van Spriel wrote:
> On 03/19/2012 10:51 PM, Matthew Garrett wrote:
> >
> >I'm dealing with a platform with some buggy system firmware that enables
> >DMA on the wireless card and leaves it running even after OS handover,
> 
> From what you showed me we are dealing with a MacBook Air using
> UEFI. As our meeting was brief I may have misunderstood, but am I
> correct to say the wireless card is active with UEFI mode to allow
> wireless network selection for netboot or something like that?

Yes.

> I also assume the proper fix should be to have the system firmware
> shutdown the wireless device properly before OS handover. My guess
> is that the device is still doing DMA transfers to some physical
> address space which is OK in UEFI context but the OS has assigned it
> for something else.

That seems to be exactly the problem.

> >triggering memory corruption. I'm working around that by disabling bus
> >mastering on the device in the bootloader, but that results in the
> >system hanging during brcmsmac init. pci_set_master() is being called in
> >the bcma code, so it's nothing that straghtforward. Any idea what might
> >be going wrong here?
> >
> 
> You showed me a boot menu. So after selecting Fedora the system is
> moving to the grub bootloader first?

Yup.

> Are you disabling bus master or the entire wireless device? Could
> you try disabling the device in the bootloader, ie.
> pci_disable_device() or something equivalent.

I'm just disabling the bus mastering. Do you expect there to be a 
behavioural difference in the core if I disable BAR decoding as well?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Arend van Spriel]

On 04/11/2012 01:51 PM, Matthew Garrett wrote:
> On Wed, Apr 11, 2012 at 01:45:45PM +0200, Arend van Spriel wrote:
>>
>> Are you disabling bus master or the entire wireless device? Could
>> you try disabling the device in the bootloader, ie.
>> pci_disable_device() or something equivalent.
>
> I'm just disabling the bus mastering. Do you expect there to be a
> behavioural difference in the core if I disable BAR decoding as well?
>

I discussed this with hardware designer and here is his quote "You would 
need to disable the DMA engines before disable PCIe master mode. If you 
didn't do that, and the DMA engines were active, the MAC would almost 
certainly hang."

So disabling the master mode solved your memory corruption, but the chip 
will hang. The DMA engines mentioned are the on-chip engines. As you 
indicated you can only use common PCI operations so I do not expect 
disabling BAR decoding will solve the hang in the chip.

Only as an experiment you could try to reset the device toggling the pci 
power state.

Gr. AvS

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Matthew Garrett]

On Wed, Apr 18, 2012 at 01:32:39PM +0200, Arend van Spriel wrote:
> I discussed this with hardware designer and here is his quote "You
> would need to disable the DMA engines before disable PCIe master
> mode. If you didn't do that, and the DMA engines were active, the
> MAC would almost certainly hang."
> 
> So disabling the master mode solved your memory corruption, but the
> chip will hang. The DMA engines mentioned are the on-chip engines.
> As you indicated you can only use common PCI operations so I do not
> expect disabling BAR decoding will solve the hang in the chip.

Yeah, I can get away with putting generic PCI operations in the 
bootloader, but a specific Broadcom driver is probably going a bit 
far...

> Only as an experiment you could try to reset the device toggling the
> pci power state.

Should putting the PCI device in D3 be sufficient for this, or would I 
also need to touch any state in the 802.11 core?

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Arend van Spriel]

On 04/18/2012 02:34 PM, Matthew Garrett wrote:
> On Wed, Apr 18, 2012 at 01:32:39PM +0200, Arend van Spriel wrote:
>> I discussed this with hardware designer and here is his quote "You
>> would need to disable the DMA engines before disable PCIe master
>> mode. If you didn't do that, and the DMA engines were active, the
>> MAC would almost certainly hang."
>>
>> So disabling the master mode solved your memory corruption, but the
>> chip will hang. The DMA engines mentioned are the on-chip engines.
>> As you indicated you can only use common PCI operations so I do not
>> expect disabling BAR decoding will solve the hang in the chip.
>
> Yeah, I can get away with putting generic PCI operations in the
> bootloader, but a specific Broadcom driver is probably going a bit
> far...
>
>> Only as an experiment you could try to reset the device toggling the
>> pci power state.
>
> Should putting the PCI device in D3 be sufficient for this, or would I
> also need to touch any state in the 802.11 core?
>

I am not sure, but D3 is a good guess. It is all a workaround as the 
real fix should be in Apple's UEFI implementation. I think it is 
unlikely we can make them do that although we probably supplied them the 
wireless part.

Gr. AvS

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Matthew Garrett]

On Wed, Apr 18, 2012 at 05:29:42PM +0200, Arend van Spriel wrote:

> I am not sure, but D3 is a good guess. It is all a workaround as the
> real fix should be in Apple's UEFI implementation. I think it is
> unlikely we can make them do that although we probably supplied them
> the wireless part.

Disabling busmastering and then D3ing the device once in the kernel 
didn't seem to work, but putting the device in D3 while in the firmware 
(and leaving busmastering enabled) seems to avoid the problem.

-- 
Matthew Garrett | mjg59@srcf.ucam.org

Re: brcmsmac hangs machine if loaded after busmastering disabled

[Arend van Spriel]

On 04/18/2012 05:37 PM, Matthew Garrett wrote:
> On Wed, Apr 18, 2012 at 05:29:42PM +0200, Arend van Spriel wrote:
>
>> I am not sure, but D3 is a good guess. It is all a workaround as the
>> real fix should be in Apple's UEFI implementation. I think it is
>> unlikely we can make them do that although we probably supplied them
>> the wireless part.
>
> Disabling busmastering and then D3ing the device once in the kernel
> didn't seem to work, but putting the device in D3 while in the firmware
> (and leaving busmastering enabled) seems to avoid the problem.
>

Yup, the master disable itself hangs the device. The device should be 
able to deal with going to D3 so you could try just doing that from the 
kernel.

Gr. AvS