Opterons and setting the pci bus master bit

Opterons and setting the pci bus master bit

[Mark Hounschell]

Problem, I have a number of dual processor Opterons that do not work 
with pci expansion chassis'. The reason that they do not work is because 
none of the cards discovered in the chassis get the pci bus master bit 
set in their command register. If I manually set this bit in our cards 
everything is fine. When we connect the same chassis to an Intel P4 box 
everything is fine. It looks like it is the kernel that sets this bit 
because we have never set it in any of our drivers, yet on the intel 
boxes it gets set. Why would this bit not be set when the chassis is 
connected to an Opteron. We are running 32-bit mode BTW. I am using a 
2.6.11.9 kernel. Is this a motherboard problem or could this be a kernel 
problem?

Thanks
Mark

Re: Opterons and setting the pci bus master bit

[linux-os (Dick Johnson)]

On Fri, 30 Sep 2005, Mark Hounschell wrote:

> Problem, I have a number of dual processor Opterons that do not work
> with pci expansion chassis'. The reason that they do not work is because
> none of the cards discovered in the chassis get the pci bus master bit
> set in their command register. If I manually set this bit in our cards
> everything is fine. When we connect the same chassis to an Intel P4 box
> everything is fine. It looks like it is the kernel that sets this bit
> because we have never set it in any of our drivers, yet on the intel
> boxes it gets set. Why would this bit not be set when the chassis is
> connected to an Opteron. We are running 32-bit mode BTW. I am using a
> 2.6.11.9 kernel. Is this a motherboard problem or could this be a kernel
> problem?
>
> Thanks
> Mark

The PCI Busmaster bit should (read must) be set by the driver to enble
bus-mastering after the dma mask is set.

   Commands are (in order):

    pci_set_dma_mask(dev, 0xffffffffULL);    // 32 bit DMA only
    pci_set_drvdata(dev, NULL);              // Harmless if unused
    pci_set_power_state(dev, 0);             // Turn it ON
    pci_set_master(dev);                     // Make bus-master
    pci_set_mwi(dev);	// Check return, different code
    pci_write_config_dword(dev, PCI_COMMAND, PCI_CONFIG_YOU_DEFINE);

   Typical bus-master PCI_CONFIG_YOU_DEFINE is:
     (PCI_COMMAND_MEMORY|PCI_COMMAND_MASTER|PCI_COMMAND_SERR)



Cheers,
Dick Johnson
Penguin : Linux version 2.6.13 on an i686 machine (5589.55 BogoMips).
Warning : 98.36% of all statistics are fiction.

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Opterons and setting the pci bus master bit

[Alan Cox]

On Gwe, 2005-09-30 at 13:10 -0400, Mark Hounschell wrote:
> everything is fine. When we connect the same chassis to an Intel P4 box 
> everything is fine. It looks like it is the kernel that sets this bit 
> because we have never set it in any of our drivers, yet on the intel 
> boxes it gets set. Why would this bit not be set when the chassis is 
> connected to an Opteron. We are running 32-bit mode BTW. I am using a 
> 2.6.11.9 kernel. Is this a motherboard problem or could this be a kernel 
> problem?

If your device needs to bus master you need to call
pci_set_master(pci_dev). The bus mastering setup prior to that really
depends on the BIOS and phase of the moon.

See Documentation/pci.txt

Alan

Re: Opterons and setting the pci bus master bit

[Mark Hounschell]

Alan Cox wrote:
> On Gwe, 2005-09-30 at 13:10 -0400, Mark Hounschell wrote:
>>everything is fine. When we connect the same chassis to an Intel P4 box 
>>everything is fine. It looks like it is the kernel that sets this bit 
>>because we have never set it in any of our drivers, yet on the intel 
>>boxes it gets set. Why would this bit not be set when the chassis is 
>>connected to an Opteron. We are running 32-bit mode BTW. I am using a 
>>2.6.11.9 kernel. Is this a motherboard problem or could this be a kernel 
>>problem?
> 
> If your device needs to bus master you need to call
> pci_set_master(pci_dev). The bus mastering setup prior to that really
> depends on the BIOS and phase of the moon.
> 
> See Documentation/pci.txt
> 
> Alan
> 
> 

Ok, thats what I'll do then.

Thanks for the info.
Mark

Re: Opterons and setting the pci bus master bit

[Benjamin Herrenschmidt]

On Fri, 2005-09-30 at 13:32 -0400, linux-os (Dick Johnson) wrote:
>     pci_set_dma_mask(dev, 0xffffffffULL);    // 32 bit DMA only
>     pci_set_drvdata(dev, NULL);              // Harmless if unused
>     pci_set_power_state(dev, 0);             // Turn it ON
>     pci_set_master(dev);                     // Make bus-master
>     pci_set_mwi(dev);   // Check return, different code
>     pci_write_config_dword(dev, PCI_COMMAND, PCI_CONFIG_YOU_DEFINE);
> 
>    Typical bus-master PCI_CONFIG_YOU_DEFINE is:
>      (PCI_COMMAND_MEMORY|PCI_COMMAND_MASTER|PCI_COMMAND_SERR)

Actually the last one is slightly bogus for things like
memory/master/etc... you shouldn't bother about these if you do things
correctly and call ... pci_enable_device() which you forgot :)

Ben.