Re: SR-IOV enabling process failed in kernel 2.6.32

Re: SR-IOV enabling process failed in kernel 2.6.32

[Greg KH]

On Tue, Dec 08, 2009 at 11:02:22AM +0200, Erez Shitrit wrote:
> Hi Greg,
>  
> We are working on InfiniBand device that supports SR-IOV.
> and lately (from version 2.6.32 ) we faced a problem:
> When the driver tried to enable SR-IOV it failed to allocate resources
> for the 2 BARS the device needs.

Hm, you do realize I'm no longer the PCI maintainer, and havn't been for
about 1 1/2 years, right?  :)

> The situation is when there are 2 BARS, the first asks for little amount
> of memory and the second asks for much larger memory requirements.
> I saw in the code (drivers/pci/iov.c) that the order of the allocation
> starts from the first BAR to the last
> The code:
> for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
>   res = iov->res + i;
>   if (!res->flags)
>       continue;
>   rc = pci_assign_resource(dev, i + PCI_IOV_RESOURCES);
> 
> This process failed for the second BAR (The big one). (in the log: "not
> enough MMIO resources for SR-IOV")
>  
> I think (IMHO) it because the allocation of the first BAR (the little)
> one was from the allocated slot of the big one (the allocation from the
> sriov_init function)
> and when the kernel tries to allocate the slot for he second BAR (the
> big one) in the place of the little one it failed.
>  
> when I changed the code to start the allocation from the last to the
> first: for (i = PCI_SRIOV_NUM_BARS; i >= 0; i--) 
> It works fine.
>  
> Is it a bug ? 

I don't know, care to make a patch that shows your exact change?

thanks,

greg k-h

RE: SR-IOV enabling process failed in kernel 2.6.32

[Erez Shitrit]

Hi Greg,
 
We are working on InfiniBand device that supports SR-IOV.
and lately (from version 2.6.32 ) we faced a problem:
When the driver tried to enable SR-IOV it failed to allocate resources
for the 2 BARS the device needs.
The situation is when there are 2 BARS, the first asks for little amount
of memory and the second asks for much larger memory requirements.
I saw in the code (drivers/pci/iov.c) that the order of the allocation
starts from the first BAR to the last.

The code:
for (i = 0; i < PCI_SRIOV_NUM_BARS; i++) {
  res = iov->res + i;
  if (!res->flags)
      continue;
  rc = pci_assign_resource(dev, i + PCI_IOV_RESOURCES);

This process failed for the second BAR (The big one). (in the log: "not
enough MMIO resources for SR-IOV")
 
I think (IMHO) it because the allocation of the first BAR (the little)
one was from the allocated slot of the big one (the allocation from the
sriov_init function)
and when the kernel tries to allocate the slot for he second BAR (the
big one) in the place of the little one it failed.
 
when I changed the code to start the allocation from the last to the
first: for (i = PCI_SRIOV_NUM_BARS; i >= 0; i--) 
It works fine.
 
Is it a bug ? 
 
Thanks, Erez