[PATCH v1 5/5] pci: keystone: add pcie driver based on designware core driver

Murali Karicheri
2014-05-19T21:10:50Z

On 5/19/2014 8:06 AM, Arnd Bergmann wrote:
> On Friday 16 May 2014 16:26:51 Murali Karicheri wrote:
>> On 5/15/2014 2:20 PM, Arnd Bergmann wrote:
>>> On Thursday 15 May 2014 13:45:08 Murali Karicheri wrote:
>>>>>> +#ifdef CONFIG_PCI_KEYSTONE
>>>>>> +/*
>>>>>> + * The KeyStone PCIe controller has maximum read request size of 256 bytes.
>>>>>> + */
>>>>>> +static void quirk_limit_readrequest(struct pci_dev *dev)
>>>>>> +{
>>>>>> + int readrq = pcie_get_readrq(dev);
>>>>>> +
>>>>>> + if (readrq > 256)
>>>>>> + pcie_set_readrq(dev, 256);
>>>>>> +}
>>>>>> +DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, quirk_limit_readrequest);
>>>>>> +#endif /* CONFIG_PCI_KEYSTONE */
>>>>> This doesn't work: you can't just limit do this for all devices just based
>>>>> on PCI_KEYSTONE being enabled, you have to check if you are actually using
>>>>> this controller.
>>>>>
>>>>> Arnd
>>>> I assume, I need to check if PCI controller's vendor ID/ device ID
>>>> match with the keystone
>>>> PCI controller's ID and call pcie_set_readrq() for all of the slave
>>>> PCI devices and do this fixup.
>>>> Is this correct understanding? If you can point me to an example code
>>>> for this that will be
>>>> really helpful so that I can avoid re-inventing the wheel.
>>> I think it would be best to move the quirk into the keystone pci driver
>>> and compare compare the dev->driver pointer of the PCI controller device.
>>>
>>> Arnd
>> Arnd,
>>
>> I will move this quirk to keystone pci driver. So in that case, I guess
>> your original comment
>> is not applicable as this quirk gets enabled only with PCI keystone
>> driver enabled. Right?
> Of course you still have to fix the bug, not just move the code into
> the driver. Otherwise it's still broken for every machine after the keystone
> driver is enabled.

Agree. I have tried following to get this work so that the quirk gets
applied only for
keystone pci controller.

#define KS_K2HK_PCI_DEVICE_ID 0xb008
#define KS_K2E_PCI_DEVICE_ID 0xb009
#define KS_K2L_PCI_DEVICE_ID 0xb00a

static u16 root_pci_ids[] =
{KS_K2HK_PCI_DEVICE_ID, KS_K2E_PCI_DEVICE_ID, KS_K2L_PCI_DEVICE_ID,
0 };
/*
* The KeyStone PCIe controller has maximum read request size of 256 bytes.
*/
static void quirk_limit_readrequest(struct pci_dev *dev)
{
struct pci_dev *root_dev;
int i = 0;

/* Apply quirk only if this bridge device is keystone */
while (root_pci_ids[i]) {
root_dev = pci_get_device(PCI_VENDOR_ID_TI, root_pci_ids[i], NULL);
if ((root_dev->class >> 8) == PCI_CLASS_BRIDGE_PCI) {
int readrq;

readrq = pcie_get_readrq(dev);
if (pcie_get_readrq(dev) > 256) {
pcie_set_readrq(dev, 256);
printk("Applied quirk\n");
}
break;
};
}
}
DECLARE_PCI_FIXUP_FINAL(PCI_ANY_ID, PCI_ANY_ID, quirk_limit_readrequest);

>
> I also agree with Jason that changing the PCI core to call
> pcie_bus_configure_settings() would be a better way of dealing with this
> if it solves the problem.

I tried following piece of code added to bios32.c

void pci_common_init_dev(struct device *parent, struct hw_pci *hw)
{
struct pci_sys_data *sys;
LIST_HEAD(head);

pci_add_flags(PCI_REASSIGN_ALL_RSRC);
if (hw->preinit)
hw->preinit();

........
....

list_for_each_entry(sys, &head, node) {
struct pci_bus *bus = sys->bus;

if (!pci_has_flag(PCI_PROBE_ONLY)) {
/*
* Size the bridge windows.
*/
pci_bus_size_bridges(bus);

/*
* Assign resources.
*/
pci_bus_assign_resources(bus);
}
/*
* Tell drivers about devices found.
*/
pci_bus_add_devices(bus);
}

// New code starts here
list_for_each_entry(sys, &head, node) {
struct pci_bus *bus = sys->bus;

/* Configure PCI Express settings */
if (bus && !pci_has_flag(PCI_PROBE_ONLY)) {
struct pci_bus *child;

list_for_each_entry(child, &bus->children, node)
pcie_bus_configure_settings(child);
}
}
// New code ends.

This seems to do different things based on the pci bootargs. The key
requirement for Keystone PCI controller
is that the MRRS can't be more than 256 bytes. The only option that
works for keystone PCI controller
is with pci=pcie_bus_perf. I see following logs:-

[ 3.382747] pcie_bus_configure_settings, config 2
[ 3.382753] pcie_bus_configure_set
[ 3.382781] pcieport 0000:00:00.0: Max Payload Size set to 256/ 256
(was 128), Max Read Rq 256
[ 3.382788] pcie_bus_configure_set
[ 3.382846] pci 0000:01:00.0: Max Payload Size set to 256/ 256 (was
128), Max Read Rq 256
[ 3.382852] pcie_bus_configure_set
[ 3.382909] pci 0000:01:00.1: Max Payload Size set to 256/ 256 (was
128), Max Read Rq 256

On ARM, by default pci_bus_config seems to be set to 0
(PCIE_BUS_TUNE_OFF). So the code doesn't get
executed for this default. But for PCIE_BUS_SAFE, it doesn't change the
mrrs at the EP and is not
good for our platform w.r.t mrrs settings. For PCIE_BUS_PERFORMANCE, it
seems to increase the payload
size as well and Keystone Payload size is limited to 128 bytes. So it is
not safe to increase the payload
size to 256 based on the log.

On other platforms, Why the PCI core try to set the payload size equal
to mrrs? Is this explained in any
PCI spec? Looks like this is done for performance? Let me know if you
want me to send a patch for
review to add the pcie_bus_configure_settings() code to
arch/arm/kernel/bios32.c

For the Keystone PCI driver, I believe. it is safe to have the quirk so
that controller can handle the
read requests properly. Let me know if the quirk code above looks good
to go.

Murali

> Arnd