Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Leonard Crestez]

On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> -	case IMX7D:
> +	case IMX8MQ:
> +		if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> +					 &imx6_pcie->gpr1x)) {
> +			dev_err(dev, "Failed to get GPR1x address\n");
> +			return -EINVAL;
> +		}

This is for distinguishing multiple controllers on the SOC but other
registers and bits might differ. Isn't it preferable to have a property
for controller id instead of adding many registers to DT?

> +
> +		if (of_property_read_u32_array(
> +			    node, "fsl,gpr12-device-type",
> +			    imx6_pcie->device_type,
> +			    ARRAY_SIZE(imx6_pcie->device_type))) {
> +			dev_err(dev, "Failed to get device type
> mask/value\n");
> +			return -EINVAL;
> +		}

The device type can be set on multiple SOCs, why are you adding a
mandatory property only for 8m?

There should probably be a separate patch with documented DT bindings.

--
Regards,
Leonard

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Andrey Smirnov]

On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>
> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> > @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> > -     case IMX7D:
> > +     case IMX8MQ:
> > +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> > +                                      &imx6_pcie->gpr1x)) {
> > +                     dev_err(dev, "Failed to get GPR1x address\n");
> > +                     return -EINVAL;
> > +             }
>
> This is for distinguishing multiple controllers on the SOC but other
> registers and bits might differ. Isn't it preferable to have a property
> for controller id instead of adding many registers to DT?
>

I liked encoding necessary info in DT directly slightly better than
encoding abstract ID and then decoding it further in the driver code.
OTOH, I am not really attached to that path. Lucas, can you comment on
this please?

> > +
> > +             if (of_property_read_u32_array(
> > +                         node, "fsl,gpr12-device-type",
> > +                         imx6_pcie->device_type,
> > +                         ARRAY_SIZE(imx6_pcie->device_type))) {
> > +                     dev_err(dev, "Failed to get device type
> > mask/value\n");
> > +                     return -EINVAL;
> > +             }
>
> The device type can be set on multiple SOCs, why are you adding a
> mandatory property only for 8m?

My thinking was that other SoCs don't really have two controllers, so
they don't really need to have that property, but, more importantly,
not forcing them to have this property should preserve backwards
compatibility with old DTBs.

>
> There should probably be a separate patch with documented DT bindings.
>

Yes, definitely, I just wanted to come up with a set of bindings
agreed on by the driver maintainers first.

Thanks,
Andrey Smirnov

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Lucas Stach]

Hi Andrey,

Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
> On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> > 
> > On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> > > @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> > > -     case IMX7D:
> > > +     case IMX8MQ:
> > > +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> > > +                                      &imx6_pcie->gpr1x)) {
> > > +                     dev_err(dev, "Failed to get GPR1x address\n");
> > > +                     return -EINVAL;
> > > +             }
> > 
> > This is for distinguishing multiple controllers on the SOC but other
> > registers and bits might differ. Isn't it preferable to have a property
> > for controller id instead of adding many registers to DT?
> > 
> 
> I liked encoding necessary info in DT directly slightly better than
> encoding abstract ID and then decoding it further in the driver code.
> OTOH, I am not really attached to that path. Lucas, can you comment on
> this please?

Yes, after rereading the patch with this in mind I agree that having
the GPR offset on DT directly is IMO the better approach than an
abstract ID.

One other thing I noticed is that we probably need some property to
encode if the clock is supplied by an external clkgen, or if the clock
is provided by the i.MX. Hardcoding this in the driver will lead to DT
backward compatibility headaches later on if someone decides to build a
board with the clock provided from the i.MX.

Regards,
Lucas

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Leonard Crestez]

On 11/26/18 8:24 PM, Andrey Smirnov wrote:
> On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:

>>> +             if (of_property_read_u32_array(
>>> +                         node, "fsl,gpr12-device-type",
>>> +                         imx6_pcie->device_type,
>>> +                         ARRAY_SIZE(imx6_pcie->device_type))) {
>>> +                     dev_err(dev, "Failed to get device type
>>> mask/value\n");
>>> +                     return -EINVAL;
>>> +             }
>>
>> The device type can be set on multiple SOCs, why are you adding a
>> mandatory property only for 8m?
> 
> My thinking was that other SoCs don't really have two controllers, so
> they don't really need to have that property, but, more importantly,
> not forcing them to have this property should preserve backwards
> compatibility with old DTBs.

The "device_type" registers determine Root Complex versus End Point 
mode. This is not yet supported on imx in upstream but it can be done on 
many soc versions.

Looking at other pcie controllers (and dwc core) this is normally done 
with a different compatible string with an "-ep" suffix. It feels wrong 
to expose bitmasks and values directly in DT instead.

For this patch you should probably just hardcode RC mode.

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Leonard Crestez]

On 11/27/18 12:06 PM, Lucas Stach wrote:
> Hi Andrey,
> 
> Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
>> On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>>>
>>> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
>>>> @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
>>>> -     case IMX7D:
>>>> +     case IMX8MQ:
>>>> +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
>>>> +                                      &imx6_pcie->gpr1x)) {
>>>> +                     dev_err(dev, "Failed to get GPR1x address\n");
>>>> +                     return -EINVAL;
>>>> +             }
>>>
>>> This is for distinguishing multiple controllers on the SOC but other
>>> registers and bits might differ. Isn't it preferable to have a property
>>> for controller id instead of adding many registers to DT?
>>
>> I liked encoding necessary info in DT directly slightly better than
>> encoding abstract ID and then decoding it further in the driver code.
>> OTOH, I am not really attached to that path. Lucas, can you comment on
>> this please?
> 
> Yes, after rereading the patch with this in mind I agree that having
> the GPR offset on DT directly is IMO the better approach than an
> abstract ID.

But it's not a single offset, for example the device_type (EP/RC) has 
bits for the two controllers side-by-side in GPR12.

Adding a "ctrl-id" property would fully describe the hardware from the 
start. If we add per-register information instead then when other 
registers are used we'll get DT compatibility headaches.

It's worth noting that 8qm/8qxp also have multiple controllers with a 
different set of bits placed differently in iomuxc.

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Andrey Smirnov]

On Tue, Nov 27, 2018 at 2:16 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>
> On 11/26/18 8:24 PM, Andrey Smirnov wrote:
> > On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> >> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
>
> >>> +             if (of_property_read_u32_array(
> >>> +                         node, "fsl,gpr12-device-type",
> >>> +                         imx6_pcie->device_type,
> >>> +                         ARRAY_SIZE(imx6_pcie->device_type))) {
> >>> +                     dev_err(dev, "Failed to get device type
> >>> mask/value\n");
> >>> +                     return -EINVAL;
> >>> +             }
> >>
> >> The device type can be set on multiple SOCs, why are you adding a
> >> mandatory property only for 8m?
> >
> > My thinking was that other SoCs don't really have two controllers, so
> > they don't really need to have that property, but, more importantly,
> > not forcing them to have this property should preserve backwards
> > compatibility with old DTBs.
>
> The "device_type" registers determine Root Complex versus End Point
> mode. This is not yet supported on imx in upstream but it can be done on
> many soc versions.
>

Yeah, I am aware, I wasn't really trying to make it possible to
configure IP block to be a EP, that is just an unavoidable consequence
of the approach taken.

> Looking at other pcie controllers (and dwc core) this is normally done
> with a different compatible string with an "-ep" suffix. It feels wrong
> to expose bitmasks and values directly in DT instead.
>
> For this patch you should probably just hardcode RC mode.

Hmm, given how the mask/value have to be different for each
controller, the only way to hardcode it  that I can think of would be
to change the property pass a bit offset instead of mask/value pair.
Is that what you are suggesting?

Thanks,
Andrey Smirnov

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Andrey Smirnov]

On Tue, Nov 27, 2018 at 2:46 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>
> On 11/27/18 12:06 PM, Lucas Stach wrote:
> > Hi Andrey,
> >
> > Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
> >> On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> >>>
> >>> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> >>>> @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> >>>> -     case IMX7D:
> >>>> +     case IMX8MQ:
> >>>> +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> >>>> +                                      &imx6_pcie->gpr1x)) {
> >>>> +                     dev_err(dev, "Failed to get GPR1x address\n");
> >>>> +                     return -EINVAL;
> >>>> +             }
> >>>
> >>> This is for distinguishing multiple controllers on the SOC but other
> >>> registers and bits might differ. Isn't it preferable to have a property
> >>> for controller id instead of adding many registers to DT?
> >>
> >> I liked encoding necessary info in DT directly slightly better than
> >> encoding abstract ID and then decoding it further in the driver code.
> >> OTOH, I am not really attached to that path. Lucas, can you comment on
> >> this please?
> >
> > Yes, after rereading the patch with this in mind I agree that having
> > the GPR offset on DT directly is IMO the better approach than an
> > abstract ID.
>
> But it's not a single offset, for example the device_type (EP/RC) has
> bits for the two controllers side-by-side in GPR12.
>

Playing devil's advocate for a bit:

More specifically, currently the following per-controller bits need to
be configured:

- Location of the "device type" field within GPR12
- GPR register to use to control PCIn_CLKREQ_B_OVERRIDE_EN and
PCIn_CLKREQ_B_OVERRIDE_EN (GPR14 vs GPR16)
- Now that Philip spoke against PCIE_CTRL_APPS_CLK_REQ being exposed
via reset controller driver, we need to know which SRC register to use
to control that bit (SRC_PCIEPHY_RCR vs. SRC_PCIE2_RCR)

Thanks,
Andrey Smirnov

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Leonard Crestez]

On 11/27/18 11:15 PM, Andrey Smirnov wrote:
> On Tue, Nov 27, 2018 at 2:46 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>> On 11/27/18 12:06 PM, Lucas Stach wrote:
>>> Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
>>>> On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
>>>>> On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
>>>>>> @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
>>>>>> -     case IMX7D:
>>>>>> +     case IMX8MQ:
>>>>>> +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
>>>>>> +                                      &imx6_pcie->gpr1x)) {
>>>>>> +                     dev_err(dev, "Failed to get GPR1x address\n");
>>>>>> +                     return -EINVAL;
>>>>>> +             }
>>>>>
>>>>> This is for distinguishing multiple controllers on the SOC but other
>>>>> registers and bits might differ. Isn't it preferable to have a property
>>>>> for controller id instead of adding many registers to DT?
>>>>
>>>> I liked encoding necessary info in DT directly slightly better than
>>>> encoding abstract ID and then decoding it further in the driver code.
>>>> OTOH, I am not really attached to that path. Lucas, can you comment on
>>>> this please?

>>> Yes, after rereading the patch with this in mind I agree that having
>>> the GPR offset on DT directly is IMO the better approach than an
>>> abstract ID.
>>
>> But it's not a single offset, for example the device_type (EP/RC) has
>> bits for the two controllers side-by-side in GPR12.
>>
> 
> Playing devil's advocate for a bit:
> 
> More specifically, currently the following per-controller bits need to
> be configured:
> 
> - Location of the "device type" field within GPR12
> - GPR register to use to control PCIn_CLKREQ_B_OVERRIDE_EN and
> PCIn_CLKREQ_B_OVERRIDE_EN (GPR14 vs GPR16)
> - Now that Philip spoke against PCIE_CTRL_APPS_CLK_REQ being exposed
> via reset controller driver, we need to know which SRC register to use
> to control that bit (SRC_PCIEPHY_RCR vs. SRC_PCIE2_RCR)

I looked a bit through bindings and there some instances of syscon-$BLH 
properties which include detailed offsets or bitmasks for $BLAH relative 
to the target syscon node.

If you're going the route of adding properties points to IOMUXC/SRC bits 
it would sense to ensure that they're also usable on other SOCs, 
otherwise you're just making 8mq more complicated. But that's hard.

But I think it's easier to deal with such SOC-specific details behind a 
compat string. Maybe the DT list has some opinion on this?

I wonder if of_alias_get_id would be a reasonable way to distinguish 
between pcie0 and pcie1 instead of adding an ctrl-id property?

--
Regards,
Leonard

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Lucas Stach]

Am Dienstag, den 27.11.2018, 13:14 -0800 schrieb Andrey Smirnov:
> On Tue, Nov 27, 2018 at 2:46 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> > 
> > On 11/27/18 12:06 PM, Lucas Stach wrote:
> > > Hi Andrey,
> > > 
> > > Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
> > > > On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> > > > > 
> > > > > On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> > > > > > @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> > > > > > -     case IMX7D:
> > > > > > +     case IMX8MQ:
> > > > > > +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> > > > > > +                                      &imx6_pcie->gpr1x)) {
> > > > > > +                     dev_err(dev, "Failed to get GPR1x address\n");
> > > > > > +                     return -EINVAL;
> > > > > > +             }
> > > > > 
> > > > > This is for distinguishing multiple controllers on the SOC but other
> > > > > registers and bits might differ. Isn't it preferable to have a property
> > > > > for controller id instead of adding many registers to DT?
> > > > 
> > > > I liked encoding necessary info in DT directly slightly better than
> > > > encoding abstract ID and then decoding it further in the driver code.
> > > > OTOH, I am not really attached to that path. Lucas, can you comment on
> > > > this please?
> > > 
> > > Yes, after rereading the patch with this in mind I agree that having
> > > the GPR offset on DT directly is IMO the better approach than an
> > > abstract ID.
> > 
> > But it's not a single offset, for example the device_type (EP/RC) has
> > bits for the two controllers side-by-side in GPR12.
> > 
> 
> Playing devil's advocate for a bit:
> 
> More specifically, currently the following per-controller bits need to
> be configured:
> 
> - Location of the "device type" field within GPR12
> - GPR register to use to control PCIn_CLKREQ_B_OVERRIDE_EN and
> PCIn_CLKREQ_B_OVERRIDE_EN (GPR14 vs GPR16)
> - Now that Philip spoke against PCIE_CTRL_APPS_CLK_REQ being exposed
> via reset controller driver, we need to know which SRC register to use
> to control that bit (SRC_PCIEPHY_RCR vs. SRC_PCIE2_RCR)

If if we need a lot of different GPR offsets, I revise my earlier
opinion. We certainly don't want lots of bit offsets and masks in the
DT. The whole GPR thing is ugly, but I would rather hide this in the
driver code, keyed by compatible and a controller id property, where we
can change things without any worries about DT backward compatibility.

Regards,
Lucas

Re: [PATCH 3/3] PCI: imx: Add support for i.MX8MQ

[Lucas Stach]

Am Mittwoch, den 28.11.2018, 10:55 +0000 schrieb Leonard Crestez:
> On 11/27/18 11:15 PM, Andrey Smirnov wrote:
> > On Tue, Nov 27, 2018 at 2:46 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> > > On 11/27/18 12:06 PM, Lucas Stach wrote:
> > > > Am Montag, den 26.11.2018, 10:24 -0800 schrieb Andrey Smirnov:
> > > > > On Tue, Nov 20, 2018 at 2:49 AM Leonard Crestez <leonard.crestez@nxp.com> wrote:
> > > > > > On Sat, 2018-11-17 at 10:12 -0800, Andrey Smirnov wrote:
> > > > > > > @@ -921,7 +1004,28 @@ static int imx6_pcie_probe(struct platform_device *pdev)
> > > > > > > -     case IMX7D:
> > > > > > > +     case IMX8MQ:
> > > > > > > +             if (of_property_read_u32(node, "fsl,iomux-gpr1x",
> > > > > > > +                                      &imx6_pcie->gpr1x)) {
> > > > > > > +                     dev_err(dev, "Failed to get GPR1x address\n");
> > > > > > > +                     return -EINVAL;
> > > > > > > +             }
> > > > > > 
> > > > > > This is for distinguishing multiple controllers on the SOC but other
> > > > > > registers and bits might differ. Isn't it preferable to have a property
> > > > > > for controller id instead of adding many registers to DT?
> > > > > 
> > > > > I liked encoding necessary info in DT directly slightly better than
> > > > > encoding abstract ID and then decoding it further in the driver code.
> > > > > OTOH, I am not really attached to that path. Lucas, can you comment on
> > > > > this please?
> > > > Yes, after rereading the patch with this in mind I agree that having
> > > > the GPR offset on DT directly is IMO the better approach than an
> > > > abstract ID.
> > > 
> > > But it's not a single offset, for example the device_type (EP/RC) has
> > > bits for the two controllers side-by-side in GPR12.
> > > 
> > 
> > Playing devil's advocate for a bit:
> > 
> > More specifically, currently the following per-controller bits need to
> > be configured:
> > 
> > - Location of the "device type" field within GPR12
> > - GPR register to use to control PCIn_CLKREQ_B_OVERRIDE_EN and
> > PCIn_CLKREQ_B_OVERRIDE_EN (GPR14 vs GPR16)
> > - Now that Philip spoke against PCIE_CTRL_APPS_CLK_REQ being exposed
> > via reset controller driver, we need to know which SRC register to use
> > to control that bit (SRC_PCIEPHY_RCR vs. SRC_PCIE2_RCR)
> 
> I looked a bit through bindings and there some instances of syscon-$BLH 
> properties which include detailed offsets or bitmasks for $BLAH relative 
> to the target syscon node.
> 
> If you're going the route of adding properties points to IOMUXC/SRC bits 
> it would sense to ensure that they're also usable on other SOCs, 
> otherwise you're just making 8mq more complicated. But that's hard.
> 
> But I think it's easier to deal with such SOC-specific details behind a 
> compat string. Maybe the DT list has some opinion on this?

I agree with this. The number of bits and offsets is too large to keep
it in DT properties, without running into backwards compat issues later
on.

> I wonder if of_alias_get_id would be a reasonable way to distinguish 
> between pcie0 and pcie1 instead of adding an ctrl-id property?

No, the alias is a arbitrary number that can change from board to
board. It can, but doesn't need to, correspond to any real hardware ID.
If we need a hardware controller ID, then there is no way around adding
a property to the PCIe controller node for this.

Regards,
Lucas