msm8909 support in a recent kernel

msm8909 support in a recent kernel

[Will Newton]

Hi all,

I'm looking at building a product using an msm8909 SoC and been
somewhat disappointed with the choice of kernels available - Qualcomm
I believe are only shipping 3.10, AOSP has some code on 3.18 but both
of these kernel releases are end of life.

I was looking at the work done to support the Dragonboard on the basis
that the msm8916 doesn't look that dissimilar to the msm8909 in terms
of the peripherals on the SoC and I was considering forward porting
the msm8909 support from 3.10/3.18 to 4.9 (probably Linaro qcomlt
tree). Does anyone have a feel for how much work this would be? Would
I be completely insane to attempt that?

Thanks,

Re: msm8909 support in a recent kernel

[Stephen Boyd]

On 11/28/2017 08:47 AM, Will Newton wrote:
> Hi all,
>
> I'm looking at building a product using an msm8909 SoC and been
> somewhat disappointed with the choice of kernels available - Qualcomm
> I believe are only shipping 3.10, AOSP has some code on 3.18 but both
> of these kernel releases are end of life.
>
> I was looking at the work done to support the Dragonboard on the basis
> that the msm8916 doesn't look that dissimilar to the msm8909 in terms
> of the peripherals on the SoC and I was considering forward porting
> the msm8909 support from 3.10/3.18 to 4.9 (probably Linaro qcomlt
> tree). Does anyone have a feel for how much work this would be? Would
> I be completely insane to attempt that?
>

It can take quite some time to do a git rebase of the 3.10 or 3.18 tree
to a newer kernel version. Even if you have knowledge of the SoC and
have been working in the kernel for quite a while it can still take
weeks if not months to move all the code over and have it all work
correctly. Copying whole drivers is not really the issue. It's teasing
out the core framework changes and porting those over that takes time.

It's not completely insane to support this SoC upstream though. You'd
have to bring in the pinctrl and clk drivers, which may be a bit of
effort. After that it should mostly be enabling various devices by
adding DT nodes and testing things out. It looks like this is 32-bit, so
getting SMP support may require some tweaks to the smp_ops code for qcom
platforms. You're right that it isn't too different from msm8916, so it
may be that most of the driver support for that SoC transfers over
nicely to this one.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: msm8909 support in a recent kernel

[Will Newton]

On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:

Hi Stephen,

>> I'm looking at building a product using an msm8909 SoC and been
>> somewhat disappointed with the choice of kernels available - Qualcomm
>> I believe are only shipping 3.10, AOSP has some code on 3.18 but both
>> of these kernel releases are end of life.
>>
>> I was looking at the work done to support the Dragonboard on the basis
>> that the msm8916 doesn't look that dissimilar to the msm8909 in terms
>> of the peripherals on the SoC and I was considering forward porting
>> the msm8909 support from 3.10/3.18 to 4.9 (probably Linaro qcomlt
>> tree). Does anyone have a feel for how much work this would be? Would
>> I be completely insane to attempt that?
>>
>
> It can take quite some time to do a git rebase of the 3.10 or 3.18 tree
> to a newer kernel version. Even if you have knowledge of the SoC and
> have been working in the kernel for quite a while it can still take
> weeks if not months to move all the code over and have it all work
> correctly. Copying whole drivers is not really the issue. It's teasing
> out the core framework changes and porting those over that takes time.

Yeah that does seem to be where the main issues lie. Forward porting
the drivers is hard due to their reliance on a bunch of out of tree
code or modified generic code that has changed considerably in the
intervening years.

> It's not completely insane to support this SoC upstream though. You'd
> have to bring in the pinctrl and clk drivers, which may be a bit of
> effort. After that it should mostly be enabling various devices by
> adding DT nodes and testing things out. It looks like this is 32-bit, so
> getting SMP support may require some tweaks to the smp_ops code for qcom
> platforms. You're right that it isn't too different from msm8916, so it
> may be that most of the driver support for that SoC transfers over
> nicely to this one.

I've started from the 8916 drivers and started to port in the changes
from the 3.18 tree that seem relevant. I have a kernel that boots and
talks over the serial. I've done a bunch of pinctrl although it is not
complete yet. I've had a look at the clocks and got the PLL working
but I think I'm probably missing a document that describes the
clocking architecture in more detail (I have the register reference
but that's a bit of a worm's eye view).

The current issue I am experiencing is the first write to an SPMI
channel causes the board to reset. I suspect this means that I have
not setup clocks correctly somewhere?

Re: msm8909 support in a recent kernel

[Stephen Boyd]

On 12/01, Will Newton wrote:
> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> 
> > It's not completely insane to support this SoC upstream though. You'd
> > have to bring in the pinctrl and clk drivers, which may be a bit of
> > effort. After that it should mostly be enabling various devices by
> > adding DT nodes and testing things out. It looks like this is 32-bit, so
> > getting SMP support may require some tweaks to the smp_ops code for qcom
> > platforms. You're right that it isn't too different from msm8916, so it
> > may be that most of the driver support for that SoC transfers over
> > nicely to this one.
> 
> I've started from the 8916 drivers and started to port in the changes
> from the 3.18 tree that seem relevant. I have a kernel that boots and
> talks over the serial. I've done a bunch of pinctrl although it is not
> complete yet. I've had a look at the clocks and got the PLL working
> but I think I'm probably missing a document that describes the
> clocking architecture in more detail (I have the register reference
> but that's a bit of a worm's eye view).
> 
> The current issue I am experiencing is the first write to an SPMI
> channel causes the board to reset. I suspect this means that I have
> not setup clocks correctly somewhere?

The SPMI controller typically always has clks enabled, so I would
be surprised if the clk was off. More likely, you're attempting
to read/write a channel that is locked down and triggering an
access control violation. Something configured incorrectly in DT
perhaps?

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: msm8909 support in a recent kernel

[Will Newton]

On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> On 12/01, Will Newton wrote:
>> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>>
>> > It's not completely insane to support this SoC upstream though. You'd
>> > have to bring in the pinctrl and clk drivers, which may be a bit of
>> > effort. After that it should mostly be enabling various devices by
>> > adding DT nodes and testing things out. It looks like this is 32-bit, so
>> > getting SMP support may require some tweaks to the smp_ops code for qcom
>> > platforms. You're right that it isn't too different from msm8916, so it
>> > may be that most of the driver support for that SoC transfers over
>> > nicely to this one.
>>
>> I've started from the 8916 drivers and started to port in the changes
>> from the 3.18 tree that seem relevant. I have a kernel that boots and
>> talks over the serial. I've done a bunch of pinctrl although it is not
>> complete yet. I've had a look at the clocks and got the PLL working
>> but I think I'm probably missing a document that describes the
>> clocking architecture in more detail (I have the register reference
>> but that's a bit of a worm's eye view).
>>
>> The current issue I am experiencing is the first write to an SPMI
>> channel causes the board to reset. I suspect this means that I have
>> not setup clocks correctly somewhere?
>
> The SPMI controller typically always has clks enabled, so I would
> be surprised if the clk was off. More likely, you're attempting
> to read/write a channel that is locked down and triggering an
> access control violation. Something configured incorrectly in DT
> perhaps?

The DT is certainly the most likely place to find the problem, the
SPMI driver etc. are mostly the same as 3.18.

The problem I am seeing is when the registers are initialized for the
s2 regulator (via SPMI), which I think is powering the CPU core (I
don't have the pm8909 docs sadly, only pm8916) and even though no bits
in the register get changed as part of the init, the writeback of the
register causes the board to reset.

In general I am having a bit if trouble understanding the regulator
setup. It seems like there are the RPM regulators - these don't seem
to be detected correctly. I get a remote_state of FLUSHING in
qcom_channel_state_worker which stops the devices being setup.

Then there are the SPMI regulators which seem like the same set of
regulators but via a different bus. Is that the case? Why is there the
need for two representations of the same regulators?

Thanks,

Re: msm8909 support in a recent kernel

[Bjorn Andersson]

On Wed 06 Dec 06:39 PST 2017, Will Newton wrote:

> On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> > On 12/01, Will Newton wrote:
> >> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> >>
> >> > It's not completely insane to support this SoC upstream though. You'd
> >> > have to bring in the pinctrl and clk drivers, which may be a bit of
> >> > effort. After that it should mostly be enabling various devices by
> >> > adding DT nodes and testing things out. It looks like this is 32-bit, so
> >> > getting SMP support may require some tweaks to the smp_ops code for qcom
> >> > platforms. You're right that it isn't too different from msm8916, so it
> >> > may be that most of the driver support for that SoC transfers over
> >> > nicely to this one.
> >>
> >> I've started from the 8916 drivers and started to port in the changes
> >> from the 3.18 tree that seem relevant. I have a kernel that boots and
> >> talks over the serial. I've done a bunch of pinctrl although it is not
> >> complete yet. I've had a look at the clocks and got the PLL working
> >> but I think I'm probably missing a document that describes the
> >> clocking architecture in more detail (I have the register reference
> >> but that's a bit of a worm's eye view).
> >>
> >> The current issue I am experiencing is the first write to an SPMI
> >> channel causes the board to reset. I suspect this means that I have
> >> not setup clocks correctly somewhere?
> >
> > The SPMI controller typically always has clks enabled, so I would
> > be surprised if the clk was off. More likely, you're attempting
> > to read/write a channel that is locked down and triggering an
> > access control violation. Something configured incorrectly in DT
> > perhaps?
> 
> The DT is certainly the most likely place to find the problem, the
> SPMI driver etc. are mostly the same as 3.18.
> 
> The problem I am seeing is when the registers are initialized for the
> s2 regulator (via SPMI), which I think is powering the CPU core (I
> don't have the pm8909 docs sadly, only pm8916) and even though no bits
> in the register get changed as part of the init, the writeback of the
> register causes the board to reset.
> 
> In general I am having a bit if trouble understanding the regulator
> setup. It seems like there are the RPM regulators - these don't seem
> to be detected correctly. I get a remote_state of FLUSHING in
> qcom_channel_state_worker which stops the devices being setup.
> 

Many regulators in the system feeds multiple subsystems, e.g. apps,
modem, wireless. To support this there is a separate co-processor - the
RPM - that take requests from all subsystems and then control the PMIC.
For these regulators the PMIC registers are locked down and accessing
these would typically cause a system reset.

To communicate with the RPM you would need to enable tcsr-mutex, smem,
apcs and smd/rpm/rpm_request like done for 8916. It's probable that the
only code you need to write is in qcom_smd-regulator.c to add the list
of regulators and their parameters.

> Then there are the SPMI regulators which seem like the same set of
> regulators but via a different bus. Is that the case? Why is there the
> need for two representations of the same regulators?
> 

Some regulators are dedicated for apps functionality, e.g. the CPU
rails, and these are directly accessible over SPMI. Probably due to
latency requirements.


So you have to get the rpm-regulators up and kicking, and you typically
don't need the spmi-regulators for a while.

Regards,
Bjorn

Re: msm8909 support in a recent kernel

[Will Newton]

On Wed, Dec 6, 2017 at 6:43 PM, Bjorn Andersson
<bjorn.andersson@linaro.org> wrote:
> On Wed 06 Dec 06:39 PST 2017, Will Newton wrote:
>
>> On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> > On 12/01, Will Newton wrote:
>> >> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> >>
>> >> > It's not completely insane to support this SoC upstream though. You'd
>> >> > have to bring in the pinctrl and clk drivers, which may be a bit of
>> >> > effort. After that it should mostly be enabling various devices by
>> >> > adding DT nodes and testing things out. It looks like this is 32-bit, so
>> >> > getting SMP support may require some tweaks to the smp_ops code for qcom
>> >> > platforms. You're right that it isn't too different from msm8916, so it
>> >> > may be that most of the driver support for that SoC transfers over
>> >> > nicely to this one.
>> >>
>> >> I've started from the 8916 drivers and started to port in the changes
>> >> from the 3.18 tree that seem relevant. I have a kernel that boots and
>> >> talks over the serial. I've done a bunch of pinctrl although it is not
>> >> complete yet. I've had a look at the clocks and got the PLL working
>> >> but I think I'm probably missing a document that describes the
>> >> clocking architecture in more detail (I have the register reference
>> >> but that's a bit of a worm's eye view).
>> >>
>> >> The current issue I am experiencing is the first write to an SPMI
>> >> channel causes the board to reset. I suspect this means that I have
>> >> not setup clocks correctly somewhere?
>> >
>> > The SPMI controller typically always has clks enabled, so I would
>> > be surprised if the clk was off. More likely, you're attempting
>> > to read/write a channel that is locked down and triggering an
>> > access control violation. Something configured incorrectly in DT
>> > perhaps?
>>
>> The DT is certainly the most likely place to find the problem, the
>> SPMI driver etc. are mostly the same as 3.18.
>>
>> The problem I am seeing is when the registers are initialized for the
>> s2 regulator (via SPMI), which I think is powering the CPU core (I
>> don't have the pm8909 docs sadly, only pm8916) and even though no bits
>> in the register get changed as part of the init, the writeback of the
>> register causes the board to reset.
>>
>> In general I am having a bit if trouble understanding the regulator
>> setup. It seems like there are the RPM regulators - these don't seem
>> to be detected correctly. I get a remote_state of FLUSHING in
>> qcom_channel_state_worker which stops the devices being setup.
>>
>
> Many regulators in the system feeds multiple subsystems, e.g. apps,
> modem, wireless. To support this there is a separate co-processor - the
> RPM - that take requests from all subsystems and then control the PMIC.
> For these regulators the PMIC registers are locked down and accessing
> these would typically cause a system reset.
>
> To communicate with the RPM you would need to enable tcsr-mutex, smem,
> apcs and smd/rpm/rpm_request like done for 8916. It's probable that the
> only code you need to write is in qcom_smd-regulator.c to add the list
> of regulators and their parameters.

Thanks, that's really helpful, I'll make sure I have all those functioning.

It does look like for the 8916 the same list of regulators (s1-s4,
l1-l18) is listed in both the spmi_bus section and the rpm_bus section
of the dts. What is it that decides whether a regulator is part of the
RPM set or the SPMI set - is it a property of the SoC that can be read
somewhere?

Re: msm8909 support in a recent kernel

[Bjorn Andersson]

On Wed 06 Dec 12:10 PST 2017, Will Newton wrote:

> On Wed, Dec 6, 2017 at 6:43 PM, Bjorn Andersson
> <bjorn.andersson@linaro.org> wrote:
> > On Wed 06 Dec 06:39 PST 2017, Will Newton wrote:
> >
> >> On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> >> > On 12/01, Will Newton wrote:
> >> >> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> >> >>
> >> >> > It's not completely insane to support this SoC upstream though. You'd
> >> >> > have to bring in the pinctrl and clk drivers, which may be a bit of
> >> >> > effort. After that it should mostly be enabling various devices by
> >> >> > adding DT nodes and testing things out. It looks like this is 32-bit, so
> >> >> > getting SMP support may require some tweaks to the smp_ops code for qcom
> >> >> > platforms. You're right that it isn't too different from msm8916, so it
> >> >> > may be that most of the driver support for that SoC transfers over
> >> >> > nicely to this one.
> >> >>
> >> >> I've started from the 8916 drivers and started to port in the changes
> >> >> from the 3.18 tree that seem relevant. I have a kernel that boots and
> >> >> talks over the serial. I've done a bunch of pinctrl although it is not
> >> >> complete yet. I've had a look at the clocks and got the PLL working
> >> >> but I think I'm probably missing a document that describes the
> >> >> clocking architecture in more detail (I have the register reference
> >> >> but that's a bit of a worm's eye view).
> >> >>
> >> >> The current issue I am experiencing is the first write to an SPMI
> >> >> channel causes the board to reset. I suspect this means that I have
> >> >> not setup clocks correctly somewhere?
> >> >
> >> > The SPMI controller typically always has clks enabled, so I would
> >> > be surprised if the clk was off. More likely, you're attempting
> >> > to read/write a channel that is locked down and triggering an
> >> > access control violation. Something configured incorrectly in DT
> >> > perhaps?
> >>
> >> The DT is certainly the most likely place to find the problem, the
> >> SPMI driver etc. are mostly the same as 3.18.
> >>
> >> The problem I am seeing is when the registers are initialized for the
> >> s2 regulator (via SPMI), which I think is powering the CPU core (I
> >> don't have the pm8909 docs sadly, only pm8916) and even though no bits
> >> in the register get changed as part of the init, the writeback of the
> >> register causes the board to reset.
> >>
> >> In general I am having a bit if trouble understanding the regulator
> >> setup. It seems like there are the RPM regulators - these don't seem
> >> to be detected correctly. I get a remote_state of FLUSHING in
> >> qcom_channel_state_worker which stops the devices being setup.
> >>
> >
> > Many regulators in the system feeds multiple subsystems, e.g. apps,
> > modem, wireless. To support this there is a separate co-processor - the
> > RPM - that take requests from all subsystems and then control the PMIC.
> > For these regulators the PMIC registers are locked down and accessing
> > these would typically cause a system reset.
> >
> > To communicate with the RPM you would need to enable tcsr-mutex, smem,
> > apcs and smd/rpm/rpm_request like done for 8916. It's probable that the
> > only code you need to write is in qcom_smd-regulator.c to add the list
> > of regulators and their parameters.
> 
> Thanks, that's really helpful, I'll make sure I have all those functioning.
> 
> It does look like for the 8916 the same list of regulators (s1-s4,
> l1-l18) is listed in both the spmi_bus section and the rpm_bus section
> of the dts. What is it that decides whether a regulator is part of the
> RPM set or the SPMI set - is it a property of the SoC that can be read
> somewhere?

I don't believe this is documented anywhere, except that by looking at
the downstream DTS, where you should be able to see which ones are
active and actually are referenced by clients.

But as I said, you can start off by only addressing the regulators
through the RPM, that should help you reach a functional state.

Regards,
Bjorn

Re: msm8909 support in a recent kernel

[Will Newton]

On Wed, Dec 6, 2017 at 9:58 PM, Bjorn Andersson
<bjorn.andersson@linaro.org> wrote:

Hi Bjorn,

>> On Wed, Dec 6, 2017 at 6:43 PM, Bjorn Andersson
>> <bjorn.andersson@linaro.org> wrote:
>> > On Wed 06 Dec 06:39 PST 2017, Will Newton wrote:
>> >
>> >> On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> >> > On 12/01, Will Newton wrote:
>> >> >> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>> >> >>
>> >> >> > It's not completely insane to support this SoC upstream though. You'd
>> >> >> > have to bring in the pinctrl and clk drivers, which may be a bit of
>> >> >> > effort. After that it should mostly be enabling various devices by
>> >> >> > adding DT nodes and testing things out. It looks like this is 32-bit, so
>> >> >> > getting SMP support may require some tweaks to the smp_ops code for qcom
>> >> >> > platforms. You're right that it isn't too different from msm8916, so it
>> >> >> > may be that most of the driver support for that SoC transfers over
>> >> >> > nicely to this one.
>> >> >>
>> >> >> I've started from the 8916 drivers and started to port in the changes
>> >> >> from the 3.18 tree that seem relevant. I have a kernel that boots and
>> >> >> talks over the serial. I've done a bunch of pinctrl although it is not
>> >> >> complete yet. I've had a look at the clocks and got the PLL working
>> >> >> but I think I'm probably missing a document that describes the
>> >> >> clocking architecture in more detail (I have the register reference
>> >> >> but that's a bit of a worm's eye view).
>> >> >>
>> >> >> The current issue I am experiencing is the first write to an SPMI
>> >> >> channel causes the board to reset. I suspect this means that I have
>> >> >> not setup clocks correctly somewhere?
>> >> >
>> >> > The SPMI controller typically always has clks enabled, so I would
>> >> > be surprised if the clk was off. More likely, you're attempting
>> >> > to read/write a channel that is locked down and triggering an
>> >> > access control violation. Something configured incorrectly in DT
>> >> > perhaps?
>> >>
>> >> The DT is certainly the most likely place to find the problem, the
>> >> SPMI driver etc. are mostly the same as 3.18.
>> >>
>> >> The problem I am seeing is when the registers are initialized for the
>> >> s2 regulator (via SPMI), which I think is powering the CPU core (I
>> >> don't have the pm8909 docs sadly, only pm8916) and even though no bits
>> >> in the register get changed as part of the init, the writeback of the
>> >> register causes the board to reset.
>> >>
>> >> In general I am having a bit if trouble understanding the regulator
>> >> setup. It seems like there are the RPM regulators - these don't seem
>> >> to be detected correctly. I get a remote_state of FLUSHING in
>> >> qcom_channel_state_worker which stops the devices being setup.
>> >>
>> >
>> > Many regulators in the system feeds multiple subsystems, e.g. apps,
>> > modem, wireless. To support this there is a separate co-processor - the
>> > RPM - that take requests from all subsystems and then control the PMIC.
>> > For these regulators the PMIC registers are locked down and accessing
>> > these would typically cause a system reset.
>> >
>> > To communicate with the RPM you would need to enable tcsr-mutex, smem,
>> > apcs and smd/rpm/rpm_request like done for 8916. It's probable that the
>> > only code you need to write is in qcom_smd-regulator.c to add the list
>> > of regulators and their parameters.
>>
>> Thanks, that's really helpful, I'll make sure I have all those functioning.
>>
>> It does look like for the 8916 the same list of regulators (s1-s4,
>> l1-l18) is listed in both the spmi_bus section and the rpm_bus section
>> of the dts. What is it that decides whether a regulator is part of the
>> RPM set or the SPMI set - is it a property of the SoC that can be read
>> somewhere?
>
> I don't believe this is documented anywhere, except that by looking at
> the downstream DTS, where you should be able to see which ones are
> active and actually are referenced by clients.
>
> But as I said, you can start off by only addressing the regulators
> through the RPM, that should help you reach a functional state.

I think I have narrowed down the issue to the fact that the qcom_smd
driver is not getting any interrupts. I get a channel created:

[    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
fifo-size: 1024
[    0.731684]  smd:rpm: new channel found: 'rpm_requests'

But on the first (and only) time through qcom_channel_state_worker the
channel is in state FLUSHING so we never create a device. Any idea why
I might not be getting any interrupts?

The irq numbers etc. all look correct as far as I can tell. smem and
tcsr-mutex also look OK. I'm not sure about apcs as I don't have clear
documentation on what it actually entails e.g. I have SAW devices
setup which seem to be part of APCS but not sure what else.

Re: msm8909 support in a recent kernel

[Will Newton]

On Thu, Dec 7, 2017 at 2:13 PM, Will Newton <will.newton@gmail.com> wrote:
> On Wed, Dec 6, 2017 at 9:58 PM, Bjorn Andersson
> <bjorn.andersson@linaro.org> wrote:
>
> Hi Bjorn,
>
>>> On Wed, Dec 6, 2017 at 6:43 PM, Bjorn Andersson
>>> <bjorn.andersson@linaro.org> wrote:
>>> > On Wed 06 Dec 06:39 PST 2017, Will Newton wrote:
>>> >
>>> >> On Wed, Dec 6, 2017 at 2:45 AM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>>> >> > On 12/01, Will Newton wrote:
>>> >> >> On Wed, Nov 29, 2017 at 6:50 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
>>> >> >>
>>> >> >> > It's not completely insane to support this SoC upstream though. You'd
>>> >> >> > have to bring in the pinctrl and clk drivers, which may be a bit of
>>> >> >> > effort. After that it should mostly be enabling various devices by
>>> >> >> > adding DT nodes and testing things out. It looks like this is 32-bit, so
>>> >> >> > getting SMP support may require some tweaks to the smp_ops code for qcom
>>> >> >> > platforms. You're right that it isn't too different from msm8916, so it
>>> >> >> > may be that most of the driver support for that SoC transfers over
>>> >> >> > nicely to this one.
>>> >> >>
>>> >> >> I've started from the 8916 drivers and started to port in the changes
>>> >> >> from the 3.18 tree that seem relevant. I have a kernel that boots and
>>> >> >> talks over the serial. I've done a bunch of pinctrl although it is not
>>> >> >> complete yet. I've had a look at the clocks and got the PLL working
>>> >> >> but I think I'm probably missing a document that describes the
>>> >> >> clocking architecture in more detail (I have the register reference
>>> >> >> but that's a bit of a worm's eye view).
>>> >> >>
>>> >> >> The current issue I am experiencing is the first write to an SPMI
>>> >> >> channel causes the board to reset. I suspect this means that I have
>>> >> >> not setup clocks correctly somewhere?
>>> >> >
>>> >> > The SPMI controller typically always has clks enabled, so I would
>>> >> > be surprised if the clk was off. More likely, you're attempting
>>> >> > to read/write a channel that is locked down and triggering an
>>> >> > access control violation. Something configured incorrectly in DT
>>> >> > perhaps?
>>> >>
>>> >> The DT is certainly the most likely place to find the problem, the
>>> >> SPMI driver etc. are mostly the same as 3.18.
>>> >>
>>> >> The problem I am seeing is when the registers are initialized for the
>>> >> s2 regulator (via SPMI), which I think is powering the CPU core (I
>>> >> don't have the pm8909 docs sadly, only pm8916) and even though no bits
>>> >> in the register get changed as part of the init, the writeback of the
>>> >> register causes the board to reset.
>>> >>
>>> >> In general I am having a bit if trouble understanding the regulator
>>> >> setup. It seems like there are the RPM regulators - these don't seem
>>> >> to be detected correctly. I get a remote_state of FLUSHING in
>>> >> qcom_channel_state_worker which stops the devices being setup.
>>> >>
>>> >
>>> > Many regulators in the system feeds multiple subsystems, e.g. apps,
>>> > modem, wireless. To support this there is a separate co-processor - the
>>> > RPM - that take requests from all subsystems and then control the PMIC.
>>> > For these regulators the PMIC registers are locked down and accessing
>>> > these would typically cause a system reset.
>>> >
>>> > To communicate with the RPM you would need to enable tcsr-mutex, smem,
>>> > apcs and smd/rpm/rpm_request like done for 8916. It's probable that the
>>> > only code you need to write is in qcom_smd-regulator.c to add the list
>>> > of regulators and their parameters.
>>>
>>> Thanks, that's really helpful, I'll make sure I have all those functioning.
>>>
>>> It does look like for the 8916 the same list of regulators (s1-s4,
>>> l1-l18) is listed in both the spmi_bus section and the rpm_bus section
>>> of the dts. What is it that decides whether a regulator is part of the
>>> RPM set or the SPMI set - is it a property of the SoC that can be read
>>> somewhere?
>>
>> I don't believe this is documented anywhere, except that by looking at
>> the downstream DTS, where you should be able to see which ones are
>> active and actually are referenced by clients.
>>
>> But as I said, you can start off by only addressing the regulators
>> through the RPM, that should help you reach a functional state.
>
> I think I have narrowed down the issue to the fact that the qcom_smd
> driver is not getting any interrupts. I get a channel created:
>
> [    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
> fifo-size: 1024
> [    0.731684]  smd:rpm: new channel found: 'rpm_requests'
>
> But on the first (and only) time through qcom_channel_state_worker the
> channel is in state FLUSHING so we never create a device. Any idea why
> I might not be getting any interrupts?
>
> The irq numbers etc. all look correct as far as I can tell. smem and
> tcsr-mutex also look OK. I'm not sure about apcs as I don't have clear
> documentation on what it actually entails e.g. I have SAW devices
> setup which seem to be part of APCS but not sure what else.

Is there any potential for version skew here between the RPM processor
and Linux? I have an rpm.mbn binary but I have no idea what is in it,
I just know it works with 3.18.

Re: msm8909 support in a recent kernel

[Stephen Boyd]

On 12/08, Will Newton wrote:
> On Thu, Dec 7, 2017 at 2:13 PM, Will Newton <will.newton@gmail.com> wrote:
> >
> > I think I have narrowed down the issue to the fact that the qcom_smd
> > driver is not getting any interrupts. I get a channel created:
> >
> > [    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
> > fifo-size: 1024
> > [    0.731684]  smd:rpm: new channel found: 'rpm_requests'
> >
> > But on the first (and only) time through qcom_channel_state_worker the
> > channel is in state FLUSHING so we never create a device. Any idea why
> > I might not be getting any interrupts?
> >
> > The irq numbers etc. all look correct as far as I can tell. smem and
> > tcsr-mutex also look OK. I'm not sure about apcs as I don't have clear
> > documentation on what it actually entails e.g. I have SAW devices
> > setup which seem to be part of APCS but not sure what else.
> 
> Is there any potential for version skew here between the RPM processor
> and Linux? I have an rpm.mbn binary but I have no idea what is in it,
> I just know it works with 3.18.

No, not really. This sounds like a similar problem that was seen
on msm8994, but I can't recall if it was ever resolved. Maybe
that was channel closing instead of flushing.

-- 
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum,
a Linux Foundation Collaborative Project

Re: msm8909 support in a recent kernel

[Will Newton]

On Fri, Dec 8, 2017 at 5:07 PM, Stephen Boyd <sboyd@codeaurora.org> wrote:
> On 12/08, Will Newton wrote:
>> On Thu, Dec 7, 2017 at 2:13 PM, Will Newton <will.newton@gmail.com> wrote:
>> >
>> > I think I have narrowed down the issue to the fact that the qcom_smd
>> > driver is not getting any interrupts. I get a channel created:
>> >
>> > [    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
>> > fifo-size: 1024
>> > [    0.731684]  smd:rpm: new channel found: 'rpm_requests'
>> >
>> > But on the first (and only) time through qcom_channel_state_worker the
>> > channel is in state FLUSHING so we never create a device. Any idea why
>> > I might not be getting any interrupts?
>> >
>> > The irq numbers etc. all look correct as far as I can tell. smem and
>> > tcsr-mutex also look OK. I'm not sure about apcs as I don't have clear
>> > documentation on what it actually entails e.g. I have SAW devices
>> > setup which seem to be part of APCS but not sure what else.
>>
>> Is there any potential for version skew here between the RPM processor
>> and Linux? I have an rpm.mbn binary but I have no idea what is in it,
>> I just know it works with 3.18.
>
> No, not really. This sounds like a similar problem that was seen
> on msm8994, but I can't recall if it was ever resolved. Maybe
> that was channel closing instead of flushing.

Are there any potential issues with coherency between the cpu and rpm?
e.g. I haven't put much thought into how the L2 is initialized.

Re: msm8909 support in a recent kernel

[Bjorn Andersson]

On Thu 07 Dec 06:13 PST 2017, Will Newton wrote:
[..]
> 
> I think I have narrowed down the issue to the fact that the qcom_smd
> driver is not getting any interrupts. I get a channel created:
> 
> [    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
> fifo-size: 1024
> [    0.731684]  smd:rpm: new channel found: 'rpm_requests'
> 

That's good.

> But on the first (and only) time through qcom_channel_state_worker the
> channel is in state FLUSHING so we never create a device. Any idea why
> I might not be getting any interrupts?
> 

I presume you mean the value of GET_RX_CHANNEL_INFO(channel, state)
here. This has been reported on 8994 as well and do stem from a
misunderstanding of mine when it comes to how the state machine is
ticking when a channel is closed.

What we see on 8994 is that the boot loader opens the rpm_requests
channel, send a few regulator requests and then put the channel (the
apps side of it) in closing state. During boot of the Linux kernel the
rpm_requests channel is then found in "closing" state.

And looking at LK, the two platforms implementing this schema are 8994
and 8909.


My two hypothesises for 8994 was that we could either:

1) Complete the closing (i.e. not leave the channel in closing/reset
state) and that might cause the other side to move to "opening" again,
or
2) take a more active role in driving the channel to opening,
based on the knowledge that Linux do have someone to interested in
communicating over this link.

The first is pretty close of being a hack and the second relies on this
loop figuring out if there is going to be a driver-device match in the
future.

Unfortunately, this being reported on 8994 gave it a low priority, so
it's been sitting far down on my todo list ever since.

> The irq numbers etc. all look correct as far as I can tell. smem and
> tcsr-mutex also look OK. I'm not sure about apcs as I don't have clear
> documentation on what it actually entails e.g. I have SAW devices
> setup which seem to be part of APCS but not sure what else.

Looked at the documentation, apcs should be defined as reg = <0x0b011000
0x1000> and qcom,ipc should be <&apcs 8 0>. I.e. exactly the same as
8916.

Regards,
Bjorn

Re: msm8909 support in a recent kernel

[Will Newton]

On Mon, Dec 11, 2017 at 6:11 AM, Bjorn Andersson
<bjorn.andersson@linaro.org> wrote:
> On Thu 07 Dec 06:13 PST 2017, Will Newton wrote:
> [..]
>>
>> I think I have narrowed down the issue to the fact that the qcom_smd
>> driver is not getting any interrupts. I get a channel created:
>>
>> [    0.728352]  smd:rpm: new channel 'rpm_requests' info-size: 88
>> fifo-size: 1024
>> [    0.731684]  smd:rpm: new channel found: 'rpm_requests'
>>
>
> That's good.
>
>> But on the first (and only) time through qcom_channel_state_worker the
>> channel is in state FLUSHING so we never create a device. Any idea why
>> I might not be getting any interrupts?
>>
>
> I presume you mean the value of GET_RX_CHANNEL_INFO(channel, state)
> here. This has been reported on 8994 as well and do stem from a
> misunderstanding of mine when it comes to how the state machine is
> ticking when a channel is closed.

Yes, that's correct.

> What we see on 8994 is that the boot loader opens the rpm_requests
> channel, send a few regulator requests and then put the channel (the
> apps side of it) in closing state. During boot of the Linux kernel the
> rpm_requests channel is then found in "closing" state.
>
> And looking at LK, the two platforms implementing this schema are 8994
> and 8909.

Is this conditional on ENABLE_SMD_SUPPORT in lk? It certainly sounds
plausible, although the version of lk that I am running looks like it
waits for an interrupt after setting the state to CLOSING and sets the
state to CLOSED so should get this right.

> My two hypothesises for 8994 was that we could either:
>
> 1) Complete the closing (i.e. not leave the channel in closing/reset
> state) and that might cause the other side to move to "opening" again,

So as I understand it the idea here is to do what lk failed to do. I
tried adding some code at the top of qcom_smd_channel_reset to detect
that a channel was in the closing state and set it to closed:

        if (remote_state == SMD_CHANNEL_CLOSING) {
                SET_RX_CHANNEL_FLAG(channel, fDSR, 0);
                SET_RX_CHANNEL_FLAG(channel, fCTS, 0);
                SET_RX_CHANNEL_FLAG(channel, fCD, 0);
                SET_RX_CHANNEL_INFO(channel, state,
SMD_CHANNEL_CLOSED);
                SET_RX_CHANNEL_INFO(channel, head, 0);
                SET_RX_CHANNEL_INFO(channel, tail, 0);
                SET_RX_CHANNEL_FLAG(channel, fTAIL, 1);
                qcom_smd_signal_channel(channel);
        }

This didn't seem to work though. The channel was now in the CLOSED
state but did not seem to move to OPENING afterward.

I also added some code to dump the state of the channel in case that is useful:

[    0.709877] TX: state 0 fDSR 0x0 fCTS 0x0 fCD 0x0 fRI 0x0 fHEAD 0x0
fTAIL 0x0 fSTATE 0x0 fBLOCKREADINTR 0x1 tail 0x120 head 0x120
[    0.728901] RX: state 4 fDSR 0x1 fCTS 0x1 fCD 0x1 fRI 0x0 fHEAD 0x0
fTAIL 0x1 fSTATE 0x1 fBLOCKREADINTR 0x1 tail 0x78 head 0x78

> or
> 2) take a more active role in driving the channel to opening,
> based on the knowledge that Linux do have someone to interested in
> communicating over this link.

Do you have an idea of what this approach might look like?

Thanks,

Re: msm8909 support in a recent kernel

[Will Newton]

On Tue, Dec 12, 2017 at 10:59 AM, Will Newton <will.newton@gmail.com> wrote:
> On Mon, Dec 11, 2017 at 6:11 AM, Bjorn Andersson
> <bjorn.andersson@linaro.org> wrote:
>> or
>> 2) take a more active role in driving the channel to opening,
>> based on the knowledge that Linux do have someone to interested in
>> communicating over this link.
>
> Do you have an idea of what this approach might look like?

After a bit more poking at the code I think I understand what you mean
more now. It looks like if we put our side of the channel into the
OPENING state instead of the CLOSED state in qcom_smd_channel_reset
then the remote state machine flips back into OPENED state and
everything seems to be pretty much working. The current code we seem
to get stuck with our end in CLOSED and the remote end in CLOSING and
nobody making the first move to open the channel.

Does making that simple change in qcom_smd_channel_reset seem OK to
you or do we need to change the driver more drastically than that?

(I am seeing an "external abort on non-linefetch" after making this
change, however it doesn't appear to be in the smd driver so may not
be directly related)

Re: msm8909 support in a recent kernel

[Bjorn Andersson]

On Tue 12 Dec 02:59 PST 2017, Will Newton wrote:

> On Mon, Dec 11, 2017 at 6:11 AM, Bjorn Andersson
> <bjorn.andersson@linaro.org> wrote:
[..]
> > or
> > 2) take a more active role in driving the channel to opening,
> > based on the knowledge that Linux do have someone to interested in
> > communicating over this link.
> 
> Do you have an idea of what this approach might look like?
> 

I found my notes from looking at this last time and implemented this. I
don't have a board where I can test that this solves your problem so
testing of the first two patches in the series I just sent would be
appreciated.

Regards,
Bjorn

Re: msm8909 support in a recent kernel

[Will Newton]

On Wed, Dec 13, 2017 at 12:01 AM, Bjorn Andersson
<bjorn.andersson@linaro.org> wrote:
> On Tue 12 Dec 02:59 PST 2017, Will Newton wrote:
>
>> On Mon, Dec 11, 2017 at 6:11 AM, Bjorn Andersson
>> <bjorn.andersson@linaro.org> wrote:
> [..]
>> > or
>> > 2) take a more active role in driving the channel to opening,
>> > based on the knowledge that Linux do have someone to interested in
>> > communicating over this link.
>>
>> Do you have an idea of what this approach might look like?
>>
>
> I found my notes from looking at this last time and implemented this. I
> don't have a board where I can test that this solves your problem so
> testing of the first two patches in the series I just sent would be
> appreciated.

Thanks for doing this. I tested the first two patches on an MSM8909
dev board and they work for me.

Tested-by: Will Newton <will.newton@gmail.com>