irqchip heirarchy DT "break" series awareness?

irqchip heirarchy DT "break" series awareness?

[Jason Cooper]

Arnd, Olof, others,

Have you been following Marc's irqchip series(es) leveraging stacked
domains to remove the abuse of gic_arch_extn ?

  Tegra LIC
  https://lkml.kernel.org/r/1426088583-15097-1-git-send-email-marc.zyngier at arm.com

  OMAP Crossbar
  https://lkml.kernel.org/r/1426088629-15377-1-git-send-email-marc.zyngier at arm.com

  Exynos PM
  https://lkml.kernel.org/r/1426088693-15724-1-git-send-email-marc.zyngier at arm.com

  shmobile, ux500, zynq irq_set_wake
  https://lkml.kernel.org/r/1426088737-15817-1-git-send-email-marc.zyngier at arm.com

  imx6 was taken by Shawn
  https://lkml.kernel.org/r/1426262737-32762-1-git-send-email-marc.zyngier at arm.com

You can find the patches in one series per branch at:

  git://git.infradead.org/users/jcooper/linux.git irqchip/stacked-tegra
  git://git.infradead.org/users/jcooper/linux.git irqchip/stacked-omap
  git://git.infradead.org/users/jcooper/linux.git irqchip/stacked-exynos
  git://git.infradead.org/users/jcooper/linux.git irqchip/stacked-irq_set_wake

I ask because with Thomas' (tglx) absence, it looks like I'm going to be
sending the pull request directly to Linus.  This has increased my
pucker factor a bit. :-)

The logistical stuff is in order.  It'll be my second pull request, and
it'll only contain the changes relevant.  The whole set has gone through
several rounds of review.  All the arm sub-arch maintainers have either
taken the part relevant to them, Acked me taking them, or failed to
object while they've been in linux-next for several weeks now.

My concern is the DT ABI stability problem.  In short, we fucked up.
When we added several of the irqchip bindings, we designed them based on
the Linux implementation (gic_arch_extn).  *Not* by describing the
hardware.

Marc's series undoes this in the best way possible.  He changes the DT
bindings to actually describe the hardware, which then gets modeled in
stacked domains quite well.

This causes two problems:

 1) Upgrade kernel, but not DTB.

	System will boot, and print a big fat warning that
	suspend/resume will not work until the DTB is upgraded.

 2) Upgrade DTB, but not kernel.

	System will fail to boot.

In light of Thomas Petazonni's well-researched talk at ELC:

  "The Device Tree as a stable ABI: a fairy tale?"
  http://events.linuxfoundation.org/sites/events/files/slides/petazzoni-dt-as-stable-abi-fairy-tale.pdf

I'm confident that #2 won't be an issue.  Distro's and OEMs seem to have
worked around the instability by keeping the dtb tied to the kernel
version.

And, on the off chance that end users upgrade their kernel, say, by
building mainline, there's a very clear warning that tells them exactly
what to do:  Upgrade the dtb as well.

Do you foresee any problem with this?  Is there anything I haven't
considered?  Or extra information I'll need to present in my pull
request?

thx,

Jason.

irqchip heirarchy DT "break" series awareness?

[Thomas Petazzoni]

Jason,

On Mon, 6 Apr 2015 14:46:47 +0000, Jason Cooper wrote:

> This causes two problems:
> 
>  1) Upgrade kernel, but not DTB.
> 
> 	System will boot, and print a big fat warning that
> 	suspend/resume will not work until the DTB is upgraded.
> 
>  2) Upgrade DTB, but not kernel.
> 
> 	System will fail to boot.

(2) has never been something that has ever planned of being guaranteed,
as far as I know. Only (1) is the supposed consequence of DT ABI
stability, but definitely not (2), so I'm unsure why you even mention
(2).

> In light of Thomas Petazonni's well-researched talk at ELC:
> 
>   "The Device Tree as a stable ABI: a fairy tale?"
>   http://events.linuxfoundation.org/sites/events/files/slides/petazzoni-dt-as-stable-abi-fairy-tale.pdf
> 
> I'm confident that #2 won't be an issue.  Distro's and OEMs seem to have
> worked around the instability by keeping the dtb tied to the kernel
> version.

I'm glad you raised my slides as an argument in a DT ABI stability
discussion :-)

However, my slides are definitely not about #2 (which as said earlier,
was never planned to be something we should worry about), but really
about #1.

But the point of the slides stand: even for a piece of hardware as
well-documented as the GIC, as widely used as the GIC, with as many
bright and smart engineers looking into it, the community has not been
able to put out a DT binding that can be kept stable. How can we expect
such a DT binding stability to occur for undocumented hardware, or
SoC-specific hardware blocks that are definitely a lot less used than
the GIC ?

Best regards,

Thomas Petazzoni
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com

irqchip heirarchy DT "break" series awareness?

[Marc Zyngier]

Hi Thomas,

On 07/04/15 10:59, Thomas Petazzoni wrote:

> But the point of the slides stand: even for a piece of hardware as
> well-documented as the GIC, as widely used as the GIC, with as many
> bright and smart engineers looking into it, the community has not been
> able to put out a DT binding that can be kept stable. How can we expect
> such a DT binding stability to occur for undocumented hardware, or
> SoC-specific hardware blocks that are definitely a lot less used than
> the GIC ?

The problem at hand is not so much the GIC itself, but the fact that
only the GIC was described in DT. The GIC binding is unchanged, but some
additional hardware is now described.

If the relationship between the GIC and the shadow interrupt controllers
had been described, we would have avoided breaking the compatibility. I
guess it was too tempting to reuse pre-DT mechanisms and to forget about
this entirely.

Thanks,

	M.
-- 
Jazz is not dead. It just smells funny...

irqchip heirarchy DT "break" series awareness?

[Jason Cooper]

On Tue, Apr 07, 2015 at 11:21:35AM +0100, Marc Zyngier wrote:
> Hi Thomas,
> 
> On 07/04/15 10:59, Thomas Petazzoni wrote:
> 
> > But the point of the slides stand: even for a piece of hardware as
> > well-documented as the GIC, as widely used as the GIC, with as many
> > bright and smart engineers looking into it, the community has not been
> > able to put out a DT binding that can be kept stable. How can we expect
> > such a DT binding stability to occur for undocumented hardware, or
> > SoC-specific hardware blocks that are definitely a lot less used than
> > the GIC ?
> 
> The problem at hand is not so much the GIC itself, but the fact that
> only the GIC was described in DT. The GIC binding is unchanged, but some
> additional hardware is now described.

Well, if that were the case we wouldn't have a break in DT
compatibility.  I suppose what we're going for here is "removed GIC
binding properties (arm,routable-irqs) that didn't describe hardware".
Similar for crossbar and the others that were inappropriately relying on
gic_arch_extn implementation to model the (incorrect) hardware
description.

> If the relationship between the GIC and the shadow interrupt controllers
> had been described, we would have avoided breaking the compatibility. I
> guess it was too tempting to reuse pre-DT mechanisms and to forget about
> this entirely.

I'm not sure tempting is the right word.  Everyone has known since this
project began that we were striving to describe the hardware.  I suspect
the reason we got to where we are is that people *assumed* the code was
describing the hardware, and so wrote bindings reflecting their
understanding.  iow, we don't have enough hardware engineers reviewing
bindings :-P

Be that as it may, I'm not trying to rehash the decision.  It's clearly
the correct thing to do.  Otherwise, I wouldn't have pulled it in.  What
I am trying to do here is make sure a) we have full awareness by
everybody not directly involved, and b) make sure I have my ducks in a
row if ThomasG/Linus raises questions regarding the pull request.

thx,

Jason.

irqchip heirarchy DT "break" series awareness?

[Mark Rutland]

> > If the relationship between the GIC and the shadow interrupt controllers
> > had been described, we would have avoided breaking the compatibility. I
> > guess it was too tempting to reuse pre-DT mechanisms and to forget about
> > this entirely.
> 
> I'm not sure tempting is the right word.  Everyone has known since this
> project began that we were striving to describe the hardware.  I suspect
> the reason we got to where we are is that people *assumed* the code was
> describing the hardware, and so wrote bindings reflecting their
> understanding.  iow, we don't have enough hardware engineers reviewing
> bindings :-P

FWIW when routable-irqs was introduced I negatively reviewed it, stating
that it was backwards, and that the crossbar driver should request the
appropriate domain dynamically [1]. We _knew_ it was wrong at the time,
but people decided to ignore that review because it was easier to do
things in a broken way.

A more general issue is that people hack in the minimal amount of DT
parsing code to an existing driver and rely on platform data and code to
do the heavy lifting, rather than doing a clean break with DT (which
would highlight deficiencies in the bndings). You end up with the worst
of both worlds, because you depend on both an incomplete DT and
implementation-specific platform data/code.

Mark.

[1] http://lists.infradead.org/pipermail/linux-arm-kernel/2013-November/211530.html

irqchip heirarchy DT "break" series awareness?

[Jason Cooper]

Hey Thomas,

On Tue, Apr 07, 2015 at 11:59:22AM +0200, Thomas Petazzoni wrote:
> Jason,
> 
> On Mon, 6 Apr 2015 14:46:47 +0000, Jason Cooper wrote:
> 
> > This causes two problems:
> > 
> >  1) Upgrade kernel, but not DTB.
> > 
> > 	System will boot, and print a big fat warning that
> > 	suspend/resume will not work until the DTB is upgraded.
> > 
> >  2) Upgrade DTB, but not kernel.
> > 
> > 	System will fail to boot.
> 
> (2) has never been something that has ever planned of being guaranteed,
> as far as I know. Only (1) is the supposed consequence of DT ABI
> stability, but definitely not (2), so I'm unsure why you even mention
> (2).

Because my goal here was to give *every* possible chance for an
objection.  Hence, "dt" and "break" in the subject line, sent to the
devicetree ML.  Also, Marc mentioned the possibility in at least the
cover letter of each of his series.  I don't want anything hidden.

But you are absolutely correct.  (2) was never guaranteed, and it's also
highly improbable as well.  See below.

> > In light of Thomas Petazonni's well-researched talk at ELC:
> > 
> >   "The Device Tree as a stable ABI: a fairy tale?"
> >   http://events.linuxfoundation.org/sites/events/files/slides/petazzoni-dt-as-stable-abi-fairy-tale.pdf
> > 
> > I'm confident that #2 won't be an issue.  Distro's and OEMs seem to have
> > worked around the instability by keeping the dtb tied to the kernel
> > version.
> 
> I'm glad you raised my slides as an argument in a DT ABI stability
> discussion :-)
> 
> However, my slides are definitely not about #2 (which as said earlier,
> was never planned to be something we should worry about), but really
> about #1.

I was referring to the slide where you mention that distros and vendors
have tied the dtbs to the kernel versions (Slide 23/27, "Usefulness").
I should have been more specific when taking something out of context.
:-P

My point, not well made, was that everyone has decided to slave the
upgrade of the dtb to the upgrade of the kernel.  There is no 'apt-get
armv7-dtbs' that has no dependency structure on a kernel package.  So we
agree, (2) was never guaranteed, and isn't probable either.

thx,

Jason.

irqchip heirarchy DT "break" series awareness?

[Thomas Petazzoni]

Jason,

On Tue, 7 Apr 2015 12:40:16 +0000, Jason Cooper wrote:

> Because my goal here was to give *every* possible chance for an
> objection.  Hence, "dt" and "break" in the subject line, sent to the
> devicetree ML.  Also, Marc mentioned the possibility in at least the
> cover letter of each of his series.  I don't want anything hidden.
> 
> But you are absolutely correct.  (2) was never guaranteed, and it's also
> highly improbable as well.  See below.

Right.


> > However, my slides are definitely not about #2 (which as said earlier,
> > was never planned to be something we should worry about), but really
> > about #1.
> 
> I was referring to the slide where you mention that distros and vendors
> have tied the dtbs to the kernel versions (Slide 23/27, "Usefulness").
> I should have been more specific when taking something out of context.
> :-P
> 
> My point, not well made, was that everyone has decided to slave the
> upgrade of the dtb to the upgrade of the kernel.  There is no 'apt-get
> armv7-dtbs' that has no dependency structure on a kernel package.  So we
> agree, (2) was never guaranteed, and isn't probable either.

And so is (1), then :-)

Thomas
-- 
Thomas Petazzoni, CTO, Free Electrons
Embedded Linux, Kernel and Android engineering
http://free-electrons.com