Re: [PATCHv3 01/19] [HACK] of: dev_node has struct device pointer

[Thierry Reding <thierry.reding-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>]

[-- Attachment #1: Type: text/plain, Size: 4996 bytes --]

On Fri, Oct 25, 2013 at 10:25:49AM +0200, Hiroshi Doyu wrote:
> Thierry Reding <thierry.reding-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote @ Fri, 25 Oct 2013 09:56:55 +0200:
> 
> > > > This patch is a part of HACK to control device instanciation order. We
> > > > have an IOMMU device(platform) which needs to be instanciated earlier
> > > > than other (platform)devices so that IOMMU driver would configure them
> > > > as IOMMU'able device.
> > > 
> > > Ideally the drivers depending on the IOMMU would return -EPROBE_DEFER if
> > > the IOMMU driver isn't set up so that you don't need to play games with
> > > probe order. Creating certain platform devices early is a really ugly
> > > and fragile solution.
> > > 
> > > Besides, probe order of device drivers is far more about link order of
> > > the kernel than it is about when of_platform_device_create() is called.
> > > Fiddling with the initcall level on the IOMMU driver (while not
> > > recommended) may very well have the effect you desire.
> > 
> > This is actually "the other problem that I'm aware of that could benefit
> > from [interrupt resolution at probe time]". My idea was that once we had
> > a way within the driver core to resolve interrupt references at probe
> > time it could be used for potentially many other resources as well. Some
> > of the resources like GPIOs and regulators are obviously not something
> > that the core can or should be requesting, but mostly resources that you
> > don't actually need to control after probing (such as interrupts) would
> > be a good fit because otherwise people would write the same boilerplate
> > over and over again.
> > 
> > IOMMUs seem to me to be in that same category. As far as I can tell, an
> > IOMMU driver registers the IOMMU for a given bus, upon which every
> > device can simply be used (mostly transparently) with that IOMMU. While
> > I haven't figured out how exactly, I'm pretty sure we can take advantage
> > of the resolution of resources at probe time within the core to both
> > keep drivers from having to do anything in particular and at the same
> > time have code to determine if the IOMMU driver hasn't been probed yet
> > and return -EPROBE_DEFER appropriately.
> 
> Can you explain the above a bit more?
> 
> Originally I thought that what Grant suggested would work ok with this
> patch.

I think the objection to these patches is that they special case the
instantiation of some devices. It's not a proper solution because it
implies various things. For example merely instantiating the IOMMU
device earlier than others is only going to work *if* the driver is
actually probed before the drivers of other devices. If you want to
build the driver as a module for example, probe order becomes entirely
non-deterministic.

So what Grant was saying is that you could possibly make it work by
forcing the driver to be loaded earlier using explicit initcall
ordering. But he also said that's not recommended because it's not a
proper solution and therefore not guaranteed to always work.

Explicit initcall ordering used to be heavily used in the past, but
there have been many efforts to move away from it. One of the solutions
introduced to help with that is deferred probing, which essentially adds
a new error code (EPROBE_DEFER) which a driver's .probe() can return to
cause it to be probed at a later point again, after other drivers have
been probed.

How this works is basically that a driver's .probe() requests whatever
resources it needs (GPIOs, clocks, regulators, ...). If any of those
resources isn't there yet (presumably because the driver providing it
hasn't been probed yet), it can return -EPROBE_DEFER to signal that not
all of its dependencies are available yet.

Instead of handling such dependencies implicitly by making sure all
resource providers are probed earlier than any of their consumers, the
dependencies are handled more explicitly, which turns out to simplify
things a lot. There's some additional work required in the core, but if
done consistently no driver needs to care about the dependencies and it
no longer matters where the resources come from. The problem is reduced
to essentially this:

	while (!resource_available())
		load_more_drivers();

So my proposed solution for the IOMMU case is to treat it the same as
any other resources. Perhaps resource isn't the right word, but at the
core the issue is the same. A device requires the services of an IOMMU
so that it can be put into the correct address space. If the IOMMU is
not available yet it cannot do that, so we simply return -EPROBE_DEFER
and cause the probe to be retried later.

Eventually the IOMMU driver will be probed and register the IOMMU. When
the earlier driver is probed again, it will be able to successfully
request to be put into the proper address space and continue with the
initialization.

Does that answer your question?

Thierry

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]