Request review of device tree documentation

Request review of device tree documentation

[Grant Likely]

I've been doing a bit of work on some introductory level documentation
of the flattened device tree.  I've got a rough copy up on the
devicetree.org wiki, and I could use some feedback.  If anyone has
some time to look at it, you can find it here:

http://devicetree.org/Device_Tree_Usage

Thanks,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Request review of device tree documentation

[Dan Malek]

Hi Grant.

On Jun 11, 2010, at 3:59 PM, Grant Likely wrote:

> I've been doing a bit of work on some introductory level documentation
> of the flattened device tree.

Wow, I feel empowered to create device trees now :-)
Seriously, I never understood this well and this is a
great document.

I have one source of confusion.  Your first Initial structure
example uses 'compatible' to describe the machine, the
paragraph below then mentions the 'model' property,
and all subsequent examples use model.

Does this mean if I use just the single line in the dts,
using 'compatible' implies the ARM machine ID?  If I
have more description I use 'model'?

Thanks.

	-- Dan

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Fri, 2010-06-11 at 16:47 -0700, Dan Malek wrote:
> Hi Grant.
> 
> On Jun 11, 2010, at 3:59 PM, Grant Likely wrote:
> 
> > I've been doing a bit of work on some introductory level documentation
> > of the flattened device tree.
> 
> Wow, I feel empowered to create device trees now :-)
> Seriously, I never understood this well and this is a
> great document.
> 
> I have one source of confusion.  Your first Initial structure
> example uses 'compatible' to describe the machine, the
> paragraph below then mentions the 'model' property,
> and all subsequent examples use model.
> 
> Does this mean if I use just the single line in the dts,
> using 'compatible' implies the ARM machine ID?  If I
> have more description I use 'model'?

Normally, "compatible" is what is used for code to match,
and model is more like a user-visible thingy.

It's possible to peek at 'model' tho, in some cases, I've seen the case
for example where things are -supposed- to be identical from an arch
point of view, have the same compatible, but later on, a quirk is found
and a test against model is used to differentiate. But that's something
to avoid in general. Better off having multiple strings in "compatible"
then, one more "generic" to have the BSP match against, and one more
"specific" that can be used if a quirk is needed.

Of course, it doesn't help that all pseries have "chrp" and nothing else
as compatible :-) But then, both IBM and Apple have been quite lax with
their (ab)use of the DT.

Cheers,
Ben.

Re: Request review of device tree documentation

[Mitch Bradley]

Benjamin Herrenschmidt wrote:
> On Fri, 2010-06-11 at 16:47 -0700, Dan Malek wrote:
>   
>> Hi Grant.
>>
>> On Jun 11, 2010, at 3:59 PM, Grant Likely wrote:
>>
>>     
>>> I've been doing a bit of work on some introductory level documentation
>>> of the flattened device tree.
>>>       
>> Wow, I feel empowered to create device trees now :-)
>> Seriously, I never understood this well and this is a
>> great document.
>>
>> I have one source of confusion.  Your first Initial structure
>> example uses 'compatible' to describe the machine, the
>> paragraph below then mentions the 'model' property,
>> and all subsequent examples use model.
>>
>> Does this mean if I use just the single line in the dts,
>> using 'compatible' implies the ARM machine ID?  If I
>> have more description I use 'model'?
>>     
>
> Normally, "compatible" is what is used for code to match,
> and model is more like a user-visible thingy.
>   

Indeed, one common use of "model" - at least in the systems I work on - 
is to display the name of the machine in a system identification banner 
that the user sees.
> It's possible to peek at 'model' tho, in some cases, I've seen the case
> for example where things are -supposed- to be identical from an arch
> point of view, have the same compatible, but later on, a quirk is found
> and a test against model is used to differentiate. But that's something
> to avoid in general. Better off having multiple strings in "compatible"
> then, one more "generic" to have the BSP match against, and one more
> "specific" that can be used if a quirk is needed.
>
> Of course, it doesn't help that all pseries have "chrp" and nothing else
> as compatible :-) But then, both IBM and Apple have been quite lax with
> their (ab)use of the DT.
>
> Cheers,
> Ben.
>
>
> _______________________________________________
> devicetree-discuss mailing list
> devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
> https://lists.ozlabs.org/listinfo/devicetree-discuss
>
>   

Re: Request review of device tree documentation

[Grant Likely]

On Fri, Jun 11, 2010 at 5:47 PM, Dan Malek <ppc6dev@digitaldans.com> wrote:
>
> Hi Grant.
>
> On Jun 11, 2010, at 3:59 PM, Grant Likely wrote:
>
>> I've been doing a bit of work on some introductory level documentation
>> of the flattened device tree.
>
> Wow, I feel empowered to create device trees now :-)
> Seriously, I never understood this well and this is a
> great document.

Hey, thanks!  That's a fantastic compliment!

> I have one source of confusion.  Your first Initial structure
> example uses 'compatible' to describe the machine, the
> paragraph below then mentions the 'model' property,
> and all subsequent examples use model.
>
> Does this mean if I use just the single line in the dts,
> using 'compatible' implies the ARM machine ID?  If I
> have more description I use 'model'?

No, that is just a reflection of a late change I made to the document.
 It should be compatible all the way down.  I was trying to keep
things simple because on a lot of boards both compatible and model
properties will exist in the root node.  I started writing with model,
and then changed it last minute without fixing everything up which is
why the document was confusing.

I've fixed it now if you want to take a look.

I also changed the property in the cpu nodes from model to compatible
so that the exact CPU version can be specified.  This isn't actually
in any spec anywhere, but I need something to properly identify the
different ARM cores.

Mitch, I know you were working on a draft ARM binding a while ago,
have you resurrected it at all?  How do you think the core should be
identified?

Cheers,
g.

Re: Request review of device tree documentation

[Mitch Bradley]

Grant Likely wrote:
>
> I also changed the property in the cpu nodes from model to compatible
> so that the exact CPU version can be specified.  This isn't actually
> in any spec anywhere, but I need something to properly identify the
> different ARM cores.
>
> Mitch, I know you were working on a draft ARM binding a while ago,
> have you resurrected it at all?

As it turns out, today I re-began ARM OFW work in earnest, after a 
hiatus of something like 10 years.

I haven't thought much about a binding yet.  I'm still getting my head 
around the current state of the ARM art.

>   How do you think the core should be identified?
>   

When I was last working on ARM, few ARM devices had enough resources to 
run a general purpose OS.   To first approximation, StrongARM was the 
only game in town - so there wasn't much need to address the 
identification question.

It seems that many of the differences at the CPU level can be determined 
by looking at "coprocessor" registers.  For what purpose(s) do we need 
to identify the core?  That will inform our choice of a core ID schema.

> Cheers,
> g.
>
>
>   

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Fri, 2010-06-11 at 22:19 -1000, Mitch Bradley wrote:
> It seems that many of the differences at the CPU level can be determined 
> by looking at "coprocessor" registers.  For what purpose(s) do we need 
> to identify the core?  That will inform our choice of a core ID schema.

The primary thing I see would be architecture version compliance,
though this is better carried additionally via a binary field in
the header or a GPR at the entry point, to help the initial asm
code to setup the MMU etc... before getting into C code.

Cheers,
Ben.

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 20:45 +1000, Benjamin Herrenschmidt wrote:
> On Fri, 2010-06-11 at 22:19 -1000, Mitch Bradley wrote:
> > It seems that many of the differences at the CPU level can be determined 
> > by looking at "coprocessor" registers.  For what purpose(s) do we need 
> > to identify the core?  That will inform our choice of a core ID schema.
> 
> The primary thing I see would be architecture version compliance,
> though this is better carried additionally via a binary field in
> the header or a GPR at the entry point, to help the initial asm
> code to setup the MMU etc... before getting into C code.

Also, if you're going to revive a "real" OF port to ARM (with client
interface etc...), should we start considering moving some of powerpc's
prom_init.c to a generic place ?

IE. prom_init is a trampoline that uses the client interface to
essentially create a flatten device-tree and enter the kernel via the
common "epapr" style entry point.

The main drawback is that it doesn't allow to keep OF alive along with
the OS, but then, only sparc does that successfully and I'm not sure
it's something that would be practical to do on ARM either.

Cheers,
Ben.

Re: Request review of device tree documentation

[Mitch Bradley]

Benjamin Herrenschmidt wrote:
> On Sat, 2010-06-12 at 20:45 +1000, Benjamin Herrenschmidt wrote:
>   
>> On Fri, 2010-06-11 at 22:19 -1000, Mitch Bradley wrote:
>>     
>>> It seems that many of the differences at the CPU level can be determined 
>>> by looking at "coprocessor" registers.  For what purpose(s) do we need 
>>> to identify the core?  That will inform our choice of a core ID schema.
>>>       
>> The primary thing I see would be architecture version compliance,
>> though this is better carried additionally via a binary field in
>> the header or a GPR at the entry point, to help the initial asm
>> code to setup the MMU etc... before getting into C code.
>>     
>
> Also, if you're going to revive a "real" OF port to ARM (with client
> interface etc...), should we start considering moving some of powerpc's
> prom_init.c to a generic place ?
>
> IE. prom_init is a trampoline that uses the client interface to
> essentially create a flatten device-tree and enter the kernel via the
> common "epapr" style entry point.
>
> The main drawback is that it doesn't allow to keep OF alive along with
> the OS, but then, only sparc does that successfully and I'm not sure
> it's something that would be practical to do on ARM either.
>   

I'm certainly going to try keeping OFW alive.  On the x86 OLPC machines, 
the ability to
dive into OFW via a SysRq key combo was very helpful for debugging some 
difficult
problems.  The team has asked me to support the feature on ARM.

> Cheers,
> Ben.
>  
>
>   

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 06:30 -1000, Mitch Bradley wrote:

> I'm certainly going to try keeping OFW alive.  On the x86 OLPC machines, 
> the ability to
> dive into OFW via a SysRq key combo was very helpful for debugging some 
> difficult
> problems.  The team has asked me to support the feature on ARM.

Oh well, if you can and can convince the ARM kernel folks to do the
necessary changes ... :-)

One thing tho, you will only benefit from the whole infrastructure we
have created accross platforms in linux if the device-tree is "sucked"
into linux at boot. IE. Linux will not do constant accesses to OF for
the DT. Even sparc converted to that now.

That means that if your device-tree has a dynamic nature, we'll need to
come up with a way to inform the kernel of changes in it so it can
update it's copy accordingly.

We also don't have much mechanisms within Linux to notify users that
things have changed either. It only really happens on IBM pseries for
hotplug, which uses a private notifier.

Ben.

Re: Request review of device tree documentation

[Grant Likely]

On Sat, Jun 12, 2010 at 4:52 PM, Benjamin Herrenschmidt
<benh@kernel.crashing.org> wrote:
> On Sat, 2010-06-12 at 06:30 -1000, Mitch Bradley wrote:
>
>> I'm certainly going to try keeping OFW alive.  On the x86 OLPC machines,
>> the ability to
>> dive into OFW via a SysRq key combo was very helpful for debugging some
>> difficult
>> problems.  The team has asked me to support the feature on ARM.
>
> Oh well, if you can and can convince the ARM kernel folks to do the
> necessary changes ... :-)

What is needed to keep OFW alive?  I've got no problem with doing so
if it isn't invasive, and as long as the same boot entry interface can
be used.

> One thing tho, you will only benefit from the whole infrastructure we
> have created accross platforms in linux if the device-tree is "sucked"
> into linux at boot. IE. Linux will not do constant accesses to OF for
> the DT. Even sparc converted to that now.
>
> That means that if your device-tree has a dynamic nature, we'll need to
> come up with a way to inform the kernel of changes in it so it can
> update it's copy accordingly.

What is the use-case for having a dynamic device tree?  I can see
keeping OFW alive being useful for some debug facilities, but once the
kernel has started, I'm really not interested in relying on firmware
to manage the hardware.  (but then again it's no secret that I'm
suspicious of anything that depends on runtime interaction with
firmware).

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Request review of device tree documentation

[Mitch Bradley]

Grant Likely wrote:
> On Sat, Jun 12, 2010 at 4:52 PM, Benjamin Herrenschmidt
> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>   
>> On Sat, 2010-06-12 at 06:30 -1000, Mitch Bradley wrote:
>>
>>     
>>> I'm certainly going to try keeping OFW alive.  On the x86 OLPC machines,
>>> the ability to
>>> dive into OFW via a SysRq key combo was very helpful for debugging some
>>> difficult
>>> problems.  The team has asked me to support the feature on ARM.
>>>       
>> Oh well, if you can and can convince the ARM kernel folks to do the
>> necessary changes ... :-)
>>     
>
> What is needed to keep OFW alive?  I've got no problem with doing so
> if it isn't invasive, and as long as the same boot entry interface can
> be used.
>   

Minimally, OFW needs to own some memory that the kernel won't steal.  
OFW on ARM
is position-independent, so it can be tucked up at the top of memory 
fairly easily.

To call back into OFW, the virtual mapping for that memory needs to be 
reestablished.
Or perhaps the MMU and caches can be turned off for the duration of the 
callback.
I don't have the details of ARM MMUs and caches reloaded into my head 
yet.  Maybe next week...

Also, for debugging, OFW typically needs access to a UART.  If the OS is 
using the UART,
it's often possible for OFW to use it just by turning off interrupts and 
polling the UART.

>   
>> One thing tho, you will only benefit from the whole infrastructure we
>> have created accross platforms in linux if the device-tree is "sucked"
>> into linux at boot. IE. Linux will not do constant accesses to OF for
>> the DT. Even sparc converted to that now.
>>
>> That means that if your device-tree has a dynamic nature, we'll need to
>> come up with a way to inform the kernel of changes in it so it can
>> update it's copy accordingly.
>>     
>
> What is the use-case for having a dynamic device tree? 

The use case for a dynamic device tree is not compelling.

In SPARC / Solaris land, Open Boot managed the non-volatile 
configuration variables, which the OS could access and modify 
dynamically as properties in /options.  The OS didn't have to know the 
storage layout nor the hardware details of the storage device.  
Convenient, but not hugely important.

>  I can see
> keeping OFW alive being useful for some debug facilities, but once the
> kernel has started, I'm really not interested in relying on firmware
> to manage the hardware. 

That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the 
firmware, there is no pressure for the firmware to "get it right".

In PC land, the current status quo is that Windows depends on ACPI so 
heavily that BIOS vendors pretty much have to get that part of the 
puzzle right.  Microsoft did a thorough job of creating certification 
tests and enforcing their use.  I'm not praising ACPI, just pointing out 
the dynamics that result from assignment of responsibility.

That said, I'm not interested in pushing the issue.  It's okay with me 
if the device tree is static as far as the kernel is concerned, and 
callbacks to OFW are only used for debugging purposes.

>  (but then again it's no secret that I'm
> suspicious of anything that depends on runtime interaction with
> firmware).
>
> g.
>
>   

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 19:39 -1000, Mitch Bradley wrote:

> Minimally, OFW needs to own some memory that the kernel won't steal.  
> OFW on ARM is position-independent, so it can be tucked up at the top of memory 
> fairly easily.

Amen :-)

> To call back into OFW, the virtual mapping for that memory needs to be 
> reestablished.

That's a nasty part unless ARM provides a usable "real mode" which
allows MMIO accesses, which I -think- it does. I don't remember the
details that much.

Maybe we could define a binding tho where we could somewhat standardize
the portion of the virtual address space used by OF. IE. from the top of
the address space down to the max size it requires. It might require
some games to play with the fixmap on ARM side tho...

Another option would be something more RTAS-like where a specific call
can be done by the OS itself to 'relocate' (not physically but virtually
in this case) OF into the OS preferred location. Be prepared to have
multiple of these called though as kernels kexec into one another.

> Or perhaps the MMU and caches can be turned off for the duration of the 
> callback.
> I don't have the details of ARM MMUs and caches reloaded into my head 
> yet.  Maybe next week...

Forgot most of it too. Looks like it's about time I read the ARM
architecture again, this sounds like fun :-)

BTW. I notice no ARM list is CCed on this discussion ... maybe we should
fix that ?

> Also, for debugging, OFW typically needs access to a UART.  If the OS is 
> using the UART, it's often possible for OFW to use it just by turning off interrupts and 
> polling the UART.

That might not be a big deal unless the OS plays with the clocks which
it -does- tend to do. It might be worth defining some kind of property
OF puts in the UART node to inform the OS not to play games and keep
that one enabled, though that could affect power management, so might
need to be conditional on some nvram option (debug-enabled?)

> The use case for a dynamic device tree is not compelling.

Right, generally not, except in virtualized environments (see my other
response).

Now, the -one- thing that WILL happen if we have something like OF that
remains alive is of course vendors will try to use it as a HAL. You know
as well as I do that it -will- happen :-)

There's two reasons that typically happen. The misguided "good" one
which is to think it helps keeping a single/more maintainable kernel
image by stuffing the horrible details of nvram, rtc, etc.. access,
poweron/off GPIOs, clock control, etc... in there. The "bad" one which
is to stash code you don't want to show the source code for (codec
control, etc...).

This is bad for so many reasons that I don't think I need to even start
listing them :-) So that's something that will have to be strongly kept
in check and fought I suspect.

To some extent, in fact, doing that sort of stuff in OF or even in RTAS
like we do on power is even worse than ACPI-like tables. At least with
those tables, the interpreter is in the operating system, thus can run
with interrupts on, scheduling on, etc... With RTAS, or client interface
calls, you'll end up with potentially huge slumps of CPU time "lost"
into the bowels of the firmware.


Cheers,
Ben.

Re: Request review of device tree documentation

[Mitch Bradley]

Benjamin Herrenschmidt wrote:
> On Sat, 2010-06-12 at 19:39 -1000, Mitch Bradley wrote:
>
>   
>> Minimally, OFW needs to own some memory that the kernel won't steal.  
>> OFW on ARM is position-independent, so it can be tucked up at the top of memory 
>> fairly easily.
>>     
>
> Amen :-)
>
>   
>> To call back into OFW, the virtual mapping for that memory needs to be 
>> reestablished.
>>     
>
> That's a nasty part unless ARM provides a usable "real mode" which
> allows MMIO accesses, which I -think- it does. I don't remember the
> details that much.
>   

IIRC - and I could be wrong - ARM does have a "real mode", but the catch 
is that
you must have the MMU on in order to use the caches, to distinguish between
memory and MMIO.  So you take a fairly hefty performance hit.

I'm running my test build right now with caches off, and the performance 
is okay for
interactive work, but I'll want to have them on for startup and 
bootloading, so as not
to negatively impact the boot time.
> Maybe we could define a binding tho where we could somewhat standardize
> the portion of the virtual address space used by OF. IE. from the top of
> the address space down to the max size it requires. It might require
> some games to play with the fixmap on ARM side tho...
>   

That would be okay as far as I'm concerned.

> Another option would be something more RTAS-like where a specific call
> can be done by the OS itself to 'relocate' (not physically but virtually
> in this case) OF into the OS preferred location. Be prepared to have
> multiple of these called though as kernels kexec into one another.
>   

That might be a bit harder, but still do-able.
>   
>> Or perhaps the MMU and caches can be turned off for the duration of the 
>> callback.
>> I don't have the details of ARM MMUs and caches reloaded into my head 
>> yet.  Maybe next week...
>>     
>
> Forgot most of it too. Looks like it's about time I read the ARM
> architecture again, this sounds like fun :-)
>
> BTW. I notice no ARM list is CCed on this discussion ... maybe we should
> fix that ?
>   

Sounds like a good idea.  Do you know which list(s) would be good 
candidates?
>   
>> Also, for debugging, OFW typically needs access to a UART.  If the OS is 
>> using the UART, it's often possible for OFW to use it just by turning off interrupts and 
>> polling the UART.
>>     
>
> That might not be a big deal unless the OS plays with the clocks which
> it -does- tend to do. It might be worth defining some kind of property
> OF puts in the UART node to inform the OS not to play games and keep
> that one enabled, though that could affect power management, so might
> need to be conditional on some nvram option (debug-enabled?)
>
>   
>> The use case for a dynamic device tree is not compelling.
>>     
>
> Right, generally not, except in virtualized environments (see my other
> response).
>
> Now, the -one- thing that WILL happen if we have something like OF that
> remains alive is of course vendors will try to use it as a HAL. You know
> as well as I do that it -will- happen :-)
>   

I tried to be very clear when I was developing OFW that is is not a 
HAL.  I knew
that it would be impractical to pin down a coherent set of assumptions 
in the face
of the many different OSs - and versions of the "same" OS - that were 
extant at the time.

Digital was fairly committed to the HAL approach on Alpha, but they had 
two different
HAL ABIs, one for VMS and a different one for Ultrix!  So they were 
unable to solve
the problem for N=2, where both OSs were under their control.
> There's two reasons that typically happen. The misguided "good" one
> which is to think it helps keeping a single/more maintainable kernel
> image by stuffing the horrible details of nvram, rtc, etc.. access,
> poweron/off GPIOs, clock control, etc... in there.

Whether or not it is "misguided" depends on your cost structure.  For 
hardware companies
that don't control (and don't want to control) the OS, it is one of only 
two possible
ways to ship product.  Either you make hardware that is 100% compatible 
with something
that the OS already supports, or you have a HAL at some level.  The PC 
industry, of course,
has played both games, and by and large has been economically successful.

>  The "bad" one which
> is to stash code you don't want to show the source code for (codec
> control, etc...).
>
> This is bad for so many reasons that I don't think I need to even start
> listing them :-) So that's something that will have to be strongly kept
> in check and fought I suspect.
>   

Either fought or embraced.  To the extent that it is possible to focus 
solely on Linux and
ARM, one could image doing a good HAL.  (The reason I say ARM-only is 
because the
only other non-x86 architecture that has any "legs" left is PowerPC, and 
PPC already
has a coherent story.)
> To some extent, in fact, doing that sort of stuff in OF or even in RTAS
> like we do on power is even worse than ACPI-like tables. At least with
> those tables, the interpreter is in the operating system, thus can run
> with interrupts on, scheduling on, etc...

I have an FCode interpreter that can live inside the OS.  It's 
considerably simpler than
the ACPI interpreter.

>  With RTAS, or client interface
> calls, you'll end up with potentially huge slumps of CPU time "lost"
> into the bowels of the firmware.
>
>
> Cheers,
> Ben.
>
>
>   

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 20:45 -1000, Mitch Bradley wrote:

> Either fought or embraced.  To the extent that it is possible to focus 
> solely on Linux and ARM, one could image doing a good HAL.

That will come with a huge amount of comunity resistance sadly, but I
can imagine distros liking it.

In general, I much prefer having all the necessary native drivers in the
kernel, and the device-tree to provide the right representation, and
avoid trying to abstract "methods" via a HAL. It's the Linux philosophy
as much as possible (even when defeated by ACPI).

But if we're going to be forced by vendors into HALs, we can also make
sure that whatever they come up with is half reasonable :-)
 
> (The reason I say ARM-only is  because the
> only other non-x86 architecture that has any "legs" left is PowerPC, and 
> PPC already has a coherent story.)

Well, ppc story isn't that coherent as far as this goes :-) We don't
keep OF alive, we have RTAS which is a pile of poo... etc...

But yeah, we are fine without a HAL, so if we stick to OF as a
bootloader and provider of the device-tree, we are fine.

> > To some extent, in fact, doing that sort of stuff in OF or even in RTAS
> > like we do on power is even worse than ACPI-like tables. At least with
> > those tables, the interpreter is in the operating system, thus can run
> > with interrupts on, scheduling on, etc...
> 
> I have an FCode interpreter that can live inside the OS.  It's 
> considerably simpler than the ACPI interpreter.

That's the one thing I purposefully avoided mentioning :-)

It's an interesting idea, I've though about it. If course, there's all
sort of things to be careful about, such as who provides the f-code with
virtual->physical mappings to devices, how they are represented, how
much of the OF "forth" environment is accessible to such f-code, locking
between f-code and kernel use of shared resources (especially true when
dealing with things like i2c devices, plls, etc...).

IE. THe base idea is simple. The implementation can be a huge can of
worms.

Cheers,
Ben.

> >  With RTAS, or client interface
> > calls, you'll end up with potentially huge slumps of CPU time "lost"
> > into the bowels of the firmware.
> >
> >
> > Cheers,
> > Ben.
> >
> >
> >   

Re: Request review of device tree documentation

[Grant Likely]

On Sun, Jun 13, 2010 at 2:29 AM, Benjamin Herrenschmidt
<benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> On Sat, 2010-06-12 at 20:45 -1000, Mitch Bradley wrote:
>
>> Either fought or embraced.  To the extent that it is possible to focus
>> solely on Linux and ARM, one could image doing a good HAL.
>
> That will come with a huge amount of comunity resistance sadly, but I
> can imagine distros liking it.
>
> In general, I much prefer having all the necessary native drivers in the
> kernel, and the device-tree to provide the right representation, and
> avoid trying to abstract "methods" via a HAL. It's the Linux philosophy
> as much as possible (even when defeated by ACPI).
>
> But if we're going to be forced by vendors into HALs, we can also make
> sure that whatever they come up with is half reasonable :-)

I think there is more to be concerned about regarding binary blobs
than HALs.  Many of the new SoCs require closed binaries to use all
the hardware right now (graphics cores in particular).

Board vendors seem to be taking the plunge and modifying the kernel
rather than trying to create a HAL for driving board specific
features.

>> (The reason I say ARM-only is  because the
>> only other non-x86 architecture that has any "legs" left is PowerPC, and
>> PPC already has a coherent story.)
>
> Well, ppc story isn't that coherent as far as this goes :-) We don't
> keep OF alive, we have RTAS which is a pile of poo... etc...
>
> But yeah, we are fine without a HAL, so if we stick to OF as a
> bootloader and provider of the device-tree, we are fine.
>
>> > To some extent, in fact, doing that sort of stuff in OF or even in RTAS
>> > like we do on power is even worse than ACPI-like tables. At least with
>> > those tables, the interpreter is in the operating system, thus can run
>> > with interrupts on, scheduling on, etc...
>>
>> I have an FCode interpreter that can live inside the OS.  It's
>> considerably simpler than the ACPI interpreter.
>
> That's the one thing I purposefully avoided mentioning :-)
>
> It's an interesting idea, I've though about it. If course, there's all
> sort of things to be careful about, such as who provides the f-code with
> virtual->physical mappings to devices, how they are represented, how
> much of the OF "forth" environment is accessible to such f-code, locking
> between f-code and kernel use of shared resources (especially true when
> dealing with things like i2c devices, plls, etc...).
>
> IE. THe base idea is simple. The implementation can be a huge can of
> worms.
>
> Cheers,
> Ben.
>
>> >  With RTAS, or client interface
>> > calls, you'll end up with potentially huge slumps of CPU time "lost"
>> > into the bowels of the firmware.
>> >
>> >
>> > Cheers,
>> > Ben.
>> >
>> >
>> >
>
>
>



-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Request review of device tree documentation

[Ben Dooks]

On Sun, Jun 13, 2010 at 11:36:57PM -0600, Grant Likely wrote:
> On Sun, Jun 13, 2010 at 2:29 AM, Benjamin Herrenschmidt
> <benh@kernel.crashing.org> wrote:
> > On Sat, 2010-06-12 at 20:45 -1000, Mitch Bradley wrote:
> >
> >> Either fought or embraced.  To the extent that it is possible to focus
> >> solely on Linux and ARM, one could image doing a good HAL.
> >
> > That will come with a huge amount of comunity resistance sadly, but I
> > can imagine distros liking it.
> >
> > In general, I much prefer having all the necessary native drivers in the
> > kernel, and the device-tree to provide the right representation, and
> > avoid trying to abstract "methods" via a HAL. It's the Linux philosophy
> > as much as possible (even when defeated by ACPI).
> >
> > But if we're going to be forced by vendors into HALs, we can also make
> > sure that whatever they come up with is half reasonable :-)
> 
> I think there is more to be concerned about regarding binary blobs
> than HALs.  Many of the new SoCs require closed binaries to use all
> the hardware right now (graphics cores in particular).
> 
> Board vendors seem to be taking the plunge and modifying the kernel
> rather than trying to create a HAL for driving board specific
> features.

In my view HALs are a bad idea, they constrain you to one calling method
and make it difficult to evolve support in the kernel. I belive it is
part of the reason that we've always tried to avoid a standardised
kernel driver interface.

-- 
Ben

Q:      What's a light-year?
A:      One-third less calories than a regular year.

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 20:45 -1000, Mitch Bradley wrote:
> > BTW. I notice no ARM list is CCed on this discussion ... maybe we
> should
> > fix that ?
> >   
> 
> Sounds like a good idea.  Do you know which list(s) would be good 
> candidates?

Forgot to reply to that one ... I'd say

 linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org

For now. If the discussion starts taking too much bandwidth, it might be
worth creating a special list for ARM device-tree / OF.

Cheers,
Ben.

Re: Request review of device tree documentation

[Grant Likely]

[cc'ing linux-arm-kernel]

On Sat, Jun 12, 2010 at 11:59 PM, Benjamin Herrenschmidt
<benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> On Sat, 2010-06-12 at 19:39 -1000, Mitch Bradley wrote:
>
>> Minimally, OFW needs to own some memory that the kernel won't steal.
>> OFW on ARM is position-independent, so it can be tucked up at the top of memory
>> fairly easily.
>
> Amen :-)
>
>> To call back into OFW, the virtual mapping for that memory needs to be
>> reestablished.
>
> That's a nasty part unless ARM provides a usable "real mode" which
> allows MMIO accesses, which I -think- it does. I don't remember the
> details that much.
>
> Maybe we could define a binding tho where we could somewhat standardize
> the portion of the virtual address space used by OF. IE. from the top of
> the address space down to the max size it requires. It might require
> some games to play with the fixmap on ARM side tho...
>
> Another option would be something more RTAS-like where a specific call
> can be done by the OS itself to 'relocate' (not physically but virtually
> in this case) OF into the OS preferred location. Be prepared to have
> multiple of these called though as kernels kexec into one another.

This sounds reasonable to me.

>> Or perhaps the MMU and caches can be turned off for the duration of the
>> callback.
>> I don't have the details of ARM MMUs and caches reloaded into my head
>> yet.  Maybe next week...
>
> Forgot most of it too. Looks like it's about time I read the ARM
> architecture again, this sounds like fun :-)
>
> BTW. I notice no ARM list is CCed on this discussion ... maybe we should
> fix that ?

cc'ing linux-arm-kernel in all my replies

>> Also, for debugging, OFW typically needs access to a UART.  If the OS is
>> using the UART, it's often possible for OFW to use it just by turning off interrupts and
>> polling the UART.
>
> That might not be a big deal unless the OS plays with the clocks which
> it -does- tend to do. It might be worth defining some kind of property
> OF puts in the UART node to inform the OS not to play games and keep
> that one enabled, though that could affect power management, so might
> need to be conditional on some nvram option (debug-enabled?)
>
>> The use case for a dynamic device tree is not compelling.
>
> Right, generally not, except in virtualized environments (see my other
> response).
>
> Now, the -one- thing that WILL happen if we have something like OF that
> remains alive is of course vendors will try to use it as a HAL. You know
> as well as I do that it -will- happen :-)
>
> There's two reasons that typically happen. The misguided "good" one
> which is to think it helps keeping a single/more maintainable kernel
> image by stuffing the horrible details of nvram, rtc, etc.. access,
> poweron/off GPIOs, clock control, etc... in there. The "bad" one which
> is to stash code you don't want to show the source code for (codec
> control, etc...).
>
> This is bad for so many reasons that I don't think I need to even start
> listing them :-) So that's something that will have to be strongly kept
> in check and fought I suspect.

With a stout 2x4 if needed.  I'm going to try very hard to dissuade
this.  Yet another reason why I only what one method for passing the
device tree into the kernel.

> To some extent, in fact, doing that sort of stuff in OF or even in RTAS
> like we do on power is even worse than ACPI-like tables. At least with
> those tables, the interpreter is in the operating system, thus can run
> with interrupts on, scheduling on, etc... With RTAS, or client interface
> calls, you'll end up with potentially huge slumps of CPU time "lost"
> into the bowels of the firmware.

Unfortunately, on the newer ARM SoCs, binary blobs are all over the
place anyway.  :-(

g.

Re: Request review of device tree documentation

[Russell King - ARM Linux]

On Sun, Jun 13, 2010 at 11:23:45PM -0600, Grant Likely wrote:
> >> Or perhaps the MMU and caches can be turned off for the duration of the
> >> callback.
> >> I don't have the details of ARM MMUs and caches reloaded into my head
> >> yet.  Maybe next week...

We've had these kinds of questions in the past.  Doing what you're asking
above is not really an option - it requires:

1. disable all IRQs
2. setup 1:1 MMU mappings for code to turn off MMU
   (requires new page table)
3. disable imprecise exceptions
4. flush caches and TLBS
5. jump to 1:1 mapping area for code to disable MMU
6. disable caches and mmu
7. call function
8. flush caches and TLBs
9. re-enable caches and mmu
10. re-enable imprecise exceptions
11. switch back to original MMU mappings
12. re-enable all IRQs

This is fine if you don't care at all about interrupt latency.
Unfortunately, most people do care about interrupt latency because
that directly affects interactivity and system performance.  The
called function could not enable interrupts or exceptions - as the
CPU vectors are in virtual space, disabling the MMU effectively
makes them disappear.

Moreover, with the MMU and caches disabled, the CPU performance is
extremely poor, so the called function will run slowly.

So, disabling the MMU isn't really viable.

Now, if the external code was fully PIC, we could then run it with
the MMU enabled.  However, this wouldn't really help - the external
code could not access any devices without knowledge of how the kernel
setup the V:P translations.

So you'd need to pass some kind of data structure giving locations of
devices to the called code - but then what if the kernel doesn't have
the device mapped?

Re: Request review of device tree documentation

[Mitch Bradley]

Russell King - ARM Linux wrote:
> On Sun, Jun 13, 2010 at 11:23:45PM -0600, Grant Likely wrote:
>   
>>>> Or perhaps the MMU and caches can be turned off for the duration of the
>>>> callback.
>>>> I don't have the details of ARM MMUs and caches reloaded into my head
>>>> yet.  Maybe next week...
>>>>         
>
> We've had these kinds of questions in the past.  Doing what you're asking
> above is not really an option - it requires:
>
> 1. disable all IRQs
> 2. setup 1:1 MMU mappings for code to turn off MMU
>    (requires new page table)
> 3. disable imprecise exceptions
> 4. flush caches and TLBS
> 5. jump to 1:1 mapping area for code to disable MMU
> 6. disable caches and mmu
> 7. call function
> 8. flush caches and TLBs
> 9. re-enable caches and mmu
> 10. re-enable imprecise exceptions
> 11. switch back to original MMU mappings
> 12. re-enable all IRQs
>
> This is fine if you don't care at all about interrupt latency.
> Unfortunately, most people do care about interrupt latency because
> that directly affects interactivity and system performance.  The
> called function could not enable interrupts or exceptions - as the
> CPU vectors are in virtual space, disabling the MMU effectively
> makes them disappear.
>
> Moreover, with the MMU and caches disabled, the CPU performance is
> extremely poor, so the called function will run slowly.
>
> So, disabling the MMU isn't really viable.
>   

None of this is a deal-breaker for the kind of debugging tasks that are 
the primary use case for the callback.

> Now, if the external code was fully PIC, we could then run it with
> the MMU enabled.  However, this wouldn't really help - the external
> code could not access any devices without knowledge of how the kernel
> setup the V:P translations.
>
> So you'd need to pass some kind of data structure giving locations of
> devices to the called code - but then what if the kernel doesn't have
> the device mapped?
>
>   

Re: Request review of device tree documentation

[Russell King - ARM Linux]

On Sun, Jun 13, 2010 at 09:45:50PM -1000, Mitch Bradley wrote:
> None of this is a deal-breaker for the kind of debugging tasks that are  
> the primary use case for the callback.

Would you mind explaining what kind of tasks these callbacks will
be used for?

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Mon, 2010-06-14 at 10:25 +0100, Russell King - ARM Linux wrote:
> On Sun, Jun 13, 2010 at 09:45:50PM -1000, Mitch Bradley wrote:
> > None of this is a deal-breaker for the kind of debugging tasks that are  
> > the primary use case for the callback.
> 
> Would you mind explaining what kind of tasks these callbacks will
> be used for?

That's one of the thing I'm "touching" on in my previous reply...

(For those who didn't quite follow, the discussion here is about
allowing a real Open Firmware implementation on ARM with the feature of
leaving OF alive while the OS is up, which is something sparc does but
we never supported on ppc).

Ideally, if you keep open firmware alive, you can drop into it when the
kernel crashes for example, or in some other circumstances.

However, there's a lot of room for abuse here and I'm worried that if it
becomes widespread, we'll start seeing vendors use that as a way to do
some kind of HAL and hide various platform methods in there (clock
control, nvram, etc...).

Another option Mitch mentioned is to have the f-code interpreter (f-code
is OF tokenized forth format) in the kernel, but that doesn't completely
solve the problem of providing it with appropriate virtual mappings,
arbitrating access to HW resources, etc etc etc

OF as a FW/bootloader is great. OF alive along with the OS can be a nice
debugging tool under some circumstances but I am a bit more dubious as
to whether that's going to work out in practice. But I'd like to -not-
see it abused as some kind of HAL.

Cheers,
Ben.

Re: Request review of device tree documentation

[Russell King - ARM Linux]

On Mon, Jun 14, 2010 at 07:36:10PM +1000, Benjamin Herrenschmidt wrote:
> However, there's a lot of room for abuse here and I'm worried that if it
> becomes widespread, we'll start seeing vendors use that as a way to do
> some kind of HAL and hide various platform methods in there (clock
> control, nvram, etc...).

This is what I'm worried about too.

As I said in my first reply in this thread, calling out from the kernel
will kill performance due to the time taken to shut down the caches and
MMU, which can only be done safely with all exceptions turned off.

The only time that it can be seriously considered is if you're calling
out to reboot, shutdown or kexec.

Re: Request review of device tree documentation

[Jamie Lokier]

Russell King - ARM Linux wrote:
> On Mon, Jun 14, 2010 at 07:36:10PM +1000, Benjamin Herrenschmidt wrote:
> > However, there's a lot of room for abuse here and I'm worried that if it
> > becomes widespread, we'll start seeing vendors use that as a way to do
> > some kind of HAL and hide various platform methods in there (clock
> > control, nvram, etc...).
> 
> This is what I'm worried about too.
> 
> As I said in my first reply in this thread, calling out from the kernel
> will kill performance due to the time taken to shut down the caches and
> MMU, which can only be done safely with all exceptions turned off.

Some applications use ARM (or other "embedded"-ish CPU) as opposed to
x86 PCs, to get predictable and reasonable interrupt latency.

x86 PCs sometimes have unpredictable interrupt latency due to those
mystery interrupts that the BIOS handles and the OS can't see or
block.  It's a different cause, but let's not duplicate the symptom
where it isn't wanted.

Even if opaque firmware callouts were fast, it would be an issue with
real-time kernels if they couldn't depend on that as an auditable fact.

-- Jamie

Re: Request review of device tree documentation

[Nicolas Pitre]

On Sun, 13 Jun 2010, Grant Likely wrote:

> [cc'ing linux-arm-kernel]
> 
> On Sat, Jun 12, 2010 at 11:59 PM, Benjamin Herrenschmidt
> > BTW. I notice no ARM list is CCed on this discussion ... maybe we should
> > fix that ?
> 
> cc'ing linux-arm-kernel in all my replies

I'm afraid this won't be enough.

I'm seeing a stream of frightening crazy talk involving ARM and some 
other stuff I still can't make head and tail of.  So, before you get a 
wholesale NAK on everything from me, please I'd suggest you guys rewind 
a bit now that the ARM list is in CC and just explain what this is all 
about and why we should feel concerned.  Then maybe the ARM savvy people 
amongst us could suggest more appropriate approaches?

Thank you.


Nicolas

Re: Request review of device tree documentation

[Grant Likely]

On Mon, Jun 14, 2010 at 7:51 AM, Nicolas Pitre <nico-vtqb6HGKxmzR7s880joybQ@public.gmane.org> wrote:
> On Sun, 13 Jun 2010, Grant Likely wrote:
>
>> [cc'ing linux-arm-kernel]
>>
>> On Sat, Jun 12, 2010 at 11:59 PM, Benjamin Herrenschmidt
>> > BTW. I notice no ARM list is CCed on this discussion ... maybe we should
>> > fix that ?
>>
>> cc'ing linux-arm-kernel in all my replies
>
> I'm afraid this won't be enough.
>
> I'm seeing a stream of frightening crazy talk involving ARM and some
> other stuff I still can't make head and tail of.  So, before you get a
> wholesale NAK on everything from me, please I'd suggest you guys rewind
> a bit now that the ARM list is in CC and just explain what this is all
> about and why we should feel concerned.  Then maybe the ARM savvy people
> amongst us could suggest more appropriate approaches?

The discussion *started* with a request to review this document:

http://devicetree.org/Device_Tree_Usage

Which is in early draft form (which is why the arm list wasn't
initially cc'd. I was soliciting feedback from the current device tree
users.  A second request for review will go out after rework is done
to the document).

In one of the reply threads Mitch stated that he is working on an ARM
project that will use Open Firmware as the bootloader, and that he'd
like the ability to keep OFW available after the kernel is booted
which is something currently done on both Sparc and OLPC x86.  Mitch
will correct me if I'm made any misrepresentations here.

Conceptually I'm not opposed to allowing OFW to stay resident
providing that it does not impose new requirements on the boot
interface (the kernel would still need to be handed the flattened
representation of the device tree) and that the code to do so is well
contained in the kernel.  The devil is of course in the details on how
feasible it is to accomplish.  ARM machines with Open Firmware are
going to be the minority, so I'm not interested in doing anything
special or out of the ordinary specifically to support it.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Request review of device tree documentation

[Nicolas Pitre]

On Mon, 14 Jun 2010, Grant Likely wrote:

> The discussion *started* with a request to review this document:
> 
> http://devicetree.org/Device_Tree_Usage
> 
> Which is in early draft form (which is why the arm list wasn't
> initially cc'd. I was soliciting feedback from the current device tree
> users.  A second request for review will go out after rework is done
> to the document).

I'm therefore assuming I can safely ignore it for now then.

> In one of the reply threads Mitch stated that he is working on an ARM
> project that will use Open Firmware as the bootloader, and that he'd
> like the ability to keep OFW available after the kernel is booted
> which is something currently done on both Sparc and OLPC x86.  Mitch
> will correct me if I'm made any misrepresentations here.

OK... but what does "keep OFW available" mean? And what for?

> Conceptually I'm not opposed to allowing OFW to stay resident
> providing that it does not impose new requirements on the boot
> interface (the kernel would still need to be handed the flattened
> representation of the device tree) and that the code to do so is well
> contained in the kernel.  The devil is of course in the details on how
> feasible it is to accomplish.

Well, you'd need to tell the kernel about what memory area not to touch 
(given that memory is not in some area the kernel will touch anyway when 
it is in its early boot stage and still not smart enough to avoid it).

Then you'll need special code to perform those steps RMK already 
mentioned.  This is a bit like the low-level code for suspend/resume 
support is doing.  This is of course if I'm still guessing right about 
the whole purpose of this.

> ARM machines with Open Firmware are
> going to be the minority, so I'm not interested in doing anything
> special or out of the ordinary specifically to support it.

This certainly doesn't have to involve the core kernel.  A special 
module may even be sufficient to keep the complexity localized.  Just 
like low-level suspend/resume code is per SOC already anyway.

But again, what for?


Nicolas

Re: Request review of device tree documentation

[Grant Likely]

On Mon, Jun 14, 2010 at 9:58 AM, Nicolas Pitre <nico-vtqb6HGKxmzR7s880joybQ@public.gmane.org> wrote:
> On Mon, 14 Jun 2010, Grant Likely wrote:
>
>> The discussion *started* with a request to review this document:
>>
>> http://devicetree.org/Device_Tree_Usage
>>
>> Which is in early draft form (which is why the arm list wasn't
>> initially cc'd. I was soliciting feedback from the current device tree
>> users.  A second request for review will go out after rework is done
>> to the document).
>
> I'm therefore assuming I can safely ignore it for now then.

Correct.

I'll let Mitch answer the rest of your questions.

g.

Re: Request review of device tree documentation

[Grant Likely]

[cc'ing linux-arm-kernel because we're discussing ARM issues]

On Sat, Jun 12, 2010 at 11:39 PM, Mitch Bradley <wmb-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org> wrote:
> Grant Likely wrote:
>>
>> On Sat, Jun 12, 2010 at 4:52 PM, Benjamin Herrenschmidt
>> <benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
>>
>>>
>>> On Sat, 2010-06-12 at 06:30 -1000, Mitch Bradley wrote:
>>>
>>>
>>>>
>>>> I'm certainly going to try keeping OFW alive.  On the x86 OLPC machines,
>>>> the ability to
>>>> dive into OFW via a SysRq key combo was very helpful for debugging some
>>>> difficult
>>>> problems.  The team has asked me to support the feature on ARM.
>>>>
>>>
>>> Oh well, if you can and can convince the ARM kernel folks to do the
>>> necessary changes ... :-)
>>>
>>
>> What is needed to keep OFW alive?  I've got no problem with doing so
>> if it isn't invasive, and as long as the same boot entry interface can
>> be used.
>>
>
> Minimally, OFW needs to own some memory that the kernel won't steal.  OFW on
> ARM
> is position-independent, so it can be tucked up at the top of memory fairly
> easily.
>
> To call back into OFW, the virtual mapping for that memory needs to be
> reestablished.
> Or perhaps the MMU and caches can be turned off for the duration of the
> callback.
> I don't have the details of ARM MMUs and caches reloaded into my head yet.
>  Maybe next week...

Remapping the MMU could be hairy, but I see no issue with marking
OFW's memory as reserved.  How does OFW currently tell the OS what
memory it should not touch because OFW is using it?  Is it device tree
data or another mechanism?

> Also, for debugging, OFW typically needs access to a UART.  If the OS is
> using the UART,
> it's often possible for OFW to use it just by turning off interrupts and
> polling the UART.

This doesn't sound onerous.

>> What is the use-case for having a dynamic device tree?
>
> The use case for a dynamic device tree is not compelling.
>
> In SPARC / Solaris land, Open Boot managed the non-volatile configuration
> variables, which the OS could access and modify dynamically as properties in
> /options.  The OS didn't have to know the storage layout nor the hardware
> details of the storage device.  Convenient, but not hugely important.

I think the assumption can be made that this will not be a use case on ARM.

>>  I can see
>> keeping OFW alive being useful for some debug facilities, but once the
>> kernel has started, I'm really not interested in relying on firmware
>> to manage the hardware.
>
> That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> firmware, there is no pressure for the firmware to "get it right".

Firmware will not get it right.  Period.  There will always be
something wrong.  It is never right on PCs.  It will never be right on
the other architectures.  That goes for OSes too, but upgrading an OS
isn't as risky as upgrading firmware.  That isn't to say that it can't
be close, but every firmware feature that the OS depends on is a
feature that could force a risky firmware upgrade when the bug in it
is discovered.

I'm also convinced that the economics are all wrong for "getting it
right" when talking about firmware.  Manufactures don't care about
firmware; they care about selling boxes.  Customers don't care about
firmware, they care about the operating system (well, that's not true
either, they care about applications).  For manufactures, once it can
boot the real operating system, there is little to no incentive to
spend any more money on firmware when the money can be better spent on
either the next product or the adding features to the operating system
of the existing product.  In fact, spending money on firmware is
actually *more risky* one a product ships, because if a firmware
upgrade goes bad, then that means product returned for repair at the
factory.

For me, this leads to two conclusions;
- That the OS should have little to no dependencies on the firmware
after it is booted so that bug fixes remain entirely in the realm of
the operating system.
- That the description of the hardware (ie Device Tree or ACPI) should
be decoupled enough from firmware that bugs in the data do not force a
firmware upgrade.  The data must always be updatabie.  Even if
configuration or data is completely corrupt, it must still be simple
to recover.

Note: I'm not critiquing OFW on either of these points.  These are
just some of my base requirements when I approach system design.

> In PC land, the current status quo is that Windows depends on ACPI so
> heavily that BIOS vendors pretty much have to get that part of the puzzle
> right.  Microsoft did a thorough job of creating certification tests and
> enforcing their use.  I'm not praising ACPI, just pointing out the dynamics
> that result from assignment of responsibility.

Yet how many boards are shipped with broken ACPI tables?  Just
bringing up ACPI in the presence of a kernel developer seems to bring
about the onset of Tourette's.  Bios provides enough data to be able
to boot the operating system, but in my experience it still requires
extra drivers to be added after installation for everything to work
right.

OTOH, I'm not had to deal with ACPI personally, so I may also be
talking out of my backside on this point.  :-)

> That said, I'm not interested in pushing the issue.  It's okay with me if
> the device tree is static as far as the kernel is concerned, and callbacks
> to OFW are only used for debugging purposes.

The current intent is to only accept the flat tree representation when
booting the kernel.  So we'll need to encode the callback pointer into
the tree somewhere.

Cheers,
g.

Re: Request review of device tree documentation

[David Gibson]

On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
[sni]
> > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> > firmware, there is no pressure for the firmware to "get it right".
> 
> Firmware will not get it right.  Period.  There will always be
> something wrong.  It is never right on PCs.  It will never be right on
> the other architectures.

Yes, yes, yes.  And there is a great deal of empirical evidence to
back that assertion.

>  That goes for OSes too, but upgrading an OS
> isn't as risky as upgrading firmware.  That isn't to say that it can't
> be close, but every firmware feature that the OS depends on is a
> feature that could force a risky firmware upgrade when the bug in it
> is discovered.

Indeed.  In fact, the general rule of thumb is really "put as much as
possible into the most easily replaced layer of the stack".  This is,
incidentally, why I've always been dubious about simple firmwares
supplying a flattened device tree rather than including the device
tree template in the kernel, cuboot style.

> I'm also convinced that the economics are all wrong for "getting it
> right" when talking about firmware.  Manufactures don't care about
> firmware; they care about selling boxes.  Customers don't care about
> firmware, they care about the operating system (well, that's not true
> either, they care about applications).  For manufactures, once it can
> boot the real operating system, there is little to no incentive to
> spend any more money on firmware when the money can be better spent on
> either the next product or the adding features to the operating system
> of the existing product.  In fact, spending money on firmware is
> actually *more risky* one a product ships, because if a firmware
> upgrade goes bad, then that means product returned for repair at the
> factory.

A good analysis.  The other side of this, is that for an OS, if you
rely on the firmware to do X, it will work when the firmware gets it
right.  If you do X yourself, it will work whether or not the firmware
gets it right.  This means that if there's even one firmware you have
to deal with out there that gets X wrong, you have to do it yourself
and then there is little to no incentive to rely on firmware even in
the cases where it does get it right.  In fact there's a disincentive,
because then you have two different code paths to test and maintain.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Request review of device tree documentation

[Nicolas Pitre]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3420 bytes --]

On Mon, 14 Jun 2010, David Gibson wrote:

> On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
> [sni]
> > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> > > firmware, there is no pressure for the firmware to "get it right".
> > 
> > Firmware will not get it right.  Period.  There will always be
> > something wrong.  It is never right on PCs.  It will never be right on
> > the other architectures.
> 
> Yes, yes, yes.  And there is a great deal of empirical evidence to
> back that assertion.
> 
> >  That goes for OSes too, but upgrading an OS
> > isn't as risky as upgrading firmware.  That isn't to say that it can't
> > be close, but every firmware feature that the OS depends on is a
> > feature that could force a risky firmware upgrade when the bug in it
> > is discovered.
> 
> Indeed.  In fact, the general rule of thumb is really "put as much as
> possible into the most easily replaced layer of the stack".  This is,
> incidentally, why I've always been dubious about simple firmwares
> supplying a flattened device tree rather than including the device
> tree template in the kernel, cuboot style.

The biggest advantage, IMHO, for adding DT to ARM, is actually to 
decouple the hardware config information and the kernel.  If in the end 
the DT has to be shipped in the kernel then we're losing all this 
advantage over the current state of things on ARM which still works 
pretty well otherwise.

In the best case, the simple firmware simply has to retrieve the 
flattened device tree from flash, and pass it to the kernel just like 
some anonymous blob.  And the simple firmware only needs to provide a 
way for that DT blob to be updatable, like through an upload of a 
replacement blob that was prepared offline.  Just like a ramdisk image 
or the like.

That doesn't need to be fancier than that, and the goal of having the DT 
data tied to the hardware instead of the kernel is achieved.

> > I'm also convinced that the economics are all wrong for "getting it
> > right" when talking about firmware.  Manufactures don't care about
> > firmware; they care about selling boxes.  Customers don't care about
> > firmware, they care about the operating system (well, that's not true
> > either, they care about applications).  For manufactures, once it can
> > boot the real operating system, there is little to no incentive to
> > spend any more money on firmware when the money can be better spent on
> > either the next product or the adding features to the operating system
> > of the existing product.  In fact, spending money on firmware is
> > actually *more risky* one a product ships, because if a firmware
> > upgrade goes bad, then that means product returned for repair at the
> > factory.
> 
> A good analysis.  The other side of this, is that for an OS, if you
> rely on the firmware to do X, it will work when the firmware gets it
> right.  If you do X yourself, it will work whether or not the firmware
> gets it right.  This means that if there's even one firmware you have
> to deal with out there that gets X wrong, you have to do it yourself
> and then there is little to no incentive to rely on firmware even in
> the cases where it does get it right.  In fact there's a disincentive,
> because then you have two different code paths to test and maintain.

Amen.


Nicolas

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: Request review of device tree documentation

[Grant Likely]

On Mon, Jun 14, 2010 at 8:59 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Mon, 14 Jun 2010, David Gibson wrote:
>
>> On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
>> Indeed.  In fact, the general rule of thumb is really "put as much as
>> possible into the most easily replaced layer of the stack".  This is,
>> incidentally, why I've always been dubious about simple firmwares
>> supplying a flattened device tree rather than including the device
>> tree template in the kernel, cuboot style.
>
> The biggest advantage, IMHO, for adding DT to ARM, is actually to
> decouple the hardware config information and the kernel.  If in the end
> the DT has to be shipped in the kernel then we're losing all this
> advantage over the current state of things on ARM which still works
> pretty well otherwise.
>
> In the best case, the simple firmware simply has to retrieve the
> flattened device tree from flash, and pass it to the kernel just like
> some anonymous blob.  And the simple firmware only needs to provide a
> way for that DT blob to be updatable, like through an upload of a
> replacement blob that was prepared offline.  Just like a ramdisk image
> or the like.
>
> That doesn't need to be fancier than that, and the goal of having the DT
> data tied to the hardware instead of the kernel is achieved.

exactly right.

g.

Re: Request review of device tree documentation

[Jamie Lokier]

Nicolas Pitre wrote:
> On Mon, 14 Jun 2010, David Gibson wrote:
> 
> > On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
> > [sni]
> > > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> > > > firmware, there is no pressure for the firmware to "get it right".
> > > 
> > > Firmware will not get it right.  Period.  There will always be
> > > something wrong.  It is never right on PCs.  It will never be right on
> > > the other architectures.
> > 
> > Yes, yes, yes.  And there is a great deal of empirical evidence to
> > back that assertion.
> > 
> > >  That goes for OSes too, but upgrading an OS
> > > isn't as risky as upgrading firmware.  That isn't to say that it can't
> > > be close, but every firmware feature that the OS depends on is a
> > > feature that could force a risky firmware upgrade when the bug in it
> > > is discovered.
> > 
> > Indeed.  In fact, the general rule of thumb is really "put as much as
> > possible into the most easily replaced layer of the stack".  This is,
> > incidentally, why I've always been dubious about simple firmwares
> > supplying a flattened device tree rather than including the device
> > tree template in the kernel, cuboot style.
> 
> The biggest advantage, IMHO, for adding DT to ARM, is actually to 
> decouple the hardware config information and the kernel.  If in the end 
> the DT has to be shipped in the kernel then we're losing all this 
> advantage over the current state of things on ARM which still works 
> pretty well otherwise.
> 
> In the best case, the simple firmware simply has to retrieve the 
> flattened device tree from flash, and pass it to the kernel just like 
> some anonymous blob.  And the simple firmware only needs to provide a 
> way for that DT blob to be updatable, like through an upload of a 
> replacement blob that was prepared offline.  Just like a ramdisk image 
> or the like.
> 
> That doesn't need to be fancier than that, and the goal of having the DT 
> data tied to the hardware instead of the kernel is achieved.

Imho that puts the DT in a similar category as initrd/initramfs, from
the bootloader's point of view.  It's another blob whose address is
passed to the kernel, just like initrd.

Some bootloaders can't update blobs independently for technical
reasons, or to be minimal.

A device I'm using does kernel updates by updating the whole romfs
boot image, which contains the kernel and other auxiliary blobs used
for booting (splash screen, early irq handlers etc.) as well as the
root filesystem.

It is done that way to pack everything together in the small flash,
and because the NOR flash eraseblocks are too large relative to the
whole flash size to use separate partitions for kernel, boot
filesystem and other blobs for booting.

Dedicating a 64kiB eraseblock out of 2MB just for a small DT would be
quite wasteful.  Dedicating two to make it powerfail-safe would be
even worse.

So requiring that a bootloader can update the DT _independently_ of
everything else is a bit much for some devices.

But requiring that it's generally treated like other separate blob,
i.e. in a similar way to initrd, and the kernel image itself, seems
not unreasonable.

Like initrd, some people will find they need to compile it in to the
kernel image to fit some bootloader they can't change, or daren't risk
changing in already rolled out devices that they want to update to a
DT-using kernel.

I won't be surprised if I see some vendor SDK containing a kernel
patch to early-parse the bootloader-supplied ROMFS image to extract
"devicetree.bin.gz" at some point :-)

We can discourage that sort of thing (but not prevent it) by ensuring
the open source bootloaders support a DT blob as easily as possible.

-- Jamie

Re: Request review of device tree documentation

[Nicolas Pitre]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 4558 bytes --]

On Mon, 14 Jun 2010, Jamie Lokier wrote:

> Nicolas Pitre wrote:
> > On Mon, 14 Jun 2010, David Gibson wrote:
> > 
> > > On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
> > > [sni]
> > > > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> > > > > firmware, there is no pressure for the firmware to "get it right".
> > > > 
> > > > Firmware will not get it right.  Period.  There will always be
> > > > something wrong.  It is never right on PCs.  It will never be right on
> > > > the other architectures.
> > > 
> > > Yes, yes, yes.  And there is a great deal of empirical evidence to
> > > back that assertion.
> > > 
> > > >  That goes for OSes too, but upgrading an OS
> > > > isn't as risky as upgrading firmware.  That isn't to say that it can't
> > > > be close, but every firmware feature that the OS depends on is a
> > > > feature that could force a risky firmware upgrade when the bug in it
> > > > is discovered.
> > > 
> > > Indeed.  In fact, the general rule of thumb is really "put as much as
> > > possible into the most easily replaced layer of the stack".  This is,
> > > incidentally, why I've always been dubious about simple firmwares
> > > supplying a flattened device tree rather than including the device
> > > tree template in the kernel, cuboot style.
> > 
> > The biggest advantage, IMHO, for adding DT to ARM, is actually to 
> > decouple the hardware config information and the kernel.  If in the end 
> > the DT has to be shipped in the kernel then we're losing all this 
> > advantage over the current state of things on ARM which still works 
> > pretty well otherwise.
> > 
> > In the best case, the simple firmware simply has to retrieve the 
> > flattened device tree from flash, and pass it to the kernel just like 
> > some anonymous blob.  And the simple firmware only needs to provide a 
> > way for that DT blob to be updatable, like through an upload of a 
> > replacement blob that was prepared offline.  Just like a ramdisk image 
> > or the like.
> > 
> > That doesn't need to be fancier than that, and the goal of having the DT 
> > data tied to the hardware instead of the kernel is achieved.
> 
> Imho that puts the DT in a similar category as initrd/initramfs, from
> the bootloader's point of view.  It's another blob whose address is
> passed to the kernel, just like initrd.

Exact.

> Some bootloaders can't update blobs independently for technical
> reasons, or to be minimal.
> 
> A device I'm using does kernel updates by updating the whole romfs
> boot image, which contains the kernel and other auxiliary blobs used
> for booting (splash screen, early irq handlers etc.) as well as the
> root filesystem.
> 
> It is done that way to pack everything together in the small flash,
> and because the NOR flash eraseblocks are too large relative to the
> whole flash size to use separate partitions for kernel, boot
> filesystem and other blobs for booting.

If you already have to update everything at once already anyway, then 
the DT blob just has to be in that update.  No need to do otherwise if 
you can't do better than that for the kernel alone.

> Dedicating a 64kiB eraseblock out of 2MB just for a small DT would be
> quite wasteful.  Dedicating two to make it powerfail-safe would be
> even worse.

I hardly see how you can be power fail safe with your 
update-everything-at-once condition above anyway.

Furthermore, if the DT is updated by the bootloader, then that means 
you're not running a live system at that point.  If power fails during 
the DT update, then you simply have to re-update it when power is back 
to allow a successful boot of the kernel afterwards.

> So requiring that a bootloader can update the DT _independently_ of
> everything else is a bit much for some devices.

In my opinion, this use case you're illustrating above simply could 
continue to _not_ use DT at all.  If your NOR flash is so small that you 
cannot spare some extra erase blocks, then this is a deeply embedded 
profile the current DT-on-ARM push is not really meant for.  You would 
be much better with a minimally configured kernel with all the hardware 
info statically compiled into the kernel and get away without all the DT 
parsing code altogether, like you're already doing today.

While I think DT for ARM has advantages, I don't see us dropping the 
legacy ARM methods anytime soon, especially for existing or extremely 
constrained targets.


Nicolas

[-- Attachment #2: Type: text/plain, Size: 150 bytes --]

_______________________________________________
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev

Re: Request review of device tree documentation

[Grant Likely]

On Mon, Jun 14, 2010 at 10:23 AM, Nicolas Pitre <nico@fluxnic.net> wrote:
> On Mon, 14 Jun 2010, Jamie Lokier wrote:
>> So requiring that a bootloader can update the DT _independently_ of
>> everything else is a bit much for some devices.
>
> In my opinion, this use case you're illustrating above simply could
> continue to _not_ use DT at all.  If your NOR flash is so small that you
> cannot spare some extra erase blocks, then this is a deeply embedded
> profile the current DT-on-ARM push is not really meant for.  You would
> be much better with a minimally configured kernel with all the hardware
> info statically compiled into the kernel and get away without all the DT
> parsing code altogether, like you're already doing today.
>
> While I think DT for ARM has advantages, I don't see us dropping the
> legacy ARM methods anytime soon, especially for existing or extremely
> constrained targets.

I completely agree.

g.

Re: Request review of device tree documentation

[Grant Likely]

On Mon, Jun 14, 2010 at 10:02 AM, Jamie Lokier <jamie@shareable.org> wrote:
> Nicolas Pitre wrote:
>> On Mon, 14 Jun 2010, David Gibson wrote:
>>
>> > On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
>> > [sni]
>> > > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
>> > > > firmware, there is no pressure for the firmware to "get it right".
>> > >
>> > > Firmware will not get it right.  Period.  There will always be
>> > > something wrong.  It is never right on PCs.  It will never be right on
>> > > the other architectures.
>> >
>> > Yes, yes, yes.  And there is a great deal of empirical evidence to
>> > back that assertion.
>> >
>> > >  That goes for OSes too, but upgrading an OS
>> > > isn't as risky as upgrading firmware.  That isn't to say that it can't
>> > > be close, but every firmware feature that the OS depends on is a
>> > > feature that could force a risky firmware upgrade when the bug in it
>> > > is discovered.
>> >
>> > Indeed.  In fact, the general rule of thumb is really "put as much as
>> > possible into the most easily replaced layer of the stack".  This is,
>> > incidentally, why I've always been dubious about simple firmwares
>> > supplying a flattened device tree rather than including the device
>> > tree template in the kernel, cuboot style.
>>
>> The biggest advantage, IMHO, for adding DT to ARM, is actually to
>> decouple the hardware config information and the kernel.  If in the end
>> the DT has to be shipped in the kernel then we're losing all this
>> advantage over the current state of things on ARM which still works
>> pretty well otherwise.
>>
>> In the best case, the simple firmware simply has to retrieve the
>> flattened device tree from flash, and pass it to the kernel just like
>> some anonymous blob.  And the simple firmware only needs to provide a
>> way for that DT blob to be updatable, like through an upload of a
>> replacement blob that was prepared offline.  Just like a ramdisk image
>> or the like.
>>
>> That doesn't need to be fancier than that, and the goal of having the DT
>> data tied to the hardware instead of the kernel is achieved.
>
> Imho that puts the DT in a similar category as initrd/initramfs, from
> the bootloader's point of view.  It's another blob whose address is
> passed to the kernel, just like initrd.
>
> Some bootloaders can't update blobs independently for technical
> reasons, or to be minimal.
>
> A device I'm using does kernel updates by updating the whole romfs
> boot image, which contains the kernel and other auxiliary blobs used
> for booting (splash screen, early irq handlers etc.) as well as the
> root filesystem.
>
> It is done that way to pack everything together in the small flash,
> and because the NOR flash eraseblocks are too large relative to the
> whole flash size to use separate partitions for kernel, boot
> filesystem and other blobs for booting.

This is totally fine.  I've got no problem with a specific firmware
requiring everything (kernel,dt,rootfs) packed into a single image
file.  Packing the image can be done at OS install time (instead of
prebuilding it) if the system builder want to retain the independent
hardware configuration aspects of using the device tree.

> Dedicating a 64kiB eraseblock out of 2MB just for a small DT would be
> quite wasteful.  Dedicating two to make it powerfail-safe would be
> even worse.
>
> So requiring that a bootloader can update the DT _independently_ of
> everything else is a bit much for some devices.

Independent update of the DT is a useful feature, but it is certainly
not a hard requirement.  It's a far more likely use-case if the kernel
and DT is stored on a filesystem instead of bare NOR flash.

> But requiring that it's generally treated like other separate blob,
> i.e. in a similar way to initrd, and the kernel image itself, seems
> not unreasonable.
>
> Like initrd, some people will find they need to compile it in to the
> kernel image to fit some bootloader they can't change, or daren't risk
> changing in already rolled out devices that they want to update to a
> DT-using kernel.

Yes, I fully expect that.  Fortunately the situation is better than it
was with powerpc.  Since the machine id is being retained, a
DT-enabled kernel can continue to support non-DT systems.  There will
not be a flag day to cut everything over to a new boot interface.

g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.

Re: Request review of device tree documentation

[Jamie Lokier]

Grant Likely wrote:
> > Like initrd, some people will find they need to compile it in to the
> > kernel image to fit some bootloader they can't change, or daren't risk
> > changing in already rolled out devices that they want to update to a
> > DT-using kernel.
> 
> Yes, I fully expect that.  Fortunately the situation is better than it
> was with powerpc.  Since the machine id is being retained, a
> DT-enabled kernel can continue to support non-DT systems.  There will
> not be a flag day to cut everything over to a new boot interface.

That's excellent, thank you.  I was a bit unsure about how that was going.

-- Jamie

Re: Request review of device tree documentation

[Mitch Bradley]

I shall try to clarify this discussion.

There are actually two different things being discussed.  The first is, 
I hope, not too controversial.  The second is so controversial as to be 
a hopeless cause.

First, the primary use case for "keeping OFW alive" is for debugging 
purposes.  OFW remains resident in memory so that, if the OS is set to 
allow it (not the default), a hot-key freezes the OS and enters OFW, 
where a human can inspect the state of devices and OS data structures. A 
high skill level is required, so it's okay if some fiddling is necessary 
to find or establish virtual addresses or do similar magic .  In my 
career of working closely with hardware manufacturers, I and others have 
found this feature to be extremely helpful.  Often it has resulted in 
the resolution of difficult problems that were blocking the ability to 
ship the product - problems that resisted other kernel debugging techniques.

The second topic is the hypothetical use of OFW as a HAL. That will not 
happen for several reasons.  The opposition to the idea is widespread 
and deeply held, and there are good arguments to support that 
opposition.   Furthermore, the economic conditions necessary for the 
creation of such a HAL do not exist in the ARM world, nor indeed in the 
Linux world in general.  (The necessary condition is the ability for one 
company to impose a substantial change by fiat - essentially a monopoly 
position.)

Shall we agree, then, that any further discussion of the HAL issue is 
"just for fun", and that nobody needs to feel threatened that it would 
actually happen?

The potential for "vendors breaking out of the debugging use case and 
turning it into a HAL" is miniscule, because

a) The callback is disabled by default
b) The technical challenges of the callback interface limit its 
applicability to specific "wizard user" scenarios
c) OFW is unlikely to achieve sufficient market penetration for the HAL 
thing to be worth doing

Re: Request review of device tree documentation

[Nicolas Pitre]

On Mon, 14 Jun 2010, Mitch Bradley wrote:

> First, the primary use case for "keeping OFW alive" is for debugging purposes.
> OFW remains resident in memory so that, if the OS is set to allow it (not the
> default), a hot-key freezes the OS and enters OFW, where a human can inspect
> the state of devices and OS data structures. A high skill level is required,
> so it's okay if some fiddling is necessary to find or establish virtual
> addresses or do similar magic .  

Why would you impose such pain on yourself in order to try to make OFW a 
viable debugging tool on ARM for live kernels, while you can achieve the 
same and more much less intrusively and so much more safely with a JTAG 
based debugger?

If the cost of a JTAG solution is a concern, you can order USB based 
JTAG dongles on the net for less than $30 and use them with OpenOCD[1].

Otherwise, what's wrong with already supported kgdb, or even kdb?

[1] http://openocd.berlios.de/web/


Nicolas

Re: Request review of device tree documentation

[Mitch Bradley]

Nicolas Pitre wrote:
> On Mon, 14 Jun 2010, Mitch Bradley wrote:
>
>   
>> First, the primary use case for "keeping OFW alive" is for debugging purposes.
>> OFW remains resident in memory so that, if the OS is set to allow it (not the
>> default), a hot-key freezes the OS and enters OFW, where a human can inspect
>> the state of devices and OS data structures. A high skill level is required,
>> so it's okay if some fiddling is necessary to find or establish virtual
>> addresses or do similar magic .  
>>     
>
> Why would you impose such pain on yourself in order to try to make OFW a 
> viable debugging tool on ARM for live kernels, while you can achieve the 
> same and more much less intrusively and so much more safely with a JTAG 
> based debugger?
>
> If the cost of a JTAG solution is a concern, you can order USB based 
> JTAG dongles on the net for less than $30 and use them with OpenOCD[1].
>   

If OFW is present on the machine, when a customer reports a problem I 
can tell them
to do x and y and z and tell me what they see.  In this manner, I have 
often solved
difficult problems in minutes or hours.

Arranging for a JTAG dongle to appear at the customer site, then getting 
it set up and
the necessary software installed and configured on a suitable host 
system, typically
requires several days at best, plus potentially a lot of fiddling 
depending on what
sort of host system the customer happens to have.

The phrase "impose such pain on yourself" presupposes that the technical 
challenges
are much harder than they actually are.  In fact, most of the pain comes 
from dealing
with the "yuck, why would you ever want to do that" argument.  I first 
experienced
that argument in 1982, when Tom Lyon - Sun's Unix driver expert at the 
time - threatened
to "scratch my disk" if I ported Forth to the Sun 1 machine.  Tom later 
recanted and
said that he was very glad that I had done so, after I used it to solve 
several stop-ship
problems that came close to killing the company.

> Otherwise, what's wrong with already supported kgdb, or even kdb?
>
> [1] http://openocd.berlios.de/web/
>   

Requires setup.  The power of "it's just there, flip a switch to turn it 
on" has to be
experienced in the heat of battle to be appreciated.

The other difference is that conventional debuggers focus on the problem of
inspecting and controlling the execution of preexisting programs, instead of
on the problem of constructing quick tests to test hypotheses.  While it is
possible to use them to "poke around", it quickly becomes cumbersome if you
need to do anything more complicated than just looking.  OFW's built-in
programming language is particularly well suited for making little test 
loops
on-the-fly.   Also, OFW has drivers for most of all of the system's 
hardware, and
those drivers are independently developed from the Linux drivers.  That 
often
serves as a valuable "second opinion" to help discover the root cause of 
hardware
misbehavior.
>
> Nicolas
>
>   

Re: Request review of device tree documentation

[Nicolas Pitre]

On Mon, 14 Jun 2010, Mitch Bradley wrote:

> Nicolas Pitre wrote:
> > On Mon, 14 Jun 2010, Mitch Bradley wrote:
> > 
> >   
> > > First, the primary use case for "keeping OFW alive" is for debugging
> > > purposes.
> > > OFW remains resident in memory so that, if the OS is set to allow it (not
> > > the
> > > default), a hot-key freezes the OS and enters OFW, where a human can
> > > inspect
> > > the state of devices and OS data structures. A high skill level is
> > > required,
> > > so it's okay if some fiddling is necessary to find or establish virtual
> > > addresses or do similar magic .      
> > 
> > Why would you impose such pain on yourself in order to try to make OFW a
> > viable debugging tool on ARM for live kernels, while you can achieve the
> > same and more much less intrusively and so much more safely with a JTAG
> > based debugger?
> > 
> > If the cost of a JTAG solution is a concern, you can order USB based JTAG
> > dongles on the net for less than $30 and use them with OpenOCD[1].
> >   
> 
> If OFW is present on the machine, when a customer reports a problem I 
> can tell them to do x and y and z and tell me what they see.  In this 
> manner, I have often solved difficult problems in minutes or hours.

That's assuming OFW is still intact somewhere and unaffected by said 
problem.

> Arranging for a JTAG dongle to appear at the customer site, then 
> getting it set up and the necessary software installed and configured 
> on a suitable host system, typically requires several days at best, 
> plus potentially a lot of fiddling depending on what sort of host 
> system the customer happens to have.

Well, if I may use the SheevaPlug as an example, the actual FT2232 chip 
currently used in most of those USB-JTAG dongle was provided directly on 
the board.  So you have this standard mini-B type USB connector on the 
side of the device from which you get both a serial console and a JTAG 
interface.  All you need is a standard USB cable, just like the one you 
get with a MP3 player or a digital camera, so there are plenty of those 
around.

Software wise, people have provided self contained packages containing 
OpenOCD, the necessary recovery binary images, and a script to bind it 
all into a nice debricking utility for when you blow your flash content 
away.

Oh and OpenOCD runs on Linux, Mac OS as well as Windows.

So there are ways to customize things and make this really straight 
forward to users.  But in the SheevaPlug case this ease of use was also 
planned further by integrating easy JTAG access into the hardware 
design.  And a couple other ARM boards out there are doing the same 
thing too.

> The phrase "impose such pain on yourself" presupposes that the 
> technical challenges are much harder than they actually are.  In fact, 
> most of the pain comes from dealing with the "yuck, why would you ever 
> want to do that" argument.  I first experienced that argument in 1982, 
> when Tom Lyon - Sun's Unix driver expert at the time - threatened to 
> "scratch my disk" if I ported Forth to the Sun 1 machine.  Tom later 
> recanted and said that he was very glad that I had done so, after I 
> used it to solve several stop-ship problems that came close to killing 
> the company.

Sure. Pioneering solutions to save your life is always worth the pain.  
But in this case some solutions were already developed and in use today.  
So all you'll be doing here is sort of reinventing the wheel with the 
only major benefit that it is a wheel that you're familiar with, while 
the rest of the crowd is using another one already.

> > Otherwise, what's wrong with already supported kgdb, or even kdb?
> > 
> > [1] http://openocd.berlios.de/web/
> >   
> 
> Requires setup.  The power of "it's just there, flip a switch to turn 
> it on" has to be experienced in the heat of battle to be appreciated.

Sure... when 1) the switch does still work even after damage was 
incurred, and 2) you have someone on-site with the appropriate knowledge 
for it.

> The other difference is that conventional debuggers focus on the problem of
> inspecting and controlling the execution of preexisting programs, instead of
> on the problem of constructing quick tests to test hypotheses.  While it is
> possible to use them to "poke around", it quickly becomes cumbersome if you
> need to do anything more complicated than just looking.  OFW's built-in
> programming language is particularly well suited for making little test loops
> on-the-fly.   

Just for completeness, OpenOCD is not itself a debugger.  It is a mean 
to provide a GDB remote debugging interface amongst other things.  It 
has its own interface that can be used autonomously, and if I'm not 
mistaken there is even a web interface to it.  And OpenOCD can be 
scripted (it contains a TCL interpreter).  So you can do all sorts of 
things with it.  The most popular usage is to reflash a hosed system.

I even saw someone use a modified OpenOCD version to wait until the CPU 
entered a particular function, have it single-stepped, and get 
statistics on cache hits and misses on a per assembly instruction 
granularity.  You just can't get that sort of info with software 
solutions running on the target as that screws up the results, nor with 
an emulator as it is usually too slow to emulate some real life 
situations.

> Also, OFW has drivers for most of all of the system's hardware, and 
> those drivers are independently developed from the Linux drivers.  
> That often serves as a valuable "second opinion" to help discover the 
> root cause of hardware misbehavior.

Sure.  I think this is a valid case, although it is quite a stretch to 
have a duplicate set of drivers there "just in case" and expect them to 
take over _live_ without skewed results.  You usually want to reboot 
into that other environment to perform your validation test, not to 
hijack the hardware from under the running OS, fiddle with it, and give 
it back to the OS hoping that everything will continue to go well.

Furthermore, those independently developed drivers are not the best 
utilization of resources.  You will hardly find people willing to 
re-implement something that already exists out there.  And if they have 
to do it once, they'll do it for Linux directly.  That's why ideas such 
as using Linux as a bootloader to boot Linux are becoming more popular.  
Even U-Boot is leveraging Linux for a lot of driver code.  Otherwise 
those duplicated drivers are simple versions for bootloader purposes 
with no similar concerns about concurrency and performance you typically 
find in a full fledged OS.

There is even a trend amongst hardware vendors to converge around 
"standardized" hardware interfaces for many class of devices, so they 
even don't have any, or very little, driver development to do.

So yes, in theory, this "second opinion" from independently developed 
drivers would be quite useful.  But in practice this is rarely 
affordable.


Nicolas

Re: Request review of device tree documentation

[Mark Brown]

On Mon, Jun 14, 2010 at 03:40:19PM -0400, Nicolas Pitre wrote:
> On Mon, 14 Jun 2010, Mitch Bradley wrote:

> > Arranging for a JTAG dongle to appear at the customer site, then 
> > getting it set up and the necessary software installed and configured 
> > on a suitable host system, typically requires several days at best, 
> > plus potentially a lot of fiddling depending on what sort of host 
> > system the customer happens to have.

> Well, if I may use the SheevaPlug as an example, the actual FT2232 chip 
> currently used in most of those USB-JTAG dongle was provided directly on 
> the board.  So you have this standard mini-B type USB connector on the 
> side of the device from which you get both a serial console and a JTAG 
> interface.  All you need is a standard USB cable, just like the one you 
> get with a MP3 player or a digital camera, so there are plenty of those 
> around.

The other thing here is of course that if JTAG is a problem serial or
other console access is likely to also be a problem - it's likely to not
be physically present and at least as painful to set up as JTAG.

Re: Request review of device tree documentation

[Mike Rapoport]

Mitch Bradley wrote:

> The second topic is the hypothetical use of OFW as a HAL. That will not 
> happen for several reasons.  The opposition to the idea is widespread 
> and deeply held, and there are good arguments to support that 
> opposition.   Furthermore, the economic conditions necessary for the 
> creation of such a HAL do not exist in the ARM world, nor indeed in the 
> Linux world in general.  (The necessary condition is the ability for one 
> company to impose a substantial change by fiat - essentially a monopoly 
> position.)
> 
> Shall we agree, then, that any further discussion of the HAL issue is 
> "just for fun", and that nobody needs to feel threatened that it would 
> actually happen?

I've recently worked with vendor versions of U-Boot for advanced ARM 
SoCs. There is already *huge* chunk of HAL code in those versions. And 
if there would be possibility to have callbacks into the firmware these 
chunks would only grow, IMHO.


> The potential for "vendors breaking out of the debugging use case and 
> turning it into a HAL" is miniscule, because
> 
> a) The callback is disabled by default
> b) The technical challenges of the callback interface limit its 
> applicability to specific "wizard user" scenarios
> c) OFW is unlikely to achieve sufficient market penetration for the HAL 
> thing to be worth doing
> 
> 
> _______________________________________________
> linux-arm-kernel mailing list
> linux-arm-kernel@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel


-- 
Sincerely yours,
Mike.

Re: Request review of device tree documentation

[Mitch Bradley]

Mike Rapoport wrote:
> Mitch Bradley wrote:
>
>> The second topic is the hypothetical use of OFW as a HAL. That will 
>> not happen for several reasons.  The opposition to the idea is 
>> widespread and deeply held, and there are good arguments to support 
>> that opposition.   Furthermore, the economic conditions necessary for 
>> the creation of such a HAL do not exist in the ARM world, nor indeed 
>> in the Linux world in general.  (The necessary condition is the 
>> ability for one company to impose a substantial change by fiat - 
>> essentially a monopoly position.)
>>
>> Shall we agree, then, that any further discussion of the HAL issue is 
>> "just for fun", and that nobody needs to feel threatened that it 
>> would actually happen?
>
> I've recently worked with vendor versions of U-Boot for advanced ARM 
> SoCs. There is already *huge* chunk of HAL code in those versions. And 
> if there would be possibility to have callbacks into the firmware 
> these chunks would only grow, IMHO.

How can there be HAL code in U-Boot unless there is already the 
possibility to have callbacks into the firmware?

It is not HAL if it can't be called.

>
>
>> The potential for "vendors breaking out of the debugging use case and 
>> turning it into a HAL" is miniscule, because
>>
>> a) The callback is disabled by default
>> b) The technical challenges of the callback interface limit its 
>> applicability to specific "wizard user" scenarios
>> c) OFW is unlikely to achieve sufficient market penetration for the 
>> HAL thing to be worth doing
>>
>>
>> _______________________________________________
>> linux-arm-kernel mailing list
>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>
>

Re: Request review of device tree documentation

[Mike Rapoport]

Mitch Bradley wrote:
> Mike Rapoport wrote:
>> Mitch Bradley wrote:
>>
>>> The second topic is the hypothetical use of OFW as a HAL. That will 
>>> not happen for several reasons.  The opposition to the idea is 
>>> widespread and deeply held, and there are good arguments to support 
>>> that opposition.   Furthermore, the economic conditions necessary for 
>>> the creation of such a HAL do not exist in the ARM world, nor indeed 
>>> in the Linux world in general.  (The necessary condition is the 
>>> ability for one company to impose a substantial change by fiat - 
>>> essentially a monopoly position.)
>>>
>>> Shall we agree, then, that any further discussion of the HAL issue is 
>>> "just for fun", and that nobody needs to feel threatened that it 
>>> would actually happen?
>>
>> I've recently worked with vendor versions of U-Boot for advanced ARM 
>> SoCs. There is already *huge* chunk of HAL code in those versions. And 
>> if there would be possibility to have callbacks into the firmware 
>> these chunks would only grow, IMHO.
> 
> How can there be HAL code in U-Boot unless there is already the 
> possibility to have callbacks into the firmware?

Currently it aims to abstract hardware from U-Boot and reuse the same HW 
access code across operating systems and bootloaders. If this code would 
have callbacks I afraid the things would became worse.

> It is not HAL if it can't be called.
> 
>>
>>
>>> The potential for "vendors breaking out of the debugging use case and 
>>> turning it into a HAL" is miniscule, because
>>>
>>> a) The callback is disabled by default
>>> b) The technical challenges of the callback interface limit its 
>>> applicability to specific "wizard user" scenarios
>>> c) OFW is unlikely to achieve sufficient market penetration for the 
>>> HAL thing to be worth doing
>>>
>>>
>>> _______________________________________________
>>> linux-arm-kernel mailing list
>>> linux-arm-kernel@lists.infradead.org
>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>
>>


-- 
Sincerely yours,
Mike.

Re: Request review of device tree documentation

[Mitch Bradley]

Mike Rapoport wrote:
> Mitch Bradley wrote:
>> Mike Rapoport wrote:
>>> Mitch Bradley wrote:
>>>
>>>> The second topic is the hypothetical use of OFW as a HAL. That will 
>>>> not happen for several reasons.  The opposition to the idea is 
>>>> widespread and deeply held, and there are good arguments to support 
>>>> that opposition.   Furthermore, the economic conditions necessary 
>>>> for the creation of such a HAL do not exist in the ARM world, nor 
>>>> indeed in the Linux world in general.  (The necessary condition is 
>>>> the ability for one company to impose a substantial change by fiat 
>>>> - essentially a monopoly position.)
>>>>
>>>> Shall we agree, then, that any further discussion of the HAL issue 
>>>> is "just for fun", and that nobody needs to feel threatened that it 
>>>> would actually happen?
>>>
>>> I've recently worked with vendor versions of U-Boot for advanced ARM 
>>> SoCs. There is already *huge* chunk of HAL code in those versions. 
>>> And if there would be possibility to have callbacks into the 
>>> firmware these chunks would only grow, IMHO.
>>
>> How can there be HAL code in U-Boot unless there is already the 
>> possibility to have callbacks into the firmware?
>
> Currently it aims to abstract hardware from U-Boot and reuse the same 
> HW access code across operating systems and bootloaders. If this code 
> would have callbacks I afraid the things would became worse.

The only way I can understand what you said is if I assume that by 
"callback", you mean the following sequence:

a) U-boot loads and executes the OS, providing to the OS the address of 
some HW access routines that it can use
b) The OS calls one of those HW access routines
c) During the execution of that HW access routine, that routine calls 
"back" into the OS, before returning.  So a call into the OS is nested 
inside a call into U-boot resident code.

If that is what you are worried about, it is not what we were 
discussing.  We were discussing - and many people were against - step (b).

Are you saying that step (b) - the OS calling into routines provided by 
U-Boot - is already the status quo?

>
>> It is not HAL if it can't be called.
>>
>>>
>>>
>>>> The potential for "vendors breaking out of the debugging use case 
>>>> and turning it into a HAL" is miniscule, because
>>>>
>>>> a) The callback is disabled by default
>>>> b) The technical challenges of the callback interface limit its 
>>>> applicability to specific "wizard user" scenarios
>>>> c) OFW is unlikely to achieve sufficient market penetration for the 
>>>> HAL thing to be worth doing
>>>>
>>>>
>>>> _______________________________________________
>>>> linux-arm-kernel mailing list
>>>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>
>>>
>
>

Re: Request review of device tree documentation

[Mike Rapoport]

Mitch Bradley wrote:
> Mike Rapoport wrote:
>> Mitch Bradley wrote:
>>> Mike Rapoport wrote:
>>>> Mitch Bradley wrote:
>>>>
>>>>> The second topic is the hypothetical use of OFW as a HAL. That will 
>>>>> not happen for several reasons.  The opposition to the idea is 
>>>>> widespread and deeply held, and there are good arguments to support 
>>>>> that opposition.   Furthermore, the economic conditions necessary 
>>>>> for the creation of such a HAL do not exist in the ARM world, nor 
>>>>> indeed in the Linux world in general.  (The necessary condition is 
>>>>> the ability for one company to impose a substantial change by fiat 
>>>>> - essentially a monopoly position.)
>>>>>
>>>>> Shall we agree, then, that any further discussion of the HAL issue 
>>>>> is "just for fun", and that nobody needs to feel threatened that it 
>>>>> would actually happen?
>>>>
>>>> I've recently worked with vendor versions of U-Boot for advanced ARM 
>>>> SoCs. There is already *huge* chunk of HAL code in those versions. 
>>>> And if there would be possibility to have callbacks into the 
>>>> firmware these chunks would only grow, IMHO.
>>>
>>> How can there be HAL code in U-Boot unless there is already the 
>>> possibility to have callbacks into the firmware?
>>
>> Currently it aims to abstract hardware from U-Boot and reuse the same 
>> HW access code across operating systems and bootloaders. If this code 
>> would have callbacks I afraid the things would became worse.
> 
> The only way I can understand what you said is if I assume that by 
> "callback", you mean the following sequence:
> 
> a) U-boot loads and executes the OS, providing to the OS the address of 
> some HW access routines that it can use
> b) The OS calls one of those HW access routines
> c) During the execution of that HW access routine, that routine calls 
> "back" into the OS, before returning.  So a call into the OS is nested 
> inside a call into U-boot resident code.
> 
> If that is what you are worried about, it is not what we were 
> discussing.  We were discussing - and many people were against - step (b).
> 
> Are you saying that step (b) - the OS calling into routines provided by 
> U-Boot - is already the status quo?

I'm also objecting the step (b) and, fortunately, it's not yet the 
status quo.
Current U-Boot/kernel implementations I've encountered still do not have 
OS calls to resident HW access routines. But if such calls would be 
allowed, my impression is that SoC vendors would make extensive use of them.

>>
>>> It is not HAL if it can't be called.
>>>
>>>>
>>>>
>>>>> The potential for "vendors breaking out of the debugging use case 
>>>>> and turning it into a HAL" is miniscule, because
>>>>>
>>>>> a) The callback is disabled by default
>>>>> b) The technical challenges of the callback interface limit its 
>>>>> applicability to specific "wizard user" scenarios
>>>>> c) OFW is unlikely to achieve sufficient market penetration for the 
>>>>> HAL thing to be worth doing
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> linux-arm-kernel mailing list
>>>>> linux-arm-kernel@lists.infradead.org
>>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>>
>>>>
>>
>>


-- 
Sincerely yours,
Mike.

Re: Request review of device tree documentation

[Mitch Bradley]

Mike Rapoport wrote:
> Mitch Bradley wrote:
>> Mike Rapoport wrote:
>>> Mitch Bradley wrote:
>>>> Mike Rapoport wrote:
>>>>> Mitch Bradley wrote:
>>>>>
>>>>>> The second topic is the hypothetical use of OFW as a HAL. That 
>>>>>> will not happen for several reasons.  The opposition to the idea 
>>>>>> is widespread and deeply held, and there are good arguments to 
>>>>>> support that opposition.   Furthermore, the economic conditions 
>>>>>> necessary for the creation of such a HAL do not exist in the ARM 
>>>>>> world, nor indeed in the Linux world in general.  (The necessary 
>>>>>> condition is the ability for one company to impose a substantial 
>>>>>> change by fiat - essentially a monopoly position.)
>>>>>>
>>>>>> Shall we agree, then, that any further discussion of the HAL 
>>>>>> issue is "just for fun", and that nobody needs to feel threatened 
>>>>>> that it would actually happen?
>>>>>
>>>>> I've recently worked with vendor versions of U-Boot for advanced 
>>>>> ARM SoCs. There is already *huge* chunk of HAL code in those 
>>>>> versions. And if there would be possibility to have callbacks into 
>>>>> the firmware these chunks would only grow, IMHO.
>>>>
>>>> How can there be HAL code in U-Boot unless there is already the 
>>>> possibility to have callbacks into the firmware?
>>>
>>> Currently it aims to abstract hardware from U-Boot and reuse the 
>>> same HW access code across operating systems and bootloaders. If 
>>> this code would have callbacks I afraid the things would became worse.
>>
>> The only way I can understand what you said is if I assume that by 
>> "callback", you mean the following sequence:
>>
>> a) U-boot loads and executes the OS, providing to the OS the address 
>> of some HW access routines that it can use
>> b) The OS calls one of those HW access routines
>> c) During the execution of that HW access routine, that routine calls 
>> "back" into the OS, before returning.  So a call into the OS is 
>> nested inside a call into U-boot resident code.
>>
>> If that is what you are worried about, it is not what we were 
>> discussing.  We were discussing - and many people were against - step 
>> (b).
>>
>> Are you saying that step (b) - the OS calling into routines provided 
>> by U-Boot - is already the status quo?
>
> I'm also objecting the step (b) and, fortunately, it's not yet the 
> status quo.
> Current U-Boot/kernel implementations I've encountered still do not 
> have OS calls to resident HW access routines. But if such calls would 
> be allowed, my impression is that SoC vendors would make extensive use 
> of them.

One could argue that a feature that vendors would use extensively is one 
that is sorely needed from their point of view.

One counterargument, of course, is that "there is a better way".  But it 
is only "better" under a cost function that values things differently 
than the vendors value them.  Were that not so, the vendors would gladly 
use the "better" way and not be tempted to use the objectionable 
feature. (Unless, of course, the vendors are just ignorant or unskilled 
- but I generally find that different cost functions cause more 
disconnects than lack of ability.)

Which of course raises the question:  How does the Linux community view 
such SoC vendors?  Are they embraced and eagerly supported, or (either 
openly or secretly) viewed as a nuisance?  How does the widespread 
objection to something that such vendors "would make extensive use of" 
mesh with that view?

>
>>>
>>>> It is not HAL if it can't be called.
>>>>
>>>>>
>>>>>
>>>>>> The potential for "vendors breaking out of the debugging use case 
>>>>>> and turning it into a HAL" is miniscule, because
>>>>>>
>>>>>> a) The callback is disabled by default
>>>>>> b) The technical challenges of the callback interface limit its 
>>>>>> applicability to specific "wizard user" scenarios
>>>>>> c) OFW is unlikely to achieve sufficient market penetration for 
>>>>>> the HAL thing to be worth doing
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> linux-arm-kernel mailing list
>>>>>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
>>>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>>>
>>>>>
>>>
>>>
>
>

Re: Request review of device tree documentation

[Vladimir Pantelic]

Mitch Bradley wrote:

>>  I'm also objecting the step (b) and, fortunately, it's not yet the
>>  status quo.
>>  Current U-Boot/kernel implementations I've encountered still do not
>>  have OS calls to resident HW access routines. But if such calls would
>>  be allowed, my impression is that SoC vendors would make extensive use
>>  of them.
>
> One could argue that a feature that vendors would use extensively is one
> that is sorely needed from their point of view.

yes, it would free them from writing these pesky linux drivers and getting
them into mainline :)

Re: Request review of device tree documentation

[Mike Rapoport]

Mitch Bradley wrote:
> Mike Rapoport wrote:
>> Mitch Bradley wrote:
>>> Mike Rapoport wrote:
>>>> Mitch Bradley wrote:
>>>>> Mike Rapoport wrote:
>>>>>> Mitch Bradley wrote:
>>>>>>
>>>>>>> The second topic is the hypothetical use of OFW as a HAL. That 
>>>>>>> will not happen for several reasons.  The opposition to the idea 
>>>>>>> is widespread and deeply held, and there are good arguments to 
>>>>>>> support that opposition.   Furthermore, the economic conditions 
>>>>>>> necessary for the creation of such a HAL do not exist in the ARM 
>>>>>>> world, nor indeed in the Linux world in general.  (The necessary 
>>>>>>> condition is the ability for one company to impose a substantial 
>>>>>>> change by fiat - essentially a monopoly position.)
>>>>>>>
>>>>>>> Shall we agree, then, that any further discussion of the HAL 
>>>>>>> issue is "just for fun", and that nobody needs to feel threatened 
>>>>>>> that it would actually happen?
>>>>>>
>>>>>> I've recently worked with vendor versions of U-Boot for advanced 
>>>>>> ARM SoCs. There is already *huge* chunk of HAL code in those 
>>>>>> versions. And if there would be possibility to have callbacks into 
>>>>>> the firmware these chunks would only grow, IMHO.
>>>>>
>>>>> How can there be HAL code in U-Boot unless there is already the 
>>>>> possibility to have callbacks into the firmware?
>>>>
>>>> Currently it aims to abstract hardware from U-Boot and reuse the 
>>>> same HW access code across operating systems and bootloaders. If 
>>>> this code would have callbacks I afraid the things would became worse.
>>>
>>> The only way I can understand what you said is if I assume that by 
>>> "callback", you mean the following sequence:
>>>
>>> a) U-boot loads and executes the OS, providing to the OS the address 
>>> of some HW access routines that it can use
>>> b) The OS calls one of those HW access routines
>>> c) During the execution of that HW access routine, that routine calls 
>>> "back" into the OS, before returning.  So a call into the OS is 
>>> nested inside a call into U-boot resident code.
>>>
>>> If that is what you are worried about, it is not what we were 
>>> discussing.  We were discussing - and many people were against - step 
>>> (b).
>>>
>>> Are you saying that step (b) - the OS calling into routines provided 
>>> by U-Boot - is already the status quo?
>>
>> I'm also objecting the step (b) and, fortunately, it's not yet the 
>> status quo.
>> Current U-Boot/kernel implementations I've encountered still do not 
>> have OS calls to resident HW access routines. But if such calls would 
>> be allowed, my impression is that SoC vendors would make extensive use 
>> of them.
> 
> One could argue that a feature that vendors would use extensively is one 
> that is sorely needed from their point of view.
> 
> One counterargument, of course, is that "there is a better way".  But it 
> is only "better" under a cost function that values things differently 
> than the vendors value them.  Were that not so, the vendors would gladly 
> use the "better" way and not be tempted to use the objectionable 
> feature. (Unless, of course, the vendors are just ignorant or unskilled 
> - but I generally find that different cost functions cause more 
> disconnects than lack of ability.)

It's hardly arguable that Linux community and decision makers at chip 
vendor companies use different cost functions. But it's not that obvious 
that vendors cost functions based on well established _technical_ analysis.
Many vendors try to use the same code base for several operating 
systems. From the high level perspective it sounds logical: use the same 
core functionality and thin adaptation layers to fit the core 
functionality into several OSes. But in reality, developing the OS 
abstraction layer and "thin" adaptation layers may become more expensive 
than developing each OS support from scratch. Besides, such code becomes 
a real mess after a while, and it's maintainability is very questionable.

> Which of course raises the question:  How does the Linux community view 
> such SoC vendors?  Are they embraced and eagerly supported, or (either 
> openly or secretly) viewed as a nuisance?  How does the widespread 
> objection to something that such vendors "would make extensive use of" 
> mesh with that view?

I cannot tell for the entire Linux community, but from what I know, such 
vendors are not much welcomed in the community.

>>
>>>>
>>>>> It is not HAL if it can't be called.
>>>>>
>>>>>>
>>>>>>
>>>>>>> The potential for "vendors breaking out of the debugging use case 
>>>>>>> and turning it into a HAL" is miniscule, because
>>>>>>>
>>>>>>> a) The callback is disabled by default
>>>>>>> b) The technical challenges of the callback interface limit its 
>>>>>>> applicability to specific "wizard user" scenarios
>>>>>>> c) OFW is unlikely to achieve sufficient market penetration for 
>>>>>>> the HAL thing to be worth doing
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> linux-arm-kernel mailing list
>>>>>>> linux-arm-kernel@lists.infradead.org
>>>>>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
>>>>>>
>>>>>>
>>>>
>>>>
>>
>>


-- 
Sincerely yours,
Mike.

Re: Request review of device tree documentation

[Jamie Lokier]

Mike Rapoport wrote:
> >Which of course raises the question:  How does the Linux community view 
> >such SoC vendors?  Are they embraced and eagerly supported, or (either 
> >openly or secretly) viewed as a nuisance?  How does the widespread 
> >objection to something that such vendors "would make extensive use of" 
> >mesh with that view?
> 
> I cannot tell for the entire Linux community, but from what I know, such 
> vendors are not much welcomed in the community.

As far as I can tell, many such vendors don't have much interest in
contributing to the Linux community either.  They use Linux code, glue
in their black-box binary modules and/or HALs, and engage poorly with
the community.  It is not just end-product makers, but upstream
component, board and SDK manufacturers too.

-- Jamie

Re: Request review of device tree documentation

[Jamie Bennett]

On 16 Jun 2010, at 12:41, Jamie Lokier wrote:
> Mike Rapoport wrote:
>>> Which of course raises the question:  How does the Linux community view 
>>> such SoC vendors?  Are they embraced and eagerly supported, or (either 
>>> openly or secretly) viewed as a nuisance?  How does the widespread 
>>> objection to something that such vendors "would make extensive use of" 
>>> mesh with that view?
>> 
>> I cannot tell for the entire Linux community, but from what I know, such 
>> vendors are not much welcomed in the community.
> 
> As far as I can tell, many such vendors don't have much interest in
> contributing to the Linux community either.  They use Linux code, glue
> in their black-box binary modules and/or HALs, and engage poorly with
> the community.  It is not just end-product makers, but upstream
> component, board and SDK manufacturers too.

That is what we are trying to change [1].

> -- Jamie

Regards,
Jamie.

[1] http://www.linaro.org

Re: Request review of device tree documentation

[Nicolas Pitre]

On Wed, 16 Jun 2010, Mike Rapoport wrote:
> Mitch Bradley wrote:
> > One counterargument, of course, is that "there is a better way".  But it is
> > only "better" under a cost function that values things differently than the
> > vendors value them.  Were that not so, the vendors would gladly use the
> > "better" way and not be tempted to use the objectionable feature. (Unless,
> > of course, the vendors are just ignorant or unskilled - but I generally find
> > that different cost functions cause more disconnects than lack of ability.)
> 
> It's hardly arguable that Linux community and decision makers at chip vendor
> companies use different cost functions. But it's not that obvious that vendors
> cost functions based on well established _technical_ analysis.
> Many vendors try to use the same code base for several operating systems. From
> the high level perspective it sounds logical: use the same core functionality
> and thin adaptation layers to fit the core functionality into several OSes.
> But in reality, developing the OS abstraction layer and "thin" adaptation
> layers may become more expensive than developing each OS support from scratch.
> Besides, such code becomes a real mess after a while, and it's maintainability
> is very questionable.

The cost function _is_ different for the Linux community and decision 
makers at chip vendor companies. I know that for having worked long 
enough at a prominent chip vendor already.

Those vendors want to ship a product and be first on the market before 
the competitors do.  They grab a kernel tree, fit in their existing HAL 
as quickly as they can, so that they are able to demo the new chip to 
potential customers even before moving to full production.  And if the 
HAL fitting work has been done into last year's kernel already, then 
they will simply reuse that kernel to minimize the effort further as in 
theory only the HAL part needs to be swapped with the new one (doesn't 
matter if last year's kernel was itself already based on a kernel from 
the year before).  Once the software appears to work, they send it to 
customers and forget about it as they move to the next chip design.  So 
here, the cost is directly related to bring-up time, and quick (or big) 
ugly software hacks are more than a convenience but rather a necessity 
if you want to win the race.

People from the Linux community care about totally different things. The 
most important factor here is _long term_ maintainability.  It is 
important that the code be clean, use a uniform coding style, and follow 
common interfaces.  This is so because the cost function here is 
directly related to the difficulty for the Linux community to evolve the 
kernel with generic improvements and new features.  If each driver has a 
different style, or rely on a different HAL, then it quickly becomes 
extremely expensive to update those drivers along with the generic 
improvements.  And if the HAL is in binary form only, or stuck behind 
some unalterable BIOS-like calls or firmware API, then it may even be 
impossible to update those drivers as the execution context assumed by 
the HAL firmware may not be guaranteed anymore (think about UP vs SMP, 
different preemption modes, different realtime kernel modes, etc.)  And 
of course it is impossible to anticipate what execution context and 
requirements the kernel might need in the future, hence this can't be 
provisioned for (and much less validated) into the HAL design in advance 
(and see above why it is next to hopeless to expect vendors to update 
their HAL for old products in a timely fashion).

So there *is* indeed a big difference between both points of view.  And 
sometimes the imperatives from each group are in total conflict.

> > Which of course raises the question:  How does the Linux community view such
> > SoC vendors?  Are they embraced and eagerly supported, or (either openly or
> > secretly) viewed as a nuisance?  How does the widespread objection to
> > something that such vendors "would make extensive use of" mesh with that
> > view?
> 
> I cannot tell for the entire Linux community, but from what I know, such
> vendors are not much welcomed in the community.

And the reverse is also true: those vendors are not amenable to more 
openness and early community involvement which could help them get to 
market with "community acceptable" support for their products due to 
NDAs and such, or simply because of the perception that the Linux 
community is putting too much value on "expendable software".  It is way 
too common to hear from hardware designers that "software is cheap"... 
which is not quite the case in the end.  That's a real pity.


Nicolas

Re: Request review of device tree documentation

[Tim Bird]

On 06/16/2010 07:39 AM, Nicolas Pitre wrote:
> The cost function _is_ different for the Linux community and decision 
> makers at chip vendor companies. I know that for having worked long 
> enough at a prominent chip vendor already.
> 
> Those vendors want to ship a product and be first on the market before 
> the competitors do.  They grab a kernel tree, fit in their existing HAL 
> as quickly as they can, so that they are able to demo the new chip to 
> potential customers even before moving to full production.  And if the 
> HAL fitting work has been done into last year's kernel already, then 
> they will simply reuse that kernel to minimize the effort further as in 
> theory only the HAL part needs to be swapped with the new one (doesn't 
> matter if last year's kernel was itself already based on a kernel from 
> the year before).  Once the software appears to work, they send it to 
> customers and forget about it as they move to the next chip design.  So 
> here, the cost is directly related to bring-up time, and quick (or big) 
> ugly software hacks are more than a convenience but rather a necessity 
> if you want to win the race.
> 
> People from the Linux community care about totally different things. The 
> most important factor here is _long term_ maintainability.  It is 
> important that the code be clean, use a uniform coding style, and follow 
> common interfaces.  This is so because the cost function here is 
> directly related to the difficulty for the Linux community to evolve the 
> kernel with generic improvements and new features.  If each driver has a 
> different style, or rely on a different HAL, then it quickly becomes 
> extremely expensive to update those drivers along with the generic 
> improvements.  And if the HAL is in binary form only, or stuck behind 
> some unalterable BIOS-like calls or firmware API, then it may even be 
> impossible to update those drivers as the execution context assumed by 
> the HAL firmware may not be guaranteed anymore (think about UP vs SMP, 
> different preemption modes, different realtime kernel modes, etc.)  And 
> of course it is impossible to anticipate what execution context and 
> requirements the kernel might need in the future, hence this can't be 
> provisioned for (and much less validated) into the HAL design in advance 
> (and see above why it is next to hopeless to expect vendors to update 
> their HAL for old products in a timely fashion).
> 
> So there *is* indeed a big difference between both points of view.  And 
> sometimes the imperatives from each group are in total conflict.

This is absolutely correct.  At the CE Linux Forum, we've been working
for years to try to get vendors more involved in the community, with
a variety of techniques, including white papers for management,
magazine articles, outreach at industry events, and hosting our
own events to introduce "industry" and "community" developers.

It is not uncommon for vendors to throw the software away (or big
chunks of it) for each release, which is completely contrary
to the goals of mainline developers.  I once had a product line
where each year for 3 years, the product team used a different
processor ARCHITECTURE.  Stuff like that significantly impairs
the argument that they'll see big wins from mainlining things.

>>> Which of course raises the question:  How does the Linux community view such
>>> SoC vendors?  Are they embraced and eagerly supported,

No.  Mostly, they get chastised and told how wrong they are,
so they just stay away.   Any work they have done is often thrown
under the bus by people who refuse to acknowledge their requirements.
(Witness the suspend blockers thread on linux-pm).

>>> or (either openly or
>>> secretly) viewed as a nuisance?  How does the widespread objection to
>>> something that such vendors "would make extensive use of" mesh with that
>>> view?
>>
>> I cannot tell for the entire Linux community, but from what I know, such
>> vendors are not much welcomed in the community.
> 
> And the reverse is also true: those vendors are not amenable to more 
> openness and early community involvement which could help them get to 
> market with "community acceptable" support for their products due to 
> NDAs and such, or simply because of the perception that the Linux 
> community is putting too much value on "expendable software".  It is way 
> too common to hear from hardware designers that "software is cheap"... 
> which is not quite the case in the end.  That's a real pity.

Also true.  Nicolas - you're on a roll today. :-)
 -- Tim

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Network Entertainment
=============================

Re: Request review of device tree documentation

[M. Warner Losh]

In message: <4C187013.5000400-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org>
            Mitch Bradley <wmb-D5eQfiDGL7eakBO8gow8eQ@public.gmane.org> writes:
: Mike Rapoport wrote:
: > Mitch Bradley wrote:
: >> Mike Rapoport wrote:
: >>> Mitch Bradley wrote:
: >>>
: >>>> The second topic is the hypothetical use of OFW as a HAL. That will
: >>>> not happen for several reasons.  The opposition to the idea is
: >>>> widespread and deeply held, and there are good arguments to support
: >>>> that opposition.  Furthermore, the economic conditions necessary for
: >>>> the creation of such a HAL do not exist in the ARM world, nor indeed
: >>>> in the Linux world in general.  (The necessary condition is the
: >>>> ability for one company to impose a substantial change by fiat -
: >>>> essentially a monopoly position.)
: >>>>
: >>>> Shall we agree, then, that any further discussion of the HAL issue is
: >>>> "just for fun", and that nobody needs to feel threatened that it would
: >>>> actually happen?
: >>>
: >>> I've recently worked with vendor versions of U-Boot for advanced ARM
: >>> SoCs. There is already *huge* chunk of HAL code in those versions. And
: >>> if there would be possibility to have callbacks into the firmware
: >>> these chunks would only grow, IMHO.
: >>
: >> How can there be HAL code in U-Boot unless there is already the
: >> possibility to have callbacks into the firmware?
: >
: > Currently it aims to abstract hardware from U-Boot and reuse the same
: > HW access code across operating systems and bootloaders. If this code
: > would have callbacks I afraid the things would became worse.
: 
: The only way I can understand what you said is if I assume that by
: "callback", you mean the following sequence:
: 
: a) U-boot loads and executes the OS, providing to the OS the address
: of some HW access routines that it can use
: b) The OS calls one of those HW access routines
: c) During the execution of that HW access routine, that routine calls
: "back" into the OS, before returning.  So a call into the OS is nested
: inside a call into U-boot resident code.
: 
: If that is what you are worried about, it is not what we were
: discussing.  We were discussing - and many people were against - step
: (b).
: 
: Are you saying that step (b) - the OS calling into routines provided
: by U-Boot - is already the status quo?

I don't know about status quo, but it certainly is supported.  There's
an option to allow for a secondary boot loader, such as FreeBSD's
/boot/loader, to call back into uboot to read things from
flash/disk/whatever, do network access, etc.  Not so much a HAL, but
more of an echo of the functionality provided by PC BIOS functions.
/boot/loader can be viewed as a mini OS that calls back into uboot to
have it do things.  Once /boot/loader loads FreeBSD, btw, it and uboot
disappear from the scene, so this isn't exactly a HAL situation...

Warner


: >
: >> It is not HAL if it can't be called.
: >>
: >>>
: >>>
: >>>> The potential for "vendors breaking out of the debugging use case and
: >>>> turning it into a HAL" is miniscule, because
: >>>>
: >>>> a) The callback is disabled by default
: >>>> b) The technical challenges of the callback interface limit its
: >>>> applicability to specific "wizard user" scenarios
: >>>> c) OFW is unlikely to achieve sufficient market penetration for the
: >>>> HAL thing to be worth doing
: >>>>
: >>>>
: >>>> _______________________________________________
: >>>> linux-arm-kernel mailing list
: >>>> linux-arm-kernel-IAPFreCvJWM7uuMidbF8XUB+6BGkLq7r@public.gmane.org
: >>>> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
: >>>
: >>>
: >
: >
: _______________________________________________
: devicetree-discuss mailing list
: devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
: https://lists.ozlabs.org/listinfo/devicetree-discuss
: 
: 

Re: Request review of device tree documentation

[Frank Rowand]

On 06/15/10 23:52, M. Warner Losh wrote:
> In message: <4C187013.5000400@firmworks.com>
>             Mitch Bradley <wmb@firmworks.com> writes:
> : Mike Rapoport wrote:
> : > Mitch Bradley wrote:
> : >> Mike Rapoport wrote:
> : >>> Mitch Bradley wrote:
> : >>>
> : >>>> The second topic is the hypothetical use of OFW as a HAL. That will
> : >>>> not happen for several reasons.  The opposition to the idea is
> : >>>> widespread and deeply held, and there are good arguments to support
> : >>>> that opposition.  Furthermore, the economic conditions necessary for
> : >>>> the creation of such a HAL do not exist in the ARM world, nor indeed
> : >>>> in the Linux world in general.  (The necessary condition is the
> : >>>> ability for one company to impose a substantial change by fiat -
> : >>>> essentially a monopoly position.)
> : >>>>
> : >>>> Shall we agree, then, that any further discussion of the HAL issue is
> : >>>> "just for fun", and that nobody needs to feel threatened that it would
> : >>>> actually happen?
> : >>>
> : >>> I've recently worked with vendor versions of U-Boot for advanced ARM
> : >>> SoCs. There is already *huge* chunk of HAL code in those versions. And
> : >>> if there would be possibility to have callbacks into the firmware
> : >>> these chunks would only grow, IMHO.
> : >>
> : >> How can there be HAL code in U-Boot unless there is already the
> : >> possibility to have callbacks into the firmware?
> : >
> : > Currently it aims to abstract hardware from U-Boot and reuse the same
> : > HW access code across operating systems and bootloaders. If this code
> : > would have callbacks I afraid the things would became worse.
> : 
> : The only way I can understand what you said is if I assume that by
> : "callback", you mean the following sequence:
> : 
> : a) U-boot loads and executes the OS, providing to the OS the address
> : of some HW access routines that it can use
> : b) The OS calls one of those HW access routines
> : c) During the execution of that HW access routine, that routine calls
> : "back" into the OS, before returning.  So a call into the OS is nested
> : inside a call into U-boot resident code.
> : 
> : If that is what you are worried about, it is not what we were
> : discussing.  We were discussing - and many people were against - step
> : (b).
> : 
> : Are you saying that step (b) - the OS calling into routines provided
> : by U-Boot - is already the status quo?
> 
> I don't know about status quo, but it certainly is supported.  There's
> an option to allow for a secondary boot loader, such as FreeBSD's
> /boot/loader, to call back into uboot to read things from
> flash/disk/whatever, do network access, etc.  Not so much a HAL, but
> more of an echo of the functionality provided by PC BIOS functions.
> /boot/loader can be viewed as a mini OS that calls back into uboot to
> have it do things.  Once /boot/loader loads FreeBSD, btw, it and uboot
> disappear from the scene, so this isn't exactly a HAL situation...

Just for reference, there is a patch request on LKML to enable calling
openfirmware from the kernel on OLPC:

   http://lkml.org/lkml/2010/6/18/336

-Frank

Re: Request review of device tree documentation

[David Gibson]

On Mon, Jun 14, 2010 at 10:59:20AM -0400, Nicolas Pitre wrote:
> On Mon, 14 Jun 2010, David Gibson wrote:
> 
> > On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
> > [sni]
> > > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
> > > > firmware, there is no pressure for the firmware to "get it right".
> > > 
> > > Firmware will not get it right.  Period.  There will always be
> > > something wrong.  It is never right on PCs.  It will never be right on
> > > the other architectures.
> > 
> > Yes, yes, yes.  And there is a great deal of empirical evidence to
> > back that assertion.
> > 
> > >  That goes for OSes too, but upgrading an OS
> > > isn't as risky as upgrading firmware.  That isn't to say that it can't
> > > be close, but every firmware feature that the OS depends on is a
> > > feature that could force a risky firmware upgrade when the bug in it
> > > is discovered.
> > 
> > Indeed.  In fact, the general rule of thumb is really "put as much as
> > possible into the most easily replaced layer of the stack".  This is,
> > incidentally, why I've always been dubious about simple firmwares
> > supplying a flattened device tree rather than including the device
> > tree template in the kernel, cuboot style.
> 
> The biggest advantage, IMHO, for adding DT to ARM, is actually to 
> decouple the hardware config information and the kernel.  If in the end 
> the DT has to be shipped in the kernel then we're losing all this 
> advantage over the current state of things on ARM which still works 
> pretty well otherwise.

Right, which is why I'm just dubious, not dead against it.  If
firmware supplies a device tree that's so awful you have to replace
most of it, then you haven't won much over having a kernel wrapper
which uses ad-hoc logic to detect the board type from whatever random
clues the firmware leaves and selects a device tree from it's library
of them based on that.  On ARM this sort of approach is probably more
effective than powerpc, even, since you could use the machine number
to select from a bag of canned device trees and still have a
multi-board kernel.

> In the best case, the simple firmware simply has to retrieve the 
> flattened device tree from flash, and pass it to the kernel just like 
> some anonymous blob.  And the simple firmware only needs to provide a 
> way for that DT blob to be updatable, like through an upload of a 
> replacement blob that was prepared offline.  Just like a ramdisk image 
> or the like.

Yes, having the firmware DT independently updateable makes most of my
concerns about it go away.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Request review of device tree documentation

[M. Warner Losh]

In message: <20100614124438.GF9323@yookeroo>
            David Gibson <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> writes:
: On Sun, Jun 13, 2010 at 11:02:15PM -0600, Grant Likely wrote:
: [sni]
: > > That's sort of a self-fulfilling prophecy.  If the OS doesn't trust the
: > > firmware, there is no pressure for the firmware to "get it right".
: > 
: > Firmware will not get it right.  Period.  There will always be
: > something wrong.  It is never right on PCs.  It will never be right on
: > the other architectures.
: 
: Yes, yes, yes.  And there is a great deal of empirical evidence to
: back that assertion.

While I'll be the first to agree with that, there's also a great deal
of empirical evidence to suggest that they get it right enough often
enough.  Otherwise nothing would boot and everything would be broken.

: >  That goes for OSes too, but upgrading an OS
: > isn't as risky as upgrading firmware.  That isn't to say that it can't
: > be close, but every firmware feature that the OS depends on is a
: > feature that could force a risky firmware upgrade when the bug in it
: > is discovered.
: 
: Indeed.  In fact, the general rule of thumb is really "put as much as
: possible into the most easily replaced layer of the stack".  This is,
: incidentally, why I've always been dubious about simple firmwares
: supplying a flattened device tree rather than including the device
: tree template in the kernel, cuboot style.

The down side of cuboot style is that your kernel will boot on fewer
boards.  The firmware enables more generic kernels.

: > I'm also convinced that the economics are all wrong for "getting it
: > right" when talking about firmware.  Manufactures don't care about
: > firmware; they care about selling boxes.  Customers don't care about
: > firmware, they care about the operating system (well, that's not true
: > either, they care about applications).  For manufactures, once it can
: > boot the real operating system, there is little to no incentive to
: > spend any more money on firmware when the money can be better spent on
: > either the next product or the adding features to the operating system
: > of the existing product.  In fact, spending money on firmware is
: > actually *more risky* one a product ships, because if a firmware
: > upgrade goes bad, then that means product returned for repair at the
: > factory.
: 
: A good analysis.  The other side of this, is that for an OS, if you
: rely on the firmware to do X, it will work when the firmware gets it
: right.  If you do X yourself, it will work whether or not the firmware
: gets it right.  This means that if there's even one firmware you have
: to deal with out there that gets X wrong, you have to do it yourself
: and then there is little to no incentive to rely on firmware even in
: the cases where it does get it right.  In fact there's a disincentive,
: because then you have two different code paths to test and maintain.

Two comments: (1) You are assuming that you are in a position to do
'X' right which isn't always possible (in this case, it is impossible
to do generically, but can be done specifically for a given board if
you know enough about the board) and (2) In this case, the wrapped fdt
path uses the same path as the get the fdt from the firmware.

Warner

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 23:07 -0600, Grant Likely wrote:
> 
> What is needed to keep OFW alive?  I've got no problem with doing so
> if it isn't invasive, and as long as the same boot entry interface can
> be used.

Well, no. OF has a well defined "client interface" which is what gets us
in prom_init.c on powerpc when r5 is non NULL. You can use it to do
things like access devices or the device-tree.

We use that to suck the device-tree, which we flatten, and then re-enter
the kernel with the "common" entry interface.

If you want to keep OF alive, you -can- probably do that too, but it's
more tricky. You need to keep the memory where OF resides reserved, OF
might need special MMU translations, you maybe need to install hooks in
OF itself for it to be able to manipulate some of those, etc...

> What is the use-case for having a dynamic device tree?  I can see
> keeping OFW alive being useful for some debug facilities, but once the
> kernel has started, I'm really not interested in relying on firmware
> to manage the hardware.  (but then again it's no secret that I'm
> suspicious of anything that depends on runtime interaction with
> firmware).

As we all are :-)

I see one genuine use of dynamic device-tree though, which is in a
virtualized environment (where the host can be Linux itself). Hotplug
can then be a matter of getting new nodes & properties fed down to the
OS by the hypervisor/firmware.

pSeries somewhat works like that, but using weirdo rtas calls. We could
do something saner for kvm or whatever other hypervisors we have some
amount of control onto, where bits of compiled dtb can be handed down
representing a sub-hierarchy of the tree.

Cheers,
Ben.

Re: Request review of device tree documentation

[Grant Likely]

On Sat, Jun 12, 2010 at 11:48 PM, Benjamin Herrenschmidt
<benh-XVmvHMARGAS8U2dJNN8I7kB+6BGkLq7r@public.gmane.org> wrote:
> On Sat, 2010-06-12 at 23:07 -0600, Grant Likely wrote:
>>
>> What is needed to keep OFW alive?  I've got no problem with doing so
>> if it isn't invasive, and as long as the same boot entry interface can
>> be used.
>
> Well, no. OF has a well defined "client interface" which is what gets us
> in prom_init.c on powerpc when r5 is non NULL. You can use it to do
> things like access devices or the device-tree.
>
> We use that to suck the device-tree, which we flatten, and then re-enter
> the kernel with the "common" entry interface.

I don't think I want to do the same on ARM.  I'd rather have the
prom_init stuff in a boot wrapper, or have OFW itself generate the
flat representation before booting the kernel.  I'm trying to
constrain the number of things that could go wrong by defining only
one way for getting the device tree data into the kernel.

> If you want to keep OF alive, you -can- probably do that too, but it's
> more tricky. You need to keep the memory where OF resides reserved, OF
> might need special MMU translations, you maybe need to install hooks in
> OF itself for it to be able to manipulate some of those, etc...
>
>> What is the use-case for having a dynamic device tree?  I can see
>> keeping OFW alive being useful for some debug facilities, but once the
>> kernel has started, I'm really not interested in relying on firmware
>> to manage the hardware.  (but then again it's no secret that I'm
>> suspicious of anything that depends on runtime interaction with
>> firmware).
>
> As we all are :-)

:-)

> I see one genuine use of dynamic device-tree though, which is in a
> virtualized environment (where the host can be Linux itself). Hotplug
> can then be a matter of getting new nodes & properties fed down to the
> OS by the hypervisor/firmware.
>
> pSeries somewhat works like that, but using weirdo rtas calls. We could
> do something saner for kvm or whatever other hypervisors we have some
> amount of control onto, where bits of compiled dtb can be handed down
> representing a sub-hierarchy of the tree.

Right.  We don't need to use OFW/RTAS to handle this use case.

g.

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sun, 2010-06-13 at 23:13 -0600, Grant Likely wrote:
> > We use that to suck the device-tree, which we flatten, and then
> re-enter
> > the kernel with the "common" entry interface.
> 
> I don't think I want to do the same on ARM.  I'd rather have the
> prom_init stuff in a boot wrapper, or have OFW itself generate the
> flat representation before booting the kernel.

But then it's no longer OF. IE. A compliant OF implementation provides a
client interface API :-)

This is going to be especially important if Mitch wants to keep OF
alive.

I suppose it could be done via a wrapper like prom_init, which flattens
the tree, and sticks somewhere in a property the address of the OF
client interface callback though it's a tad awkward. If well defined, I
suppose Mitch might even be able to make his OF natively boot kernels
that way but that's of course up to him.

> I'm trying to constrain the number of things that could go wrong by
> defining only one way for getting the device tree data into the
> kernel.

I understand, and the flattened method is the most versatile, I'm just
pointing out the situation here :-)

> Right.  We don't need to use OFW/RTAS to handle this use case.

Definitely not. It will depend on whatever hypervisor interface is
implemented in a given environment. Though I do like the idea of passing
precompiled bits of .dtb around for hotplug :-) We could make that a
standard way of KVM to do things in embedded space.

Cheers,
Ben.

Re: Request review of device tree documentation

[Mitch Bradley]

Benjamin Herrenschmidt wrote:
> On Sun, 2010-06-13 at 23:13 -0600, Grant Likely wrote:
>   
>>> We use that to suck the device-tree, which we flatten, and then
>>>       
>> re-enter
>>     
>>> the kernel with the "common" entry interface.
>>>       
>> I don't think I want to do the same on ARM.  I'd rather have the
>> prom_init stuff in a boot wrapper, or have OFW itself generate the
>> flat representation before booting the kernel.
>>     
>
> But then it's no longer OF. IE. A compliant OF implementation provides a
> client interface API :-)
>
> This is going to be especially important if Mitch wants to keep OF
> alive.
>
> I suppose it could be done via a wrapper like prom_init, which flattens
> the tree, and sticks somewhere in a property the address of the OF
> client interface callback though it's a tad awkward. If well defined, I
> suppose Mitch might even be able to make his OF natively boot kernels
> that way but that's of course up to him.
>   

I'm willing to create a flattened tree.  I can provide both a client 
interface and a flattened tree.  The kernel probably won't use the 
client interface to any significant extent.

Way back in the misty annals of history, I dreamed of having a common 
interface between firmware and OSs.  That didn't happen.  Every OS 
insisted on defining its own interface and creating a custom bootloader, 
or in some cases a half dozen of them.
>   
>> I'm trying to constrain the number of things that could go wrong by
>> defining only one way for getting the device tree data into the
>> kernel.
>>     
>
> I understand, and the flattened method is the most versatile, I'm just
> pointing out the situation here :-)
>
>   
>> Right.  We don't need to use OFW/RTAS to handle this use case.
>>     
>
> Definitely not. It will depend on whatever hypervisor interface is
> implemented in a given environment. Though I do like the idea of passing
> precompiled bits of .dtb around for hotplug :-) We could make that a
> standard way of KVM to do things in embedded space.
>
> Cheers,
> Ben.
>
>
>   

Re: Request review of device tree documentation

[Olof Johansson]

On Sat, Jun 12, 2010 at 12:53:59AM -0600, Grant Likely wrote:

> I also changed the property in the cpu nodes from model to compatible
> so that the exact CPU version can be specified.  This isn't actually
> in any spec anywhere, but I need something to properly identify the
> different ARM cores.

I don't see why this has to be in the device tree. It's not there on PPC.
The exact CPU version is found through the PVR register on PPC, ARM also
has cpu version and feature registers for runtime probing.


-Olof

Re: Request review of device tree documentation

[Grant Likely]

On Sat, Jun 12, 2010 at 4:15 PM, Olof Johansson <olof@lixom.net> wrote:
> On Sat, Jun 12, 2010 at 12:53:59AM -0600, Grant Likely wrote:
>
>> I also changed the property in the cpu nodes from model to compatible
>> so that the exact CPU version can be specified.  This isn't actually
>> in any spec anywhere, but I need something to properly identify the
>> different ARM cores.
>
> I don't see why this has to be in the device tree. It's not there on PPC.
> The exact CPU version is found through the PVR register on PPC, ARM also
> has cpu version and feature registers for runtime probing.

I had been told that they runtime registers weren't reliable.  If I'm
misinformed, then yes the compatible value can be dropped from the cpu
nodes.

g.

Re: [microblaze-uclinux] Re: Request review of device tree documentation

[Edgar E. Iglesias]

On Sat, Jun 12, 2010 at 05:09:36PM -0600, Grant Likely wrote:
> On Sat, Jun 12, 2010 at 4:15 PM, Olof Johansson <olof@lixom.net> wrote:
> > On Sat, Jun 12, 2010 at 12:53:59AM -0600, Grant Likely wrote:
> >
> >> I also changed the property in the cpu nodes from model to compatible
> >> so that the exact CPU version can be specified.  This isn't actually
> >> in any spec anywhere, but I need something to properly identify the
> >> different ARM cores.
> >
> > I don't see why this has to be in the device tree. It's not there on PPC.
> > The exact CPU version is found through the PVR register on PPC, ARM also
> > has cpu version and feature registers for runtime probing.
> 
> I had been told that they runtime registers weren't reliable.  If I'm
> misinformed, then yes the compatible value can be dropped from the cpu
> nodes.

Hi all,

If it's not too painful to keep, I think it will be valuable if one
want's to instantiate HW based/from this same device tree. For example
for emulators.

Cheers

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Fri, 2010-06-11 at 16:59 -0600, Grant Likely wrote:
> I've been doing a bit of work on some introductory level documentation
> of the flattened device tree.  I've got a rough copy up on the
> devicetree.org wiki, and I could use some feedback.  If anyone has
> some time to look at it, you can find it here:
> 
> http://devicetree.org/Device_Tree_Usage

Quite nice. Maybe the introduction could use a very  quick blurb on the
various data types that dtc supports for properties, and something on
labels & phandles (references to nodes).

I just flew over it. I'll try to give you a more in depth review later.

Cheers,
Ben.

Re: Request review of device tree documentation

[Benjamin Herrenschmidt]

On Sat, 2010-06-12 at 13:00 +1000, Benjamin Herrenschmidt wrote:
> 
> Quite nice. Maybe the introduction could use a very  quick blurb on
> the
> various data types that dtc supports for properties, and something on
> labels & phandles (references to nodes).
> 
> I just flew over it. I'll try to give you a more in depth review
> later.

Oh, yes, btw, Dan is right, you do seem to mixup "model" and
"compatible" all over the place :-)

Maybe a paragraph on the new proposed clock binding that Jeremy is
working would be of use btw.

Cheers,
Ben.

Re: Request review of device tree documentation

[Jeremy Kerr]

hi Ben,

> Maybe a paragraph on the new proposed clock binding that Jeremy is
> working would be of use btw.

Here's one I prepared earlier:

http://devicetree.org/ClockBindings

:)

Cheers,


Jeremy

Re: Request review of device tree documentation

[Grant Likely]

On Sun, Jun 13, 2010 at 7:12 AM, Jeremy Kerr <jeremy.kerr@canonical.com> wrote:
> hi Ben,
>
>> Maybe a paragraph on the new proposed clock binding that Jeremy is
>> working would be of use btw.
>
> Here's one I prepared earlier:
>
> http://devicetree.org/ClockBindings

Yup, but the documents have difference purposes.  ClockBindings is the
formal description.  Device_Tree_Usage leads someone step by step how
everything actually fits together.  I think it would be useful to add
a clock binding section, especially with how important it is shaping
up to be for the ARM SoC work we're doing.

g.

Re: Request review of device tree documentation

[David Gibson]

On Fri, Jun 11, 2010 at 04:59:46PM -0600, Grant Likely wrote:
> I've been doing a bit of work on some introductory level documentation
> of the flattened device tree.  I've got a rough copy up on the
> devicetree.org wiki, and I could use some feedback.  If anyone has
> some time to look at it, you can find it here:
> 
> http://devicetree.org/Device_Tree_Usage

Sorry I haven't replied sooner, I've been away, then sick and
generally preoccupied.  Still here are some comments now.

How Addressing Works:

 * Small inconsistency you use "address1", "address2" then "unit-address3".

 * Perhaps re-emphasise that a parent's #*-cells properties govern the
   children's reg properties, not its own, since this is a common
   misunderstanding..

Non Memory Mapped Devices:

 * Your phrasing here suggests that non-memory-maped == zero
   size-cells, which is not always true.

Ranges (Address Translation):

 * Third paragraph, first sentence is a grammatical dogs' breakfast,

How Interrupts Work:

 * Bogus paragraph break partway through first sentence.

 * At the end you say the second cell indicates the interrupt's
   polarity, but you don't specify how this is encoded.  It might be
   worth emphasising that while most interrupt specifiers do include
   trigger and polarity type information, the encoding of it can and
   does vary between interrupt controllers.

Advanced Sample Machine:

 * The unit address in the name shouldn't have a "0x" prefix

Advanced Interrupt Mapping:

 * Perhaps worth noting that while a PCI *card* will use INTA..INTD,
   on-board PCI devices can, and frequently do, have interrupts wired
   side-band to the PCI bus, directly to the main interrupt
   controller.

 * In your example, you're muddying the waters of your previous usage
   of interrupt-parent.  The PCI child nodes have the PCI top-level
   node as their implicit interrupt parent, because its their first
   ancestor with an interrupt-map, and we hit that before the
   interrupt-parent property specified at the very top level.  This
   means amongst other things that if there are PCI devices with
   seperately wired interrupts, they must explicitly set
   interrupt-parent to bypass the normal PCI interrupt mapping.


-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

Re: Request review of device tree documentation

[Grant Likely]

On Thu, Aug 05, 2010 at 02:43:25PM +1000, David Gibson wrote:
> On Fri, Jun 11, 2010 at 04:59:46PM -0600, Grant Likely wrote:
> > I've been doing a bit of work on some introductory level documentation
> > of the flattened device tree.  I've got a rough copy up on the
> > devicetree.org wiki, and I could use some feedback.  If anyone has
> > some time to look at it, you can find it here:
> > 
> > http://devicetree.org/Device_Tree_Usage
> 
> Sorry I haven't replied sooner, I've been away, then sick and
> generally preoccupied.  Still here are some comments now.

Thanks David.  Reworked as per comments.  You can see the diff here:

http://www.devicetree.org/mediawiki/index.php?title=Device_Tree_Usage&diff=228&oldid=227

g.

> 
> How Addressing Works:
> 
>  * Small inconsistency you use "address1", "address2" then "unit-address3".
> 
>  * Perhaps re-emphasise that a parent's #*-cells properties govern the
>    children's reg properties, not its own, since this is a common
>    misunderstanding..
> 
> Non Memory Mapped Devices:
> 
>  * Your phrasing here suggests that non-memory-maped == zero
>    size-cells, which is not always true.
> 
> Ranges (Address Translation):
> 
>  * Third paragraph, first sentence is a grammatical dogs' breakfast,
> 
> How Interrupts Work:
> 
>  * Bogus paragraph break partway through first sentence.
> 
>  * At the end you say the second cell indicates the interrupt's
>    polarity, but you don't specify how this is encoded.  It might be
>    worth emphasising that while most interrupt specifiers do include
>    trigger and polarity type information, the encoding of it can and
>    does vary between interrupt controllers.
> 
> Advanced Sample Machine:
> 
>  * The unit address in the name shouldn't have a "0x" prefix
> 
> Advanced Interrupt Mapping:
> 
>  * Perhaps worth noting that while a PCI *card* will use INTA..INTD,
>    on-board PCI devices can, and frequently do, have interrupts wired
>    side-band to the PCI bus, directly to the main interrupt
>    controller.
> 
>  * In your example, you're muddying the waters of your previous usage
>    of interrupt-parent.  The PCI child nodes have the PCI top-level
>    node as their implicit interrupt parent, because its their first
>    ancestor with an interrupt-map, and we hit that before the
>    interrupt-parent property specified at the very top level.  This
>    means amongst other things that if there are PCI devices with
>    seperately wired interrupts, they must explicitly set
>    interrupt-parent to bypass the normal PCI interrupt mapping.
> 
> 
> -- 
> David Gibson			| I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
> 				| _way_ _around_!
> http://www.ozlabs.org/~dgibson

Re: Request review of device tree documentation

[Grant Likely]

[cc'ing the mailing list as a call for help]

On Sun, Jun 13, 2010 at 2:59 AM, Stephan Gatzka <stephan-tNItQxeJkt8dnm+yROfE0A@public.gmane.org> wrote:
> Hi Grant,
>
> I just startet to edit a PCI example for the device tree documentation (it's
> a draft).
>
> It would be very helpful to have a literature reference tag <ref>bla
> bla</ref> like wikipedia. Is that possible?

Probably.  How do I add that?  :-)  I've got no problem with it, but
I'm not an expert with mediawiki so it will take me a while to figure
it out.

Is there anyone out there familiar with mediawiki who would like to
help me maintain the site?

Cheers,
g.

Re: Request review of device tree documentation

[Grant Likely]

On Sun, Jun 13, 2010 at 2:03 PM, Grant Likely <grant.likely-s3s/WqlpOiPyB63q8FvJNQ@public.gmane.org> wrote:
> [cc'ing the mailing list as a call for help]
>
> On Sun, Jun 13, 2010 at 2:59 AM, Stephan Gatzka <stephan-tNItQxeJkt8dnm+yROfE0A@public.gmane.org> wrote:
>> Hi Grant,
>>
>> I just startet to edit a PCI example for the device tree documentation (it's
>> a draft).
>>
>> It would be very helpful to have a literature reference tag <ref>bla
>> bla</ref> like wikipedia. Is that possible?
>
> Probably.  How do I add that?  :-)  I've got no problem with it, but
> I'm not an expert with mediawiki so it will take me a while to figure
> it out.

Well, that was easy.  I just added the Cite.php extension to the site.
 <ref> tags are now available.

g.

Re: Request review of device tree documentation

[Stephan Gatzka]

[-- Attachment #1.1: Type: text/plain, Size: 187 bytes --]

Hi Grant,

> Well, that was easy.  I just added the Cite.php extension to the site.
>   <ref>  tags are now available.
>
> g.

Thanks, that really helps.

Regards,

Stephan


[-- Attachment #1.2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 5136 bytes --]

[-- Attachment #2: Type: text/plain, Size: 192 bytes --]

_______________________________________________
devicetree-discuss mailing list
devicetree-discuss-uLR06cmDAlY/bJ5BZ2RsiQ@public.gmane.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: Request review of device tree documentation

[Grant Likely]

On Sat, Jun 12, 2010 at 11:33 AM, Stephan Gatzka <stephan-tNItQxeJkt8dnm+yROfE0A@public.gmane.org> wrote:
> Hi Grant,
>
>> I've been doing a bit of work on some introductory level documentation
>> of the flattened device tree.  I've got a rough copy up on the
>> devicetree.org wiki, and I could use some feedback.  If anyone has
>> some time to look at it, you can find it here:
>>
>> http://devicetree.org/Device_Tree_Usage
>>
>> Thanks,
>> g.
>>
>
> this looks good. Maybe an example of a complete host/PCI bridge might be
> helpful. Probably I can write something during the next week.

Hi Stephan,

I see that you started drafting some of this on the wiki.  Thanks for
the draft you've done so far.  Some comments:

- Instead, of using an MPC5200 example, add a pci bus to the sample
Coyote's Revenge system used in the rest of the page and describe
that.  The goal of this document is to lead a user step-by-step how
each part of the device tree works.  So, instead of plopping down the
complete PCI bus node, the document should gradually build it up, and
talk about each element as it is added.  Focus on how it all works
together.

- It would be userful to also show a PCI-to-PCI bridge, and maybe a
fixed PCI device as children of the host bridge node.

- The current PCI nodes on all powerpc boards depend on a device_type
= "PCI" property, but I'd like to look at moving away from that for
new PCI controllers (device_type describes facilities in real open
firmware.  It shouldn't have any meaning in the flattened device
tree).  I need to look into the details though to see if it is
feasible or not.

- Describing the interrupt-map property will be particularly fiddly.
It could have a section all to itself before you even get to talking
about PCI irq swizzling.

Thanks again for the help!
g.