Xilinx devicetrees

Xilinx devicetrees

[David H. Lynch Jr.]

    I am following developments regarding device trees for xilinx boards
both here and on the microblaze list.

    I am trying to get a grasp on what they will really do for me and
what using them will demand.

    Please correct any misperceptions:

     As I understand it devicetrees are basically just a tree structured
binary  database describing the hardware.
    They have some heritage in OpenFirmware.
    There are tools to convert  some human readable representations into
the binary form.
    There appear to be tools to take xilinx firmware projects and create
a devicetree database from it
    A BSP using devicetree's configures its hardware, drivers etc, by
querying the devicetree database.  
    It it possible to pass the device tree database independent of the
kernel itself some what similar to the way many bootloaders pass initrd
filesystems.
   
    So in the end I write a BSP that could support a wide variety of
hardware and compile a single kernel that could be passed different
devicetree databases representing different xilinx firmware, and still
hope to work.
    But in return for making the BSP more generic (sort of), I now have
to somehow get the correct devicetree database passed for each different
firmware set that I load.

    I am having some difficulty grasping the significant advantages to
this.
    If the firmware for a given target is not fairly static - and I load
different firmware depending on what I am doing all the time, then I
know have to manage both a bit file for the firmware, and a devicetree
file describing it to the kernel.
    Currently for base hardware we maintain as much design consistancy
as possible accross all our different cards/firmware.
    particular hardware/firmware blocks/IP's may or may not be present -
but if present they are always the same - the Same Uartlite at the same
location, on the same irq, same for PIC's, TEMAC's ...
    For the most part it makes the most sense for us to use code to
detect the presence/absence of specific baseline hardware and then to
load non-base custom drivers after boot.

    What am  missing about devicetrees that would make me more
interested in them ?
      
  




-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Xilinx devicetrees

[Grant Likely]

On 11/24/07, David H. Lynch Jr. <dhlii@dlasys.net> wrote:
>     I am following developments regarding device trees for xilinx boards
> both here and on the microblaze list.
>
>     I am trying to get a grasp on what they will really do for me and
> what using them will demand.
>
>     Please correct any misperceptions:
>
>      As I understand it devicetrees are basically just a tree structured
> binary  database describing the hardware.
>     They have some heritage in OpenFirmware.
>     There are tools to convert  some human readable representations into
> the binary form.
>     There appear to be tools to take xilinx firmware projects and create
> a devicetree database from it
>     A BSP using devicetree's configures its hardware, drivers etc, by
> querying the devicetree database.
>     It it possible to pass the device tree database independent of the
> kernel itself some what similar to the way many bootloaders pass initrd
> filesystems.

Yes, you are correct in all of the above.

One more point; it is also possible to bind the device tree up with
the kernel so you've got a single image just like old times.  :-)

>
>     So in the end I write a BSP that could support a wide variety of
> hardware and compile a single kernel that could be passed different
> devicetree databases representing different xilinx firmware, and still
> hope to work.
>     But in return for making the BSP more generic (sort of), I now have
> to somehow get the correct devicetree database passed for each different
> firmware set that I load.

Yes; either by changes the device tree blob; or having a different
kernel+device tree image for each FPGA bitstream.  In the later case,
the kernel can be compiled once and then bound to multiple dt blobs
(creating multiple images)

>
>     I am having some difficulty grasping the significant advantages to
> this.
>     If the firmware for a given target is not fairly static - and I load
> different firmware depending on what I am doing all the time, then I
> know have to manage both a bit file for the firmware, and a devicetree
> file describing it to the kernel.
>     Currently for base hardware we maintain as much design consistancy
> as possible accross all our different cards/firmware.
>     particular hardware/firmware blocks/IP's may or may not be present -
> but if present they are always the same - the Same Uartlite at the same
> location, on the same irq, same for PIC's, TEMAC's ...
>     For the most part it makes the most sense for us to use code to
> detect the presence/absence of specific baseline hardware and then to
> load non-base custom drivers after boot.

The board description has to live *somewhere*.  For powerpc (and
microblaze) we've decided that for generic stuff, it makes sense to
use the device tree data structure to describe the hardware.  It makes
the platform code simpler because the platform code no longer needs to
explicitly instantiate drivers for each device on the board.  Instead
it registers part of the device tree with the of_platform bus and lets
the drivers handle binding themselves.

That being said, using the device tree does not preclude using
platform code to setup devices *where it makes sense to do so*.  Many
things are one-off board specific things that are not well described
with the device tree.  For example, we've been debating how to handle
on board sound which use common codec chips, but have custom wireup.
The codec chip can use a common driver, but the wireup is handled with
an ALSA 'fabric' driver that is pretty much a one-off for the board.
It probably makes more sense for the fabric driver to be instantiated
by the platform code rather than trying to do a full device tree
representation.

>
>     What am  missing about devicetrees that would make me more
> interested in them ?

To put it all in perspective, for Virtex support in arch/ppc
registration of devices is data driven.  Look at
arch/ppc/syslib/virtex_devices.c which contains a table of platform
devices which the Virtex platform code iterates over.  In arch/powerpc
we're *still* data driven; it's just that the data is no longer
compiled into a static structure.  Plus, in the transition we've moved
from needed per-device platform data structures to a more expressive
format.  Also, per-board platform support code has become much simpler
(compare ml300.c in arch/ppc with platforms/40x/virtex.c)

Cheers,
g.


-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

RE: Xilinx devicetrees

[Stephen Neuendorffer]

[-- Attachment #1: Type: text/plain, Size: 2181 bytes --]




-----Original Message-----
From: linuxppc-embedded-bounces+stephen=neuendorffer.name@ozlabs.org on behalf of Grant Likely
Sent: Sat 11/24/2007 9:12 AM
To: David H. Lynch Jr.
Cc: linuxppc-embedded
Subject: Re: Xilinx devicetrees
 
On 11/24/07, David H. Lynch Jr. <dhlii@dlasys.net> wrote:

>>     I am having some difficulty grasping the significant advantages to
>> this.
>>     If the firmware for a given target is not fairly static - and I load
>> different firmware depending on what I am doing all the time, then I
>> know have to manage both a bit file for the firmware, and a devicetree
>> file describing it to the kernel.

The device tree file is meta-information about your bitfile.  Think of it as 'part of the firmware'.  In the (hopefully short) future, it won't even have to be managed, it will just be one of the things that is generated by the EDK flow.

> That being said, using the device tree does not preclude using
> platform code to setup devices *where it makes sense to do so*.  Many
> things are one-off board specific things that are not well described
> with the device tree.  For example, we've been debating how to handle
> on board sound which use common codec chips, but have custom wireup.
> The codec chip can use a common driver, but the wireup is handled with
> an ALSA 'fabric' driver that is pretty much a one-off for the board.
> It probably makes more sense for the fabric driver to be instantiated
> by the platform code rather than trying to do a full device tree
> representation.

Actually, even this is still driven by the device tree, because the platform code binds to the toplevel 'compatible' property...  It's just 'different' from a standard device driver.  

>>
>>     What am  missing about devicetrees that would make me more
>> interested in them ?

You won't have to write a bunch of code that deciphers which fpga firmware you are running on..  That information will be found with the firmware and can be dealt with in a generic way.  If you already HAVE that code, you can keep using it, but maintaining that kind of code is probably not where you want to spend your time.

Steve

[-- Attachment #2: Type: text/html, Size: 2897 bytes --]

Re: Xilinx devicetrees

[David H. Lynch Jr.]

    First;
          I am not deliberately trying to be obstructive. It is apparent
that the ppc kernel is moving towards devicetrees and initially I
thought that sounded like  a good idea.
          Now I am trying to reconcile the positive benefits with my
perception of the negative side effects.

> Yes, you are correct in all of the above.
>
> One more point; it is also possible to bind the device tree up with
> the kernel so you've got a single image just like old times.  :-)
>   
    But that is not actually the same is dynamically detecting
configuration.

    On an ordinary PC where the critical core configuration is somewhat
fixed and the rest can be determined by firmware (code), this makes a
great deal of sense.
    In a system where the hardware itself is actually firmware and there
is little or no startup software to query the device and build a device
tree dynamically for you, it is of more questionable value.
    Maybe if there is some mechanism existed to have the devicetree
stored into the FPGA firmware where there is a natural link between the
implimented hardware and its description. But I am not gathering things
going in that direction and storing the device tree in the FPGA consumes
valuable FPGA resources.

>
> The board description has to live *somewhere*.  
    I have done alot of code for many purposes where the code went to a
great deal of effort to figure out its own environment dynamically.
    Some (relatively minimal) assumptions have to be made. And some
balance has to be struck between code complexity and dynamic flexibility.
    though sometimes dynamic detection can be simpler.
    Part of what bothers me about devicetrees, is that the entire design
seems to presume either standard hardware with minimal permutations, or
fairly complex firmware - like boot roms to dynamically build atleast
parts of the device tree.

>
> That being said, using the device tree does not preclude using
> platform code to setup devices *where it makes sense to do so*.  Many
> things are one-off board specific things that are not well described
> with the device tree.  For example, we've been debating how to handle
> on board sound which use common codec chips, but have custom wireup.
> The codec chip can use a common driver, but the wireup is handled with
> an ALSA 'fabric' driver that is pretty much a one-off for the board.
> It probably makes more sense for the fabric driver to be instantiated
> by the platform code rather than trying to do a full device tree
> representation.
>   
       So I can hard code some relatively simple stripped devicetree
that  may do little more than specify the CPU, major elements of the
memory map (maybe), and say the PIC, and then let the BSP, detect things
like the UART(s), NIC, ...


>   In arch/powerpc
> we're *still* data driven; it's just that the data is no longer
> compiled into a static structure.  Plus, in the transition we've moved
> from needed per-device platform data structures to a more expressive
> format.  Also, per-board platform support code has become much simpler
> (compare ml300.c in arch/ppc with platforms/40x/virtex.c)
>   

    I have not pulled your tree in a while - are devicetree's in your
current git tree ?

    Thanks for the remarks.

    Again, I am just trying to figure out how to keep my Pico code in
sync and hopefully make my life better rather than worse at the same time.
    Unfortunately, I am coming to the conclusion that DeviceTrees are
probably not going to make it that much better,
    but they are probably not going to make it that much worse either.
   

-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Xilinx devicetrees

[David H. Lynch Jr.]

Stephen Neuendorffer wrote:
>
> -----Original Message-----
> From: linuxppc-embedded-bounces+stephen=neuendorffer.name@ozlabs.org on behalf of Grant Likely
> Sent: Sat 11/24/2007 9:12 AM
> To: David H. Lynch Jr.
> Cc: linuxppc-embedded
> Subject: Re: Xilinx devicetrees
>  
> On 11/24/07, David H. Lynch Jr. <dhlii@dlasys.net> wrote:
>
>   
>>>     I am having some difficulty grasping the significant advantages to
>>> this.
>>>     If the firmware for a given target is not fairly static - and I load
>>> different firmware depending on what I am doing all the time, then I
>>> know have to manage both a bit file for the firmware, and a devicetree
>>> file describing it to the kernel.
>>>       
>
> The device tree file is meta-information about your bitfile.  Think of it as 'part of the firmware'.  In the (hopefully short) future, it won't even have to be managed, it will just be one of the things that is generated by the EDK flow.
>   
    Part of the bad news is that I have been kept so busy on the
software side of things, I have had very very little time to play with
xilinx tools and firmware. But overall at Pico we have a love hate
relationship with them. Our products are primarily wrapped arround
FPGA's particularly Xilinx.
We love what we can do. But there are days that I here loud muttering
about completely rewriting all the xilinx software tools - there is a
surprisingly large amount that many of the Pico firmware people do not
use already.
    Regardless, I think I saw a post of yours on the microblaze list
about exporting a devicetree list with the firmware.
    that is probably a better solution that what exists now.

> You won't have to write a bunch of code that deciphers which fpga firmware you are running on..  That information will be found with the firmware and can be dealt with in a generic way.  If you already HAVE that code, you can keep using it, but maintaining that kind of code is probably not where you want to spend your time.
>   
    I am curious - with the firmware is not the same as in the firmware.
    But since you brought up deciphering firmware - to me and to Pico,
and I gather to alot of others, providing indentity information within
the firmware is a really really important issue - one which xilinx seems
to be unable to make up its mind about.
    There are frequent posts addressing issues like this driver only
works with this version of some IP - but there is no way to query the IP
to enough detail to determine whether the driver will work. It is one
thing to make version registers an option. It is entirely different to
just omit them entirely or change the IP without changing the version.
Our clients beat us up for things like that. DevieTrees do nto solve
that problem.

    Anyway, my .02 would be to put the device tree into the firmware. In
a perfect world - litterally in the firmware so that when the firmware
loads the device tree data is already in the FPGA somewhere that it can
be easily ready - oh and do that without consuming many FPGA resources.
    But in a more likely realworld scenario - append the information to
the .bit file in some fashion so that if can easily be found and passed on.

    Binding it to a kernel, is a non-starter for us. That means a
permuation of multiple different OS kernels for each bit file we might
have. That is huge number. I am tasked with getting one binary for each
OS to work with nearly every device(hardware) we make including
addressing options that change with firmware AND the whim of the user.
    But life might not be to unpleasant - it might even actually be
better, if our kernel loader just extracted the devicetree from the end
of the currently executing firmware.



-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

RE: Xilinx devicetrees

[Stephen Neuendorffer]

[-- Attachment #1: Type: text/plain, Size: 3931 bytes --]




-----Original Message-----
From: David H. Lynch Jr. [mailto:dhlii@dlasys.net]
Sent: Sun 11/25/2007 1:37 AM
To: Stephen Neuendorffer
Cc: Grant Likely; linuxppc-embedded
Subject: Re: Xilinx devicetrees
 
Stephen Neuendorffer wrote:
>
> -----Original Message-----
> From: linuxppc-embedded-bounces+stephen=neuendorffer.name@ozlabs.org on behalf of Grant Likely
> Sent: Sat 11/24/2007 9:12 AM
> To: David H. Lynch Jr.
> Cc: linuxppc-embedded
> Subject: Re: Xilinx devicetrees
>  
>     Regardless, I think I saw a post of yours on the microblaze list
> about exporting a devicetree list with the firmware.
>     that is probably a better solution that what exists now.

I agree.. that's why I'm working on it. :)  But just to clarify: I don't work directly on Xilinx products, but more in advanced development.

>     I am curious - with the firmware is not the same as in the firmware.
>     But since you brought up deciphering firmware - to me and to Pico,
> and I gather to alot of others, providing indentity information within
> the firmware is a really really important issue - one which xilinx seems
> to be unable to make up its mind about.
>     There are frequent posts addressing issues like this driver only
> works with this version of some IP - but there is no way to query the IP
> to enough detail to determine whether the driver will work. It is one
> thing to make version registers an option. It is entirely different to
> just omit them entirely or change the IP without changing the version.
> Our clients beat us up for things like that. DevieTrees do nto solve
> that problem.

I know these issues are high priorities within the EDK development group.  One advantage of device trees is that this information can be included without using any additional FPGA resources.

>     Anyway, my .02 would be to put the device tree into the firmware. In
> a perfect world - litterally in the firmware so that when the firmware
> loads the device tree data is already in the FPGA somewhere that it can
> be easily ready - oh and do that without consuming many FPGA resources.
>     But in a more likely realworld scenario - append the information to
> the .bit file in some fashion so that if can easily be found and passed on.

I've experimented with putting this information into BRAM.  Typically compressed device trees should easily fit within a single BRAM.  However, I think in most scenarios this is actually overkill.  Storing the device tree with the bitstream is probably good enough, and likely cheaper since the bitstream is likely in bulk storage.  This might be configuration flash (which is accessible after booting), SystemAce (in which case, the systemace image could load the device tree into a known location in memory).  In other systems the bitstream and the device tree might be combined into an executable file along with processor code for configuring the FPGA.  This might be used with an external processor or a partially reconfigured system.

>    Binding it to a kernel, is a non-starter for us.

I agree that this is not the best way of leveraging the power of device trees.  The point is that by using a device tree, you haven't lost anything you can do currently.  In the future we might have one kernel which supports all versions of all our IP, along with all flavors of microblaze and powerpc...  You would only ever need to recompile this kernel as a final optimization, if at all.

> I am tasked with getting one binary for each
> OS to work with nearly every device(hardware) we make including
> addressing options that change with firmware AND the whim of the user.
>     But life might not be to unpleasant - it might even actually be
> better, if our kernel loader just extracted the devicetree from the end
> of the currently executing firmware.

Device trees are a data driven way of doing exactly this.

Steve


[-- Attachment #2: Type: text/html, Size: 4931 bytes --]

Re: Xilinx devicetrees

[Grant Likely]

On 11/25/07, David H. Lynch Jr. <dhlii@dlasys.net> wrote:
> > One more point; it is also possible to bind the device tree up with
> > the kernel so you've got a single image just like old times.  :-)
> >
>     But that is not actually the same is dynamically detecting
> configuration.

nope, not at all the same as dynamic detection; just backwards
compatibility with the way we do it now for arch/ppc.

>     On an ordinary PC where the critical core configuration is somewhat
> fixed and the rest can be determined by firmware (code), this makes a
> great deal of sense.
>     In a system where the hardware itself is actually firmware and there
> is little or no startup software to query the device and build a device
> tree dynamically for you, it is of more questionable value.
>     Maybe if there is some mechanism existed to have the devicetree
> stored into the FPGA firmware where there is a natural link between the
> implimented hardware and its description. But I am not gathering things
> going in that direction and storing the device tree in the FPGA consumes
> valuable FPGA resources.

No, it doesn't make sense to store is in the FPGA fabric; but if the
design already contains BRAM then it could be placed there and
reclaimed for other purposes after booting.  Or anywhere in RAM for
that matter.  I don't know how feasible is to load a device tree into
SDRAM at FPGA config time.

If the FPGA bitstream is stored in processor accessable flash, then
the best place would be tacked onto the end of the bitstream file.  I
believe Xilinx is planning for EDK to be responsible to generate the
.dts file for the design.

>
> >
> > The board description has to live *somewhere*.
>     I have done alot of code for many purposes where the code went to a
> great deal of effort to figure out its own environment dynamically.
>     Some (relatively minimal) assumptions have to be made. And some
> balance has to be struck between code complexity and dynamic flexibility.
>     though sometimes dynamic detection can be simpler.
>     Part of what bothers me about devicetrees, is that the entire design
> seems to presume either standard hardware with minimal permutations, or
> fairly complex firmware - like boot roms to dynamically build atleast
> parts of the device tree.

Ah; I think I see the disconnect we're having.  Device trees are not
about, *and never have been about*, device detection.  The device tree
is simply the communication medium used to describe the hardware.  It
doesn't matter if you choose to use a 100% dynamically generated
device tree from intelligent firmware or a 100% static device tree
blob.  All that matters is that the kernel is able to trust the
information handed to it by firmware.

Device detection is not a problem that the device tree is designed to solve.

It is designed to solve the problem of telling the kernel what the
platform looks like (which occurs after the detection stage).

> > That being said, using the device tree does not preclude using
> > platform code to setup devices *where it makes sense to do so*.  Many
> > things are one-off board specific things that are not well described
> > with the device tree.  For example, we've been debating how to handle
> > on board sound which use common codec chips, but have custom wireup.
> > The codec chip can use a common driver, but the wireup is handled with
> > an ALSA 'fabric' driver that is pretty much a one-off for the board.
> > It probably makes more sense for the fabric driver to be instantiated
> > by the platform code rather than trying to do a full device tree
> > representation.
> >
>        So I can hard code some relatively simple stripped devicetree
> that  may do little more than specify the CPU, major elements of the
> memory map (maybe), and say the PIC, and then let the BSP, detect things
> like the UART(s), NIC, ...

If devices *are* detectable, then they don't need to be in the device
tree.  That's why we don't describe PCI devices in the FDT for
embedded boards.  (We *could* but we *don't*)  :-)

> >   In arch/powerpc
> > we're *still* data driven; it's just that the data is no longer
> > compiled into a static structure.  Plus, in the transition we've moved
> > from needed per-device platform data structures to a more expressive
> > format.  Also, per-board platform support code has become much simpler
> > (compare ml300.c in arch/ppc with platforms/40x/virtex.c)
> >
>
>     I have not pulled your tree in a while - are devicetree's in your
> current git tree ?

arch/powerpc support for Virtex is now in Linus' mainline tree which
will become 2.6.24

>
>     Thanks for the remarks.
>
>     Again, I am just trying to figure out how to keep my Pico code in
> sync and hopefully make my life better rather than worse at the same time.
>     Unfortunately, I am coming to the conclusion that DeviceTrees are
> probably not going to make it that much better,
>     but they are probably not going to make it that much worse either.

No, unfortunately they don't deal with the problem you're facing
(which I'm facing also).  But it will be solved if we figure out a
sane way to bind the device tree up with the FPGA bitstream without
consuming FPGA resources.

Cheers,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

Re: Xilinx devicetrees

[David H. Lynch Jr.]

Grant Likely wrote:
> nope, not at all the same as dynamic detection; just backwards
> compatibility with the way we do it now for arch/ppc.
>   
    Other things being equal a common architecture is preferable to a
collection of independent ones.


>
>
> No, it doesn't make sense to store is in the FPGA fabric; but if the
> design already contains BRAM then it could be placed there and
> reclaimed for other purposes after booting.  Or anywhere in RAM for
> that matter.  I don't know how feasible is to load a device tree into
> SDRAM at FPGA config time.
>   
    I am not expert on this, but at Pico we already store our boot
monitor in the .bit files in BRAM.
    But that is not free.  It is one of the reasons we do not use
u-boot. Our boot monitor must fit into 16K of BRAM.
    Must be able to perform selftests on critical hardware, support a
flash file system, load bit files from flash to the FGA, load and
exectute elf files, allow a small set of user commands, and handle
hosted vs. standalone operation.
    And aparently extract the devicetree from a bit file and pass it to
a linux kernel.


> Ah; I think I see the disconnect we're having.  Device trees are not
> about, *and never have been about*, device detection.  The device tree
> is simply the communication medium used to describe the hardware.  It
> doesn't matter if you choose to use a 100% dynamically generated
> device tree from intelligent firmware or a 100% static device tree
> blob.  All that matters is that the kernel is able to trust the
> information handed to it by firmware.
>
> Device detection is not a problem that the device tree is designed to solve.
>
> It is designed to solve the problem of telling the kernel what the
> platform looks like (which occurs after the detection stage).
>   
    In static or fairly static hardware, that's fine. Even in somewhat
dynamic hardware with large quantities of startup resources - like a PC.
    But in highly dynamic hardware with fairly limited resources it
starts to become an issue.

    Still if xilinx intends to generate the device tree with the bit
file - even better appended to the bit file or embedded in the FPGA if
feasible,
    this could still work.

    I do not see Detection as independent of communication.
    Aside from a very minimal core, If device drivers can do a good job
of validating their hardware (back to my version registers issue in
another post) and I just load them willy nilly and let the ones that
have no hardware fail (Gross over simplification, but still viable) then
a communication scheme is meaningless. Going the opposite direction,  if
I do not have the resources to detect the hardware and build the
devicetree dynamically, AND I have  highly dynamic hardware, AND I do
not have an easy method I can trust of aquiring another source for the
hardware description, devicetree's aren't any help. There are alot of
AND's there but most if not all appear to be present with FPGA based
systems.


> arch/powerpc support for Virtex is now in Linus' mainline tree which
> will become 2.6.24
>   

    Guess it is time to pull Linus again.
> No, unfortunately they don't deal with the problem you're facing
> (which I'm facing also).  But it will be solved if we figure out a
> sane way to bind the device tree up with the FPGA bitstream without
> consuming FPGA resources.
>   
    Note to Xilinx:

       There MUST be some way of binding a device description to a bit file.

    neither building it into the FPGA fabric nor appending it to the end
of the bit file are perfect solutions,
    The former is more powerfull and flexible but wastes precious
resources. The later is more complex and puts more burdens on
    software developers, and may be completely unfeasible in some
environments - not mine fortunately.

    Regardless, something must be done. An odd collection of devicetree
files co-mingled with a collection of bit files, is little better than
xparameter files all over the place.

    And once again a plea to ALWAYS make version/capabilities registers
atleast an optional part of every design.
    Embeddeding a device tree into a design might be fairly expensive. a
pair of read only 32 bit registers is damn near free - basically the
FPGA equivalent of atmost 64 diodes or resistors.
> Cheers,
> g.
>
>   


-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

RE: Xilinx devicetrees

[Stephen Neuendorffer]

[-- Attachment #1: Type: text/plain, Size: 4423 bytes --]


I understand that you're trying to be somewhat of a devil's advocate here, but (as I mentioned in my other email), alot of these are issues we're attempting to tackle.
More comments below.

-----Original Message-----
From: linuxppc-embedded-bounces+stephen=neuendorffer.name@ozlabs.org on behalf of David H. Lynch Jr.
Sent: Sun 11/25/2007 2:55 PM
To: Grant Likely; linuxppc-embedded
Subject: Re: Xilinx devicetrees
 
Grant Likely wrote:
>    I am not expert on this, but at Pico we already store our boot
> monitor in the .bit files in BRAM.
>     But that is not free.  It is one of the reasons we do not use
> u-boot. Our boot monitor must fit into 16K of BRAM.
>     Must be able to perform selftests on critical hardware, support a
> flash file system, load bit files from flash to the FGA, load and
> exectute elf files, allow a small set of user commands, and handle
> hosted vs. standalone operation.
>     And aparently extract the devicetree from a bit file and pass it to
> a linux kernel.

Once you can load a bitstream from flash, loading the device tree from flash
should be practically free.  In any event, why do you do this rather than just run out of the flash (or a ram copy of the flash?)

>     In static or fairly static hardware, that's fine. Even in somewhat
> dynamic hardware with large quantities of startup resources - like a PC.
>     But in highly dynamic hardware with fairly limited resources it
> starts to become an issue.

As Grant says, the dynamic detection doesn't have to be done in the boot loader, it could be done in the platform code.  You can largely ignore the device trees, or always boot with a core device tree and figure it all out later (perhaps using version registers).  I anticipate that the 'standard flow' will have standard platform code for any board that uses a complete device tree.  If you have the need to do something extraordinary, then you should feel free to hack away...  However, It doesn't seem to me to be really necessary in your case, assuming that the device tree is packaged (somehow, TBD) along with the bitstream.

>> No, unfortunately they don't deal with the problem you're facing
>> (which I'm facing also).  But it will be solved if we figure out a
>> sane way to bind the device tree up with the FPGA bitstream without
>> consuming FPGA resources.
>>   
>    Note to Xilinx:
>
>       There MUST be some way of binding a device description to a bit file.
>
>    neither building it into the FPGA fabric nor appending it to the end
> of the bit file are perfect solutions,
>    The former is more powerfull and flexible but wastes precious
> resources. The later is more complex and puts more burdens on
>    software developers, and may be completely unfeasible in some
> environments - not mine fortunately.

I don't understand the 'burden on software developers'.  The code to do this will just be standard code.  The worst that one can say is:
1) I need several KB additional non volatile storage.  Given the size of the FPGA bitstream, this can't be a huge constraint.
2) I can't use compile time optimization based on xparameters as easily.  Anyone want to implement the alternatives mechanism on ppc and microblaze?
3) Some additional boot time.  However, again, this seems marginal.

>    Regardless, something must be done. An odd collection of devicetree
> files co-mingled with a collection of bit files, is little better than
> xparameter files all over the place.

Certainly..  But in a sense, these are all intermediate files on the path to the image on the board.  That (and how it is interpreted by the platform code) should be generated in a consistent fashion by EDK.  See my other email for some of the possibilities.  Are there specific reasons why you think those proposals are inadequate?  Now is the time when we can take criticism, with the goal towards making a good end solution.

>    And once again a plea to ALWAYS make version/capabilities registers
> atleast an optional part of every design.
>    Embeddeding a device tree into a design might be fairly expensive. a
> pair of read only 32 bit registers is damn near free - basically the
> FPGA equivalent of atmost 64 diodes or resistors.

Actually, device trees actually seem to be cheaper (in the whole system sense) than such registers.  Unless there is something I don't understand?

Steve

[-- Attachment #2: Type: text/html, Size: 5534 bytes --]

Re: Xilinx devicetrees

[Grant Likely]

On 11/26/07, Koss, Mike (Mission Systems) <mike.koss@ngc.com> wrote:
> DL>    And once again a plea to ALWAYS make version/capabilities registers
> DL> atleast an optional part of every design.
> DL>    Embeddeding a device tree into a design might be fairly expensive. a
> DL> pair of read only 32 bit registers is damn near free - basically the
> DL> FPGA equivalent of atmost 64 diodes or resistors.
>
> SN> Actually, device trees actually seem to be cheaper (in the whole system
> sense) than such registers.  Unless there is something I don't understand?
>
> The issue here is that the hardware changed and the driver doesn't support
> it. I think this would be fixed by having information passed to the driver
> in the platform_device struct to specify information, since its not able to
> be discerned by the physical hardware information: version registers, etc.

This is exactly the information that should be encoded in the
'compatible' property of the device tree.  (instead of platform_data;
platform_data is no longer required with the of_platform bus binding)

*If* edk is generating our device tree(s) for us, *then* version
registers are not needed by Linux.

Cheers,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

Re: Xilinx devicetrees

[David H. Lynch Jr.]

Grant Likely wrote:
> On 11/26/07, Koss, Mike (Mission Systems) <mike.koss@ngc.com> wrote:
>   
>> DL>    And once again a plea to ALWAYS make version/capabilities registers
>> DL> atleast an optional part of every design.
>> DL>    Embeddeding a device tree into a design might be fairly expensive. a
>> DL> pair of read only 32 bit registers is damn near free - basically the
>> DL> FPGA equivalent of atmost 64 diodes or resistors.
>>
>> SN> Actually, device trees actually seem to be cheaper (in the whole system
>> sense) than such registers.  Unless there is something I don't understand?
>>     
    First the decoding for the register is almost certainly already
present for the other registers for the device.
    After that - assuming that an FPGA can impliment a read only
register as easily as discrete logic -  it should be damn near the
    most trivial peice of hardware imaginable. It is simpler than 64
bits of RAM. It can be simpler than 64bits of ROM.
    I do not think there is a way in the world that devicetrees can more
cheaply provide the same information.
    They might me more flexible, or powerful, but 2 64bit read only
registers.

    Anyway I am not arguing that you should not do devicetrees. Just
that you should do version/capabilities registers - atleast as a IP
option always.
    Every OS does nto support devicetree's. Every application of an FPGA
not going to have that as an option. But every peice of software that
can access and I/O device can access its version and capabilities registers.


> *If* edk is generating our device tree(s) for us, *then* version
> registers are not needed by Linux.
>   
    I want both. I want version registers - because they are ALWAYS
available.
    They are available to software running on the FPGA that has no clue
what bit file it is running on,how it got to be running.
    Devicetrees may provide a great deal more information bit it is not
hard to come up with scenarios where that information might not
    be present. It is like the security arguments about biometric
identification. I can forget my key card, my password, ... But I am
unlikely to forget my thumbprint or retina.
   

-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Xilinx devicetrees

[David H. Lynch Jr.]

Koss, Mike (Mission Systems) wrote:
> Time for my $.02, since I am heavily weighting my future designs on the
> use of the device trees. :) (and b/c I don't check my work e-mail from
> home ;P)
>
> ________________________________
>
> From: Stephen Neuendorffer [mailto:stephen.neuendorffer@xilinx.com] 
> Sent: Sunday, November 25, 2007 6:59 PM
> To: David H. Lynch Jr.; Grant Likely; linuxppc-embedded
> Subject: RE: Xilinx devicetrees
>
>
> SN> the device tree is packaged (somehow, TBD) along with the bitstream.
>   
    If xilinx wants to optionally incorporate the device tree into the
bitstream in a fashion that can easily be worked out by software - that
would be excellent.


>  
> I don't know if packaging the device tree with the bitstream is the best
> way to go. It's possible that it could lead to headaches for certain
> systems that have security restrictions. 
    Make it an option. Where are the systems with security restrictions
going to get their hardware information ?
    Besides - though I am not aware off the top of my head of a
bitstream decompliler, still if you have access to the bitstream you
have access to the information about the device. We deal with alot of
high security applications. Security restrictions have to make sense.
    blocking devicetrees for security reasons is like telling somebody
they can have the manual in computer readable form, but not on paper.
    Further if you are going to boot an OS that requires devicetrees
then the devicetree must be somewhere - whether it is appended to the
bitstream, appended to the kernel, compiled into the kernel, in a
separate file, it still has to be present.



> The same could be said for
> using it w/ the SystemACE to load it into RAM after the image. (which is
> what I'm currently doing for my 2 linux images in lieu of a true on-chip
> bootloader). I am already taking the security concerns into account for
> future revisions of the hardware wrt to using a SystemACE, and am
> planning on moving the device trees into NV storage like FLASH.
>   
    I am not working with MLXXX boards. Pico has no System ACE. The card
has an FPGA, flash, ram and a few other things all in a
CF/Cardbus/expressbus formfactor. You program it in a host with  our
tools. You run it either in the host or standalone.

>
> SN> I don't understand the 'burden on software developers'.  The code to
> do this will just be standard code.  
    I got 16K of RAM for my boot loader and that 16K has to do alot more
than just manage device trees.
    I think we have 2K left. In that I have to fit scripting, and
ethernet, so where am I supposed to fit the standard code ?
    If the devicetree is say at the end of the bit file I can probably
copy it to ram and pass a pointer to linux in maybe a couple of hundred
bytes.
    If I have to do much more it ain't happening.


> The worst that one can say is:
> SN> 1) I need several KB additional non volatile storage.  Given the
> size of the FPGA bitstream, this can't be a huge constraint.
>   
       Several KB is NOT happening. The bitstream is in flash. Flash is
not a limited reasorce for  us.
       FPGA cells, and BRAM are precious as gold. The more we use the
less our clients have.
     
       Different systems are going to have different resource constraints.
       What is unlimited for me may be severely limited for someone
else. What is unlimted for you may be seriously limited for me.
      

> I do agree that using more FPGA resources is not a solution to the
> problem. I'm already hitting 80% usage on a FX60 and trying to squeeze
> more real estate for storage of the device tree seems silly. Especially
> since that would require that every image have this extra hardware built
> into it just to support booting a Linux kernel. Why should I have to
> have different hardware to boot linux, versus non-kernel, xilkernel, or
> other (GHS, LynxOS, etc..)?
>   
    One of the problems is that neither devicetrees, nor any of the ways
of tracking them are one size fits all solutions.
    I do GHS work, it would be nice if GHS supported devicetrees and
maybe it will.
    bound to the kernel, in a separate file, appended to the bitstream
and in the FPGA fabric all have pluses and minuses.
    One reason I am harping on version registers is they are extremely
cost cheap, can easily be made optional, and can be fairly easily
incorporated into any OS (or no OS - we do that too). They are not a
replacement for devicetrees. But they still have very important uses in
many environments.

>
> SN> Certainly..  But in a sense, these are all intermediate files on the
> path to the image on the board.
    Unless we are going to teach linux to read and interpret .bit files
while loading, then devicetrees are intermediate in an entirely
different sense.
    The hardware works fine regardless of whether their is a devicetree.
But Linux may not boot.

     My objective is to get alot of software - linux, GHS, and
standalone apps, to all load - from a single executables, accross
multiple different bit images on several different (though deliberately
similar) products. My Linux kernel needs to work regardless of the Pico
card and regardless of the bit image, as done my GHS kernel, and ....
And I need to do that in a severly resource constrained environment.
    I would hope that should make sense of my harping about
version/capabilities registers. If I have maybe 10 possible peices of IP
that may/maynot be present in a given FPGA and I am trying to
dynamically configure - Linux/GHS/... to adapt to whatever it encounters
and work as long as it is a viable combination. At best devicetrees are
an extremly complex way of solving that - while version registers are a
trivial way.
    As an example I know that if I have a UartLite, Keyhole, TEMAC, PIC,
... they will always be at specific addresses. We never move them.
    But I do not know for sure it they are present. A version register
provides a fairly safe very efficient means of checking that a device is
present (and establishing its version and maybe capabilites) - device
trees might do the same thing but have alot higher overhead to do so.
   
    I deal more with presence/absence issues and maybe what version is
the IP rather than hardware is the IP and what IRQ is it using.




-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

Re: Xilinx devicetrees

[David H. Lynch Jr.]

Stephen Neuendorffer wrote:
> In any event, why do you do this rather than just run out of the flash (or a ram copy of the flash?)
>   
    This is not literally true, but  the parameters are nearly the same
- pretend we are using NAND flash  and storing executables as elf files.

    We can not run from flash because:
       The code has not been relocated
       the flash is not persistent
    We do load the flash/elf file into DRAM  dealing with elf issues
along the way and then execute it.   
    We can do something similar for devicetrees, actually just passing
Linux the offset from the begining of flash to the devicetree would be
sufficient. Or if finding the devicetree inside the bitsstream can be
accomplished fairly simply, just passing the offset of the bit file.


-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

RE: Xilinx devicetrees

[Stephen Neuendorffer]

=20

> -----Original Message-----
> From: David H. Lynch Jr. [mailto:dhlii@dlasystems.com]=20
> Sent: Monday, November 26, 2007 1:17 PM
> To: Koss, Mike (Mission Systems)
> Cc: Stephen Neuendorffer; Grant Likely; linuxppc-embedded
> Subject: Re: Xilinx devicetrees
>=20
> Koss, Mike (Mission Systems) wrote:
> > Time for my $.02, since I am heavily weighting my future=20
> designs on the
> > use of the device trees. :) (and b/c I don't check my work=20
> e-mail from
> > home ;P)
> >
> > ________________________________
> >
> > From: Stephen Neuendorffer [mailto:stephen.neuendorffer@xilinx.com]=20
> > Sent: Sunday, November 25, 2007 6:59 PM
> > To: David H. Lynch Jr.; Grant Likely; linuxppc-embedded
> > Subject: RE: Xilinx devicetrees
> >
> >
>=20
> >
> > SN> I don't understand the 'burden on software developers'.=20
>  The code to
> > do this will just be standard code. =20
>     I got 16K of RAM for my boot loader and that 16K has to=20
> do alot more
> than just manage device trees.
>     I think we have 2K left. In that I have to fit scripting, and
> ethernet, so where am I supposed to fit the standard code ?
>     If the devicetree is say at the end of the bit file I can probably
> copy it to ram and pass a pointer to linux in maybe a couple=20
> of hundred
> bytes.
>     If I have to do much more it ain't happening.

Personally, It sounds like you're trying to do to much...  The only
thing that that code really needs to do is to find the 'real' boot code:
Maybe Uboot stored in flash? :)  I'm not naive enough to suggest that I
understand all the constraints of your system, but it is another
solution.

> > The worst that one can say is:
> > SN> 1) I need several KB additional non volatile storage.  Given the
> > size of the FPGA bitstream, this can't be a huge constraint.
> >  =20
>        Several KB is NOT happening. The bitstream is in=20
> flash. Flash is
> not a limited reasorce for  us.
>        FPGA cells, and BRAM are precious as gold. The more we use the
> less our clients have.

Device tree should probably be stored in flash, in your case....

> >
> > SN> Certainly..  But in a sense, these are all intermediate=20
> files on the
> > path to the image on the board.
>     Unless we are going to teach linux to read and interpret=20
> .bit files
> while loading, then devicetrees are intermediate in an entirely
> different sense.
>     The hardware works fine regardless of whether their is a=20
> devicetree.
> But Linux may not boot.

I think this philosphy really minimized the value of software...  The
hardware doesn't work unless the software that comes with it *also*
works. :)

>      My objective is to get alot of software - linux, GHS, and
> standalone apps, to all load - from a single executables, accross
> multiple different bit images on several different (though=20
> deliberately
> similar) products. My Linux kernel needs to work regardless=20
> of the Pico
> card and regardless of the bit image, as done my GHS kernel, and ....
> And I need to do that in a severly resource constrained environment.
>     I would hope that should make sense of my harping about
> version/capabilities registers. If I have maybe 10 possible=20
> peices of IP
> that may/maynot be present in a given FPGA and I am trying to
> dynamically configure - Linux/GHS/... to adapt to whatever it=20
> encounters
> and work as long as it is a viable combination. At best=20
> devicetrees are
> an extremly complex way of solving that - while version=20
> registers are a
> trivial way.

It sounds like the real problem that you have is that even if you get
device trees working for Linux, you still have the same problem for GHS,
so from your perspective "device trees don't help"

Steve

Re: Xilinx devicetrees

[Grant Likely]

On 11/26/07, Stephen Neuendorffer <stephen.neuendorffer@xilinx.com> wrote:
>
>
> > -----Original Message-----
> > From: David H. Lynch Jr. [mailto:dhlii@dlasystems.com]
> > Sent: Monday, November 26, 2007 1:17 PM
> > To: Koss, Mike (Mission Systems)
> > Cc: Stephen Neuendorffer; Grant Likely; linuxppc-embedded
> > Subject: Re: Xilinx devicetrees
> >
> >      My objective is to get alot of software - linux, GHS, and
> > standalone apps, to all load - from a single executables, accross
> > multiple different bit images on several different (though
> > deliberately
> > similar) products. My Linux kernel needs to work regardless
> > of the Pico
> > card and regardless of the bit image, as done my GHS kernel, and ....
> > And I need to do that in a severly resource constrained environment.
> >     I would hope that should make sense of my harping about
> > version/capabilities registers. If I have maybe 10 possible
> > peices of IP
> > that may/maynot be present in a given FPGA and I am trying to
> > dynamically configure - Linux/GHS/... to adapt to whatever it
> > encounters
> > and work as long as it is a viable combination. At best
> > devicetrees are
> > an extremly complex way of solving that - while version
> > registers are a
> > trivial way.

I disagree.  I'm not disputing that version registers are valuable.
But I *am* disputing their value for device detection.  It may work in
your specific case of devices always in certain locations, but it does
not help the general case where devices can be instantiated anywhere
in the 4GB address space.

>
> It sounds like the real problem that you have is that even if you get
> device trees working for Linux, you still have the same problem for GHS,
> so from your perspective "device trees don't help"

In embedded power.org land, device trees are becoming the recommended
method for describing platform configuration for all embedded powerpc
software.  Not just Linux.  So from that perspective device trees
might help.

Cheers,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

RE: Xilinx devicetrees

[Koss, Mike (Mission Systems)]

=20
>> I don't know if packaging the device tree with the bitstream is the=20
>> best way to go. It's possible that it could lead to headaches for=20
>> certain systems that have security restrictions.
>    Make it an option. Where are the systems with security restrictions
going to get their hardware information ?
>    Besides - though I am not aware off the top of my head of a
bitstream decompliler, still if you have access to the bitstream you
have access to
> the information about the device. We deal with alot of high security
applications. Security restrictions have to make sense.
>    blocking devicetrees for security reasons is like telling somebody
they can have the manual in computer readable form, but not on paper.
>    Further if you are going to boot an OS that requires devicetrees
then the devicetree must be somewhere - whether it is appended to the
bitstream,
> appended to the kernel, compiled into the kernel, in a separate file,
it still has to be present.

The devicetree itself is not the security concern, but the retrieval of
it from the bitstream. If it is stored in the FPGA, it may be that
accessing that stored information may not be probable at run-time.

>> The worst that one can say is:
>> SN> 1) I need several KB additional non volatile storage.  Given the
>> size of the FPGA bitstream, this can't be a huge constraint.
>>  =20
>       Several KB is NOT happening. The bitstream is in flash. Flash is
not a limited reasorce for  us.
>       FPGA cells, and BRAM are precious as gold. The more we use the
less our clients have.

Which is why I'm proposing that it be an "extra" file that can be stored
however. For one system, it could be stored in the FPGA using BRAM and
it could be automatically available to kernel. For another system, it
could be stored in FLASH and the bootloader may have to copy the data
from FLASH to RAM.

>> I do agree that using more FPGA resources is not a solution to the=20
>> problem. I'm already hitting 80% usage on a FX60 and trying to
squeeze=20
>> more real estate for storage of the device tree seems silly.=20
>> Especially since that would require that every image have this extra=20
>> hardware built into it just to support booting a Linux kernel. Why=20
>> should I have to have different hardware to boot linux, versus=20
>> non-kernel, xilkernel, or other (GHS, LynxOS, etc..)?
>>  =20
>    One of the problems is that neither devicetrees, nor any of the
ways of tracking them are one size fits all solutions.
>    I do GHS work, it would be nice if GHS supported devicetrees and
maybe it will.
>    bound to the kernel, in a separate file, appended to the bitstream
and in the FPGA fabric all have pluses and minuses.
>    One reason I am harping on version registers is they are extremely
cost cheap, can easily be made optional, and can be fairly easily
incorporated > into any OS (or no OS - we do that too). They are not a
replacement for devicetrees. But they still have very important uses in
many environments.

>
>> SN> Certainly..  But in a sense, these are all intermediate files on=20
>> SN> the
>> path to the image on the board.
>    Unless we are going to teach linux to read and interpret .bit files
while loading, then devicetrees are intermediate in an entirely
different
> sense.
>    The hardware works fine regardless of whether their is a
devicetree.
> But Linux may not boot.

You are correct, but I think a single self-contained file that has all
of the hardware descriptions is much preferred to many header files, and
#ifdef macro's to make sense of what is actually available at run-time.

>     My objective is to get alot of software - linux, GHS, and
standalone apps, to all load - from a single executables, accross
multiple different
> bit images on several different (though deliberately
> similar) products. My Linux kernel needs to work regardless of the
Pico card and regardless of the bit image, as done my GHS kernel, and
....
> And I need to do that in a severly resource constrained environment.
>    I would hope that should make sense of my harping about
version/capabilities registers. If I have maybe 10 possible peices of IP
that may/maynot
> be present in a given FPGA and I am trying to dynamically configure -
Linux/GHS/... to adapt to whatever it encounters and work as long as it
is a
> viable combination. At best devicetrees are an extremly complex way of
solving that - while version registers are a trivial way.
>    As an example I know that if I have a UartLite, Keyhole, TEMAC,
PIC, ... they will always be at specific addresses. We never move them.
>    But I do not know for sure it they are present. A version register
provides a fairly safe very efficient means of checking that a device is
> present (and establishing its version and maybe capabilites) - device
trees might do the same thing but have alot higher overhead to do so.

For your system, yes, you never move the location of the devices. But
for others, we may add other pieces or multiple pieces and require them
in multiple/different places. That's the beauty of the device tree. You
*can* create that one monolithic kernel that has support for ANY type of
hardware, and at boot-time a device tree is provided to the kernel that
defines all of the devices that are physically available in your FPGA.

I'm not trying to argue for/against the version registers. Just that a
version register is short-sighted with regards to the larger FPGA-based
view. Version registers work wonderfully when everything is staticly
located, like in your image builds or a SoC like 440GX. For general
usage for others creating their own FPGA images with varying hardware
setups, statically defined location version register checking is not
feasible.

The device tree approach can "easily" be adapted to other systems, such
as GHS, through adaptors in your corresponding BSP.

-- Mike

Re: Xilinx devicetrees

[John Williams]

Hi folks,

Stephen Neuendorffer wrote:

>  >    Binding it to a kernel, is a non-starter for us.
> 
> I agree that this is not the best way of leveraging the power of device 
> trees.  The point is that by using a device tree, you haven't lost 
> anything you can do currently.  In the future we might have one kernel 
> which supports all versions of all our IP, along with all flavors of 
> microblaze and powerpc...  You would only ever need to recompile this 
> kernel as a final optimization, if at all.

I strongly support the OF / device tree work being done, from its own 
perspective and also as a path to MicroBlaze/PPC unification, however 
there is one critical difference that I have not seen adequately 
addressed yet.

MicroBlaze is a highly configurable CPU in terms of its instruction set, 
features and so on.  To make use of this, it is essential that each 
kernel image be compiled to match the CPU configuration.  While a 
generic kernel, making use of no features (MUL, DIV, Shift, MSR ops etc) 
would run on any CPU, performance would be abysmal.

In my view it's not acceptable to present these as options for the user 
to select at kernel config time. With N yes/no parameters, there is 1 
correct configuration, and 2^N-1 incorrect ones.  The odds of the user 
falling upon a configuration that at worst fails to boot, or at best is 
not optimally matched to the hardware, are high.

This same issue also applies to C libraries and apps - they must be 
compiled with prior knowledge of the CPU.  This is why our 
microblaze-uclinux-gcc toolchain, with multilib'd uClibc, is almost 400meg!

Wrapping every mul, div, shift etc in a function call is clearly not 
feasible.  Things like the msrset/msrclr ops have a modest but 
measurable impact on kernel code size and performance - it's just not 
reasonable to add any level of indirection in there.

I have thought about dynamic (boot-time) code re-writing as one 
possibility here, but it very quickly gets nasty.  All of the 
"optimised" opcodes (MUL/DIV/Shift etc) are smaller than their emulated 
counterparts, so in principle we could re-write the text segment with 
the optimised opcode, then NOP pad, but that's still inefficient.  As 
soon as we start talking about dynamic code relocation, re-writing 
relative offsets in loops, ... yuck..  We'd be putting half of mb-ld 
into the arch early boot code (or bootloader...)

The opposite approach, to build with all instructions enabled and 
install exception handlers to deal with the fixups, is also pretty awful.

I find myself asking the question - for what use cases does the current 
static approach used in MicroBlaze (with the PetaLinux BSP / 
Kconfig.auto) *not* work?

One compromise approach might be to have a script in 
arch/microblaze/scripts, called by the arch Makefile, that cracks open 
the DT at build time and extracts appropriate cpu flags.

Finally, what is the LKML position on DT files going into the kernel 
source tree?

Regards,

John

Re: Xilinx devicetrees

[Grant Likely]

On 11/27/07, John Williams <jwilliams@itee.uq.edu.au> wrote:
> Hi folks,
>
> Stephen Neuendorffer wrote:
>
> >  >    Binding it to a kernel, is a non-starter for us.
> >
> > I agree that this is not the best way of leveraging the power of device
> > trees.  The point is that by using a device tree, you haven't lost
> > anything you can do currently.  In the future we might have one kernel
> > which supports all versions of all our IP, along with all flavors of
> > microblaze and powerpc...  You would only ever need to recompile this
> > kernel as a final optimization, if at all.
>
> I strongly support the OF / device tree work being done, from its own
> perspective and also as a path to MicroBlaze/PPC unification, however
> there is one critical difference that I have not seen adequately
> addressed yet.
>
> MicroBlaze is a highly configurable CPU in terms of its instruction set,
> features and so on.  To make use of this, it is essential that each
> kernel image be compiled to match the CPU configuration.  While a
> generic kernel, making use of no features (MUL, DIV, Shift, MSR ops etc)
> would run on any CPU, performance would be abysmal.

Looks like you've found the point of diminishing returns.  :-)  It is
probably not appropriate to try and encode CPU configuration into the
device tree *unless* it is only used to determine whether or not the
kernel as compiled will run on that CPU; but even by then you're
already running code on the platform.  :-)

>
> In my view it's not acceptable to present these as options for the user
> to select at kernel config time. With N yes/no parameters, there is 1
> correct configuration, and 2^N-1 incorrect ones.  The odds of the user
> falling upon a configuration that at worst fails to boot, or at best is
> not optimally matched to the hardware, are high.
>
> This same issue also applies to C libraries and apps - they must be
> compiled with prior knowledge of the CPU.  This is why our
> microblaze-uclinux-gcc toolchain, with multilib'd uClibc, is almost 400meg!
>
> Wrapping every mul, div, shift etc in a function call is clearly not
> feasible.  Things like the msrset/msrclr ops have a modest but
> measurable impact on kernel code size and performance - it's just not
> reasonable to add any level of indirection in there.
>
> I have thought about dynamic (boot-time) code re-writing as one
> possibility here, but it very quickly gets nasty.  All of the
> "optimised" opcodes (MUL/DIV/Shift etc) are smaller than their emulated
> counterparts, so in principle we could re-write the text segment with
> the optimised opcode, then NOP pad, but that's still inefficient.  As
> soon as we start talking about dynamic code relocation, re-writing
> relative offsets in loops, ... yuck..  We'd be putting half of mb-ld
> into the arch early boot code (or bootloader...)
>
> The opposite approach, to build with all instructions enabled and
> install exception handlers to deal with the fixups, is also pretty awful.
>
> I find myself asking the question - for what use cases does the current
> static approach used in MicroBlaze (with the PetaLinux BSP /
> Kconfig.auto) *not* work?
>
> One compromise approach might be to have a script in
> arch/microblaze/scripts, called by the arch Makefile, that cracks open
> the DT at build time and extracts appropriate cpu flags.
>
> Finally, what is the LKML position on DT files going into the kernel
> source tree?

arch/powerpc is awash with .dts files in the kernel tree.  It's a
practice that is encouraged  for most of the embedded boards.  :-)

However, for something like xilinx virtex ppc and microblaze it
probably doesn't make much sense for anything other than the Xilinx
reference design bitstreams.  (at least in mainline)

Cheers,
g.

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

RE: Xilinx devicetrees

[Stephen Neuendorffer]

=20
> -----Original Message-----
> From: John Williams [mailto:jwilliams@itee.uq.edu.au]=20
> Sent: Tuesday, November 27, 2007 3:55 PM
> To: Stephen Neuendorffer
> Cc: David H. Lynch Jr.; linuxppc-embedded; Michal Simek
> Subject: Re: Xilinx devicetrees
>=20
>=20
> MicroBlaze is a highly configurable CPU in terms of its=20
> instruction set,=20
> features and so on.  To make use of this, it is essential that each=20
> kernel image be compiled to match the CPU configuration.  While a=20
> generic kernel, making use of no features (MUL, DIV, Shift,=20
> MSR ops etc)=20
> would run on any CPU, performance would be abysmal.

I think the userspace is actually much more critical than the kernel for
most of these things (with the exception of msrset/msrclr, and the
barrel shifter perhaps).  Unfortunately, even if you implement an
alternatives-style mechanism for the kernel, you're still hosed for
userspace.  Once I a big enough system, it's just unfeasible to
recompile everything anyway.  I think this is where autoconfig starts to
break down.

> In my view it's not acceptable to present these as options=20
> for the user=20
> to select at kernel config time. With N yes/no parameters, there is 1=20
> correct configuration, and 2^N-1 incorrect ones.  The odds of=20
> the user=20
> falling upon a configuration that at worst fails to boot, or=20
> at best is=20
> not optimally matched to the hardware, are high.

Yes.  Autoconfig does handle this in a fairly nice way.

> This same issue also applies to C libraries and apps - they must be=20
> compiled with prior knowledge of the CPU.  This is why our=20
> microblaze-uclinux-gcc toolchain, with multilib'd uClibc, is=20
> almost 400meg!
>=20
> Wrapping every mul, div, shift etc in a function call is clearly not=20
> feasible.  Things like the msrset/msrclr ops have a modest but=20
> measurable impact on kernel code size and performance - it's just not=20
> reasonable to add any level of indirection in there.
>=20
> I have thought about dynamic (boot-time) code re-writing as one=20
> possibility here, but it very quickly gets nasty.  All of the=20
> "optimised" opcodes (MUL/DIV/Shift etc) are smaller than=20
> their emulated=20
> counterparts, so in principle we could re-write the text segment with=20
> the optimised opcode, then NOP pad, but that's still inefficient.  As=20
> soon as we start talking about dynamic code relocation, re-writing=20
> relative offsets in loops, ... yuck..  We'd be putting half of mb-ld=20
> into the arch early boot code (or bootloader...)
>=20
> The opposite approach, to build with all instructions enabled and=20
> install exception handlers to deal with the fixups, is also=20
> pretty awful.

It's not nice, I agree.  I think the key principle should be that I
should be able to get a system working as quickly as possible, and I
might optimize things later.  One thing that device trees will allow is
for *all* the drivers to get compiled in to the kernel, and only as a
late stage operation does a designer need to start throwing things away.
Using traps I can easily start with a 'kitchen sink' design, and start
discarding processor features, relying on the traps.  When I get low
enough down on the performance curve, I can uas an autoconfig-type
mechanism to regain a little of what I lost by optimizing away the trap
overhead.=20

Personally, I think the easiest way out of all this is to just have less
configurability.  For microblaze in general, this is too much of a
restriction, but microblaze used as a control processor running linux,
there are probably just a few design points that really make sense
(probably size optimized: no options except maybe msrset/msrclr, and the
kitchen sink).  If we go that far, we don't really need people to ever
run autoconfig, or kernels, or anything.  Especially considering there
is no easy way of selecting which of the 2^N design points I want
*anyway*. :)

> I find myself asking the question - for what use cases does=20
> the current=20
> static approach used in MicroBlaze (with the PetaLinux BSP /=20
> Kconfig.auto) *not* work?
>=20
> One compromise approach might be to have a script in=20
> arch/microblaze/scripts, called by the arch Makefile, that=20
> cracks open=20
> the DT at build time and extracts appropriate cpu flags.

Hmm... interesting idea, although parsing the source is likely
difficult...  It's probably not worth it to go this far, I think.   As
you say, why doesn't autoconfig of today work fine for this.

> Finally, what is the LKML position on DT files going into the kernel=20
> source tree?

Source .dts go in and get compiled to binary blobs at compile time.  The
'big' recent controversy is whether the source->binary compiler dtc
should be mirrored in the Linux tree or not.

Steve

Re: Xilinx devicetrees

[John Williams]

Stephen Neuendorffer wrote:

>>From: John Williams [mailto:jwilliams@itee.uq.edu.au] 

>>MicroBlaze is a highly configurable CPU in terms of its 
>>instruction set, 
>>features and so on.  To make use of this, it is essential that each 
>>kernel image be compiled to match the CPU configuration.  While a 
>>generic kernel, making use of no features (MUL, DIV, Shift, 
>>MSR ops etc) 
>>would run on any CPU, performance would be abysmal.
> 
> I think the userspace is actually much more critical than the kernel for
> most of these things (with the exception of msrset/msrclr, and the
> barrel shifter perhaps).  Unfortunately, even if you implement an
> alternatives-style mechanism for the kernel, you're still hosed for
> userspace.  

I haven't benchmarks each option on the kernel - you are right that 
shift is a big one, but mul I think is also important, given that every 
array access generates an integer multiply,

Once I a big enough system, it's just unfeasible to
> recompile everything anyway.  I think this is where autoconfig starts to
> break down.

I'm not sure I agree, here, given that most people building MicroBlaze 
systems are doing so with uClinux-dist (or PetaLinux), you can do a full 
rebuilt, kernel libs apps, in a couple of minutes.  Much shorter than 
sythnesis and P&R, that's for sure (and runtime is linear in size, 
unlike P&R :)

> It's not nice, I agree.  I think the key principle should be that I
> should be able to get a system working as quickly as possible, and I
> might optimize things later.  One thing that device trees will allow is
> for *all* the drivers to get compiled in to the kernel, and only as a
> late stage operation does a designer need to start throwing things away.
> Using traps I can easily start with a 'kitchen sink' design, and start
> discarding processor features, relying on the traps.  When I get low
> enough down on the performance curve, I can uas an autoconfig-type
> mechanism to regain a little of what I lost by optimizing away the trap
> overhead. 

OK, but now we have a kernel dependent on *3* files - a DTS, a 
Kconfig.auto, and (indirectly) the bitstream itself.

> Personally, I think the easiest way out of all this is to just have less
> configurability.  For microblaze in general, this is too much of a
> restriction, but microblaze used as a control processor running linux,
> there are probably just a few design points that really make sense
> (probably size optimized: no options except maybe msrset/msrclr, and the
> kitchen sink).  If we go that far, we don't really need people to ever
> run autoconfig, or kernels, or anything.  Especially considering there
> is no easy way of selecting which of the 2^N design points I want
> *anyway*. :)

My experience tells me that if the microblaze can be configured in a 
particular way, *someone* will want to do it (and still boot linux on 
it!)  We still have people building MicroBlaze 3.00 in Spartan2E, with 
EDK 6.3.  And autoconfig works!  Exceptions on/off, MMU on/off (runtime 
configurable on that?).

Our ability to plug into the backend design database of EDK presents a 
great opportunity - truly automatically configured kernels.  I think we 
have a responsibility to leverage that power.    We are already there 
with the static approach, I think we just need to make sure that 
persists into the dynamic approach, and that we find a good mix of the two.

There are of course some semantic issues that the EDK cannot 
automatically resolve - relative ordering and priority of multiple 
peripheral instances for example.

>>One compromise approach might be to have a script in 
>>arch/microblaze/scripts, called by the arch Makefile, that 
>>cracks open 
>>the DT at build time and extracts appropriate cpu flags.
> 
> Hmm... interesting idea, although parsing the source is likely
> difficult...  It's probably not worth it to go this far, I think.   As
> you say, why doesn't autoconfig of today work fine for this.

Well, copying multiple configuration files into the kernel is not ideal. 
  Surely a little perl or python script would do the trick?  DTS syntax 
is pretty clean, just find the CPU node and off we go.  Multiple CPU, 
well... :)

>>Finally, what is the LKML position on DT files going into the kernel 
>>source tree?
> 
> 
> Source .dts go in and get compiled to binary blobs at compile time.  The
> 'big' recent controversy is whether the source->binary compiler dtc
> should be mirrored in the Linux tree or not.

OK.

Another thing I suggested to Michal recently, perhaps we need 
kernel/lib/libof to store common OF / DT handling code.  Much better 
than duplicating it accross microblaze and PPC, and maybe other arch's 
would also see the light..  That would also add a claim for the DTC to 
go in scripts/

John

Re: Xilinx devicetrees

[Jon Loeliger]

John Williams wrote:

> Well, copying multiple configuration files into the kernel is not ideal. 

Well, some advances in the DTC are being developed to help
mitigate the proliferation of DTS files there.  It's still
slow going, but we're working on it...

>   Surely a little perl or python script would do the trick?  DTS syntax 
> is pretty clean, 

Are you looking at the same DTS I am? :-)


> Another thing I suggested to Michal recently, perhaps we need 
> kernel/lib/libof to store common OF / DT handling code.  Much better 
> than duplicating it accross microblaze and PPC, and maybe other arch's 
> would also see the light..

Not sure exactly what you mean here, but there are libfdt
efforts in the DTC source repo already.  That code is being
used in U-Boot.  There is also already a shared library  of
OF/DT handling code in the kernel that is being used by
both PowerPC and SPARC today.  I believe there are plans to
head it towards the same libfdt as well.

>  That would also add a claim for the DTC to 
> go in scripts/

Which, for the record, I'm not really convinced is the
right thing to do yet.

> John

Thanks,
jdl

RE: Xilinx devicetrees

[Stephen Neuendorffer]

=20

> -----Original Message-----
> From:=20
> linuxppc-embedded-bounces+stephen=3Dneuendorffer.name@ozlabs.org
> =20
> [mailto:linuxppc-embedded-bounces+stephen=3Dneuendorffer.name@oz
labs.org] On Behalf Of John Williams
> Sent: Tuesday, November 27, 2007 4:53 PM
> To: Stephen Neuendorffer
> Cc: Michal Simek; linuxppc-embedded
> Subject: Re: Xilinx devicetrees
>=20
> Stephen Neuendorffer wrote:
>=20
> >>From: John Williams [mailto:jwilliams@itee.uq.edu.au]=20
>=20
> >>MicroBlaze is a highly configurable CPU in terms of its=20
> >>instruction set,=20
> >>features and so on.  To make use of this, it is essential that each=20
> >>kernel image be compiled to match the CPU configuration.  While a=20
> >>generic kernel, making use of no features (MUL, DIV, Shift,=20
> >>MSR ops etc)=20
> >>would run on any CPU, performance would be abysmal.
> >=20
> > I think the userspace is actually much more critical than=20
> the kernel for
> > most of these things (with the exception of msrset/msrclr, and the
> > barrel shifter perhaps).  Unfortunately, even if you implement an
> > alternatives-style mechanism for the kernel, you're still hosed for
> > userspace. =20
>=20
> I haven't benchmarks each option on the kernel - you are right that=20
> shift is a big one, but mul I think is also important, given=20
> that every=20
> array access generates an integer multiply,

Good point.

>=20
> Once I a big enough system, it's just unfeasible to
> > recompile everything anyway.  I think this is where=20
> autoconfig starts to
> > break down.
>=20
> I'm not sure I agree, here, given that most people building=20
> MicroBlaze=20
> systems are doing so with uClinux-dist (or PetaLinux), you=20
> can do a full=20
> rebuilt, kernel libs apps, in a couple of minutes.  Much shorter than=20
> sythnesis and P&R, that's for sure (and runtime is linear in size,=20
> unlike P&R :)

Let's not bash the P&R guys too much...  You try working on a problem
that is known to be insolvable, and where no matter how many people you
make happier, you'll always get bashed by the guy whose design no longer
meets timing. :)

> > It's not nice, I agree.  I think the key principle should be that I
> > should be able to get a system working as quickly as possible, and I
> > might optimize things later.  One thing that device trees=20
> will allow is
> > for *all* the drivers to get compiled in to the kernel, and=20
> only as a
> > late stage operation does a designer need to start throwing=20
> things away.
> > Using traps I can easily start with a 'kitchen sink'=20
> design, and start
> > discarding processor features, relying on the traps.  When I get low
> > enough down on the performance curve, I can uas an autoconfig-type
> > mechanism to regain a little of what I lost by optimizing=20
> away the trap
> > overhead.=20
>=20
> OK, but now we have a kernel dependent on *3* files - a DTS, a=20
> Kconfig.auto, and (indirectly) the bitstream itself.

The kernel might be generated from them, but it is not *dependent* on
them.  The same kernel can run on other hardware, with other dts's.  If
there was a traps mechanism (or alternatives), then it could also run on
other designs with different processor features.

> Another thing I suggested to Michal recently, perhaps we need=20
> kernel/lib/libof to store common OF / DT handling code.  Much better=20
> than duplicating it accross microblaze and PPC, and maybe=20
> other arch's=20
> would also see the light..  That would also add a claim for=20
> the DTC to=20
> go in scripts/

There's some shared code in 2.6.24-rc in drivers/of.  Now that things
are settling down in terms of bugs, I'll probably transition to pushing
a 2.6.24-rc branch soon.  However, the few brief conversations I've had
suggest that what microblaze might need or want has very little
influence until it is visible in mainline.  Once that happens, I think
it will be easy to justify putting code like libfdt and some of the
kernel of/dt code in a common place. =20

So, John: would you care to make a goal (for say 2.6.25 or 26) of
working with me to get the microblaze into mainline?  I think the
community exists to keep things maintained, but I'm guessing that it
would help to have an existing LKMLer or two take a look over the code.
Given the move towards device trees, getting someone from powerpc would
seem to be natural.  Anybody interested?

Steve

Re: Xilinx devicetrees

[Grant Likely]

On 11/28/07, Stephen Neuendorffer <stephen.neuendorffer@xilinx.com> wrote:
> So, John: would you care to make a goal (for say 2.6.25 or 26) of
> working with me to get the microblaze into mainline?  I think the
> community exists to keep things maintained, but I'm guessing that it
> would help to have an existing LKMLer or two take a look over the code.
> Given the move towards device trees, getting someone from powerpc would
> seem to be natural.  Anybody interested?

I'll provide as much support as I can, given that I'm not very
familiar with microblaze.

Cheers,
g

-- 
Grant Likely, B.Sc., P.Eng.
Secret Lab Technologies Ltd.
grant.likely@secretlab.ca
(403) 399-0195

Re: Xilinx devicetrees

[David H. Lynch Jr.]

John Williams wrote:
>
> I'm not sure I agree, here, given that most people building MicroBlaze
> systems are doing so with uClinux-dist (or PetaLinux), you can do a
> full rebuilt, kernel libs apps, in a couple of minutes. Much shorter
> than sythnesis and P&R, that's for sure (and runtime is linear in
> size, unlike P&R :)
I am mostly a lurker in this debate. Pico has V5 cards that we intend to
run MicroBlaze Linux on, but as I would end up with the Linux work and I
have spent most of the past 6 months getting our host software working
under Linux, OpenBSd, and OS X, the time to play with the MicroBlaze
just was not there - but might be soon.

I did take note that Xilinx purportedly has an MMU for the MB now. This
is particularly intriguing.
To cast in my .02 - Pico is likely to either not run the MB on our V5's
or run a fairly bloated one.
uClinux only interests us - because the MB would not run full blown
Linux. Given the possibility that it might, we become less and less
interested, in the smallest MB posible. I know this contradicts many
things I have said about our position on Our PPC Linux, but our clients
who want Linux really want the whole thing. At this moment I do not have
a full appreciation of all the MB instruction options and emulating them
through exception handlers. But my guess is that we would turn
everything on that has any significant impact on linux kernel performance.

I am glad that John mentioned MB/PPC convergence. Particularly with an
MMU it is our hope that our PPC BSP for our V4FX boards ddifferes as
little as possible from our MB BSP for V4/V5 LX boards.

I am not looking to tell someone else how to spend their time - but Pico
would have little interest in dynamically mangling a kernel to adapt to
different CPU parameters. It sounds like alot of work, and alot of
grief. We would design our own CPU first.



>
> My experience tells me that if the microblaze can be configured in a
> particular way, *someone* will want to do it (and still boot linux on
> it!) We still have people building MicroBlaze 3.00 in Spartan2E, with
> EDK 6.3. And autoconfig works! Exceptions on/off, MMU on/off (runtime
> configurable on that?).
I would still venture that with very few exception people are going to
gravitate to the extremes, mostly all off, and mostly all on.
Mostly on would probably run full blown MMU Linux, and mostly off would
either run uClinux or nothing at all. Frankly I am not sure that with
everything mostly off, you wouldn't just pick a much smaller less
powerful core - something 8 or 16 bit that is a fraction of the size,
then the CPU become a logic element, rather than something to run an OS
- I.E. I can build a graphics processor in an FPGA implimenting most
things in hardware, but ploping a cheap (small) CPU in to do general
purpose low performance administrative tasks, that might take more space
in hardware.

Other things that would be higher on my hitlist would be seeing the GCC
MicroBlaze CPU support find its way into distribution GCC's.


-- 
Dave Lynch 					  	    DLA Systems
Software Development:  				         Embedded Linux
717.627.3770 	       dhlii@dlasys.net 	  http://www.dlasys.net
fax: 1.253.369.9244 			           Cell: 1.717.587.7774
Over 25 years' experience in platforms, languages, and technologies too numerous to list.

"Any intelligent fool can make things bigger and more complex... It takes a touch of genius - and a lot of courage to move in the opposite direction."
Albert Einstein

RE: Xilinx devicetrees

[Koss, Mike (Mission Systems)]

[-- Attachment #1: Type: text/plain, Size: 7713 bytes --]

Time for my $.02, since I am heavily weighting my future designs on the
use of the device trees. :) (and b/c I don't check my work e-mail from
home ;P)

________________________________

From: Stephen Neuendorffer [mailto:stephen.neuendorffer@xilinx.com] 
Sent: Sunday, November 25, 2007 6:59 PM
To: David H. Lynch Jr.; Grant Likely; linuxppc-embedded
Subject: RE: Xilinx devicetrees

DL>    I am not expert on this, but at Pico we already store our boot
monitor in the .bit files in BRAM.
DL>     But that is not free.  It is one of the reasons we do not use
u-boot. Our boot monitor must fit into 16K of BRAM.
DL>     Must be able to perform selftests on critical hardware, support
a flash file system, load bit files from flash to the FGA, load and
exectute elf files, allow a small set of user commands, 
DL>  and handle hosted vs. standalone operation.
DL>     And aparently extract the devicetree from a bit file and pass it
to a linux kernel.

SN> Once you can load a bitstream from flash, loading the device tree
from flash
SN> should be practically free.  In any event, why do you do this rather
than just run out of the flash (or a ram copy of the flash?)
 
This is the approach that I am currently taking with our future design.
In our system, we actually have 2 physical systems running Linux inside
the V4 (that's why there are 2 PowerPCs :) ). My current plan would be
to have the device trees stored in FLASH along w/ a single linux image.
Our bootloader on the board would then copy that information from FLASH
to RAM for both systems. This allows us to have two physically different
systems setup with different hardware, but have one linux image with two
different device trees. With a standard linux kernel weighing it at ~
3-5MB (including initramfs) that's a hefty savings in FLASH for us. Plus
it can considerably lower boot time instead of having to verify multiple
large images.

DL>  In static or fairly static hardware, that's fine. Even in somewhat
dynamic hardware with large quantities of startup resources - like a PC.
DL>  But in highly dynamic hardware with fairly limited resources it
starts to become an issue.

SN> As Grant says, the dynamic detection doesn't have to be done in the
boot loader, it could be done in the platform code.  You can largely
ignore 
SN> the device trees, or always boot with a core device tree and figure
it all out later (perhaps using version registers).  I anticipate that 
SN> the 'standard flow' will have standard platform code for any board
that uses a complete device tree.  If you have the need to do something 
SN> extraordinary, then you should feel free to hack away...  However,
It doesn't seem to me to be really necessary in your case, assuming that

SN> the device tree is packaged (somehow, TBD) along with the bitstream.
 
I don't know if packaging the device tree with the bitstream is the best
way to go. It's possible that it could lead to headaches for certain
systems that have security restrictions. The same could be said for
using it w/ the SystemACE to load it into RAM after the image. (which is
what I'm currently doing for my 2 linux images in lieu of a true on-chip
bootloader). I am already taking the security concerns into account for
future revisions of the hardware wrt to using a SystemACE, and am
planning on moving the device trees into NV storage like FLASH.

GL> No, unfortunately they don't deal with the problem you're facing
GL> (which I'm facing also).  But it will be solved if we figure out a
GL> sane way to bind the device tree up with the FPGA bitstream without
GL> consuming FPGA resources.
  
DL>    Note to Xilinx:
DL>       There MUST be some way of binding a device description to a
bit file.
DL>    neither building it into the FPGA fabric nor appending it to the
end
DL> of the bit file are perfect solutions,
DL>    The former is more powerfull and flexible but wastes precious
DL> resources. The later is more complex and puts more burdens on
DL>    software developers, and may be completely unfeasible in some
DL> environments - not mine fortunately.

SN> I don't understand the 'burden on software developers'.  The code to
do this will just be standard code.  The worst that one can say is:
SN> 1) I need several KB additional non volatile storage.  Given the
size of the FPGA bitstream, this can't be a huge constraint.
SN> 2) I can't use compile time optimization based on xparameters as
easily.  Anyone want to implement the alternatives mechanism on ppc and
microblaze?
SN> 3) Some additional boot time.  However, again, this seems marginal.
 
I do agree that using more FPGA resources is not a solution to the
problem. I'm already hitting 80% usage on a FX60 and trying to squeeze
more real estate for storage of the device tree seems silly. Especially
since that would require that every image have this extra hardware built
into it just to support booting a Linux kernel. Why should I have to
have different hardware to boot linux, versus non-kernel, xilkernel, or
other (GHS, LynxOS, etc..)?

DL>    Regardless, something must be done. An odd collection of
devicetree
DL> files co-mingled with a collection of bit files, is little better
than
DL> xparameter files all over the place.

SN> Certainly..  But in a sense, these are all intermediate files on the
path to the image on the board.  That 
SN> (and how it is interpreted by the platform code) should be generated
in a consistent fashion by EDK.  
SN> See my other email for some of the possibilities.  Are there
specific reasons why you think those 
SN> proposals are inadequate?  Now is the time when we can take
criticism, with the goal towards making 
SN> a good end solution.
 
One solution I've been thinking through (in lieu of direct support from
EDK) is to use a tcl script with xps to traverse the hardware tree and
generate the device tree. It seems like it should be relatively trivial
to obtain the information. It's just going to be a pain to write all the
handlers for each different linux driver: temac, interrupt controller,
DMA controller, etc.

In reality the best way to handle this would be to have EDK generate the
device tree as part of the library/bsp build process. Now, what I'd like
to see with regards to this is the ability to change the handler for the
generating a specific device information. An example could be the temac.
If at some point in the future the temac needs new/more information to
support its configuration/run-time then having to get a patch from
Xilinx for a EDK is way too slow. The developers should be giving the
opportunity to inject a new handler into the various parts of the device
tree generation. That way when the kernel patch is submitted, an EDK
device generator patch will be submitted at the same time to keep
everything in sync.

DL>    And once again a plea to ALWAYS make version/capabilities
registers
DL> atleast an optional part of every design.
DL>    Embeddeding a device tree into a design might be fairly
expensive. a
DL> pair of read only 32 bit registers is damn near free - basically the
DL> FPGA equivalent of atmost 64 diodes or resistors.

SN> Actually, device trees actually seem to be cheaper (in the whole
system sense) than such registers.  Unless there is something I don't
understand?
 
The issue here is that the hardware changed and the driver doesn't
support it. I think this would be fixed by having information passed to
the driver in the platform_device struct to specify information, since
its not able to be discerned by the physical hardware information:
version registers, etc.


[-- Attachment #2: Type: text/html, Size: 11617 bytes --]

RE: Xilinx devicetrees

[Stephen Neuendorffer]

[-- Attachment #1: Type: text/plain, Size: 3699 bytes --]




-----Original Message-----
From: Koss, Mike (Mission Systems) [mailto:mike.koss@ngc.com]
Sent: Mon 11/26/2007 7:31 AM
To: Stephen Neuendorffer; David H. Lynch Jr.; Grant Likely; linuxppc-embedded
Subject: RE: Xilinx devicetrees
 
Time for my $.02, since I am heavily weighting my future designs on the
use of the device trees. :) (and b/c I don't check my work e-mail from
home ;P)

________________________________


SN> As Grant says, the dynamic detection doesn't have to be done in the
boot loader, it could be done in the platform code.  You can largely
ignore 
SN> the device trees, or always boot with a core device tree and figure
it all out later (perhaps using version registers).  I anticipate that 
SN> the 'standard flow' will have standard platform code for any board
that uses a complete device tree.  If you have the need to do something 
SN> extraordinary, then you should feel free to hack away...  However,
It doesn't seem to me to be really necessary in your case, assuming that

SN> the device tree is packaged (somehow, TBD) along with the bitstream.
 
> I don't know if packaging the device tree with the bitstream is the best
> way to go. It's possible that it could lead to headaches for certain
> systems that have security restrictions. The same could be said for
> using it w/ the SystemACE to load it into RAM after the image. (which is
> what I'm currently doing for my 2 linux images in lieu of a true on-chip
> bootloader). I am already taking the security concerns into account for
> future revisions of the hardware wrt to using a SystemACE, and am
> planning on moving the device trees into NV storage like FLASH.

'with' not 'in'. either using SystemAce, or a flash image.

> One solution I've been thinking through (in lieu of direct support from
> EDK) is to use a tcl script with xps to traverse the hardware tree and
> generate the device tree. It seems like it should be relatively trivial
> to obtain the information. It's just going to be a pain to write all the
> handlers for each different linux driver: temac, interrupt controller,
> DMA controller, etc.
> In reality the best way to handle this would be to have EDK generate the
> device tree as part of the library/bsp build process. 
We have a python script to do this.  The main problem with just looking at the mhs file is that you lose all the defaults for each IP.  Hence, we've also written a BSP generator to do this.  both are at git://git.xilinx.com/gen-mhs-devtree.py
Once I can verify that they work in the mainline tree, I'll be sending out the patches that make this all work.

> Now, what I'd like
> to see with regards to this is the ability to change the handler for the
> generating a specific device information. An example could be the temac.
> If at some point in the future the temac needs new/more information to
> support its configuration/run-time then having to get a patch from
> Xilinx for a EDK is way too slow. The developers should be giving the
> opportunity to inject a new handler into the various parts of the device
> tree generation. That way when the kernel patch is submitted, an EDK
> device generator patch will be submitted at the same time to keep
> everything in sync.

Interesting idea..  I've been trying to figure out how to architect this better, but it requires some information passing within EDK that isnot today supported.  I don't see at all how to synchronize this with patches to the kernel, tho.  My approach is to describe the hardware as completely and faithfully as we can (given the information in EDK), and let the drivers do whatever with it that they want to.

Steve



[-- Attachment #2: Type: text/html, Size: 4504 bytes --]

RE: Xilinx devicetrees

[Koss, Mike (Mission Systems)]

[-- Attachment #1: Type: text/plain, Size: 5409 bytes --]

________________________________

From: Stephen Neuendorffer [mailto:stephen.neuendorffer@xilinx.com] 
Sent: Wednesday, December 12, 2007 11:52 PM
To: Koss, Mike (Mission Systems); David H. Lynch Jr.; Grant Likely;
linuxppc-embedded
Subject: RE: Xilinx devicetrees

SN> As Grant says, the dynamic detection doesn't have to be done in the
boot loader, it could be done in the platform code.  You can largely
ignore
SN> the device trees, or always boot with a core device tree and figure
it all out later (perhaps using version registers).  I anticipate that
SN> the 'standard flow' will have standard platform code for any board
that uses a complete device tree.  If you have the need to do something
SN> extraordinary, then you should feel free to hack away...  However,
It doesn't seem to me to be really necessary in your case, assuming that

SN> the device tree is packaged (somehow, TBD) along with the bitstream.

> I don't know if packaging the device tree with the bitstream is the
best
> way to go. It's possible that it could lead to headaches for certain
> systems that have security restrictions. The same could be said for
> using it w/ the SystemACE to load it into RAM after the image. (which
is
> what I'm currently doing for my 2 linux images in lieu of a true
on-chip
> bootloader). I am already taking the security concerns into account
for
> future revisions of the hardware wrt to using a SystemACE, and am
> planning on moving the device trees into NV storage like FLASH.

SN> 'with' not 'in'. either using SystemAce, or a flash image.
 
Ah! See, I though that there was the mention of trying to put it into
the bitstream and that is what I was worried about since retrieval of
that from the bitstream file is doable but if it was being stored in the
FPGA somehow and being retrieved there (????) that could lead to
problems for some systems. 

> One solution I've been thinking through (in lieu of direct support
from
> EDK) is to use a tcl script with xps to traverse the hardware tree and
> generate the device tree. It seems like it should be relatively
trivial
> to obtain the information. It's just going to be a pain to write all
the
> handlers for each different linux driver: temac, interrupt controller,
> DMA controller, etc.
> In reality the best way to handle this would be to have EDK generate
the
> device tree as part of the library/bsp build process.
 
SN> We have a python script to do this.  The main problem with just
looking 
SN> at the mhs file is that you lose all the defaults for each IP.
Hence, we've 
SN> also written a BSP generator to do this.  both are at 
SN> git://git.xilinx.com/gen-mhs-devtree.py
SN> Once I can verify that they work in the mainline tree, I'll be
sending out 
SN> the patches that make this all work.

> Now, what I'd like
> to see with regards to this is the ability to change the handler for
the
> generating a specific device information. An example could be the
temac.
> If at some point in the future the temac needs new/more information to
> support its configuration/run-time then having to get a patch from
> Xilinx for a EDK is way too slow. The developers should be giving the
> opportunity to inject a new handler into the various parts of the
device
> tree generation. That way when the kernel patch is submitted, an EDK
> device generator patch will be submitted at the same time to keep
> everything in sync.

SN> Interesting idea..  I've been trying to figure out how to architect
this 
SN> better, but it requires some information passing within EDK that
isnot 
SN> today supported.  I don't see at all how to synchronize this with 
SN> patches to the kernel, tho.  My approach is to describe the hardware

SN> as completely and faithfully as we can (given the information in
EDK), 
SN> and let the drivers do whatever with it that they want to.

You'll have to correct if I'm wrong here, but from what I've been
reading up about EDK and its built-in Tcl interface, one can load their
system.xmp with corresponding mhs and then use Tcl to traverse the
device information to create the same information as found using the
generated BSP. This would allow for a Tcl system that could be setup
such that a main (unchanging, or slowly changing) Tcl script that would
start the EDK definition traversal and upon finding a new device it
would use a registered 'handler' Tcl function. These handlers could be
stored as seperate script/files and would be registered at the start of
the main script either via a config file or by searching a directory and
looking for tags stored at the top of the Tcl files in comments. These
driver handlers would be passed the handle to the system definition and
then know how to gather the information they need to create their entry
in the device tree. This approach gets around the issue of losing
defaults found in the mhs file.

I originally looked at trying to perform the same thing using just
device drivers in EDK, but I think I found some pitfall.  Oh, I think it
was that I would have to choose the OS for the processor and EDK wants
to build the library, but there isn't anything to compile for Linux (or
I wasn't compiling anything for the linux BSP) and that was adding extra
time to the download.bit generation and that is already a long build
process.
 
-- Mike 


[-- Attachment #2: Type: text/html, Size: 8524 bytes --]

RE: Xilinx devicetrees

[Stephen Neuendorffer]

=20

> -----Original Message-----
> From: Koss, Mike (Mission Systems) [mailto:mike.koss@ngc.com]=20
> Sent: Thursday, December 13, 2007 5:49 AM
> To: Stephen Neuendorffer; David H. Lynch Jr.; Grant Likely;=20
> linuxppc-embedded
> Subject: RE: Xilinx devicetrees
>=20
> SN> Interesting idea..  I've been trying to figure out how to=20
> architect this=20
> SN> better, but it requires some information passing within=20
> EDK that isnot=20
> SN> today supported.  I don't see at all how to synchronize this with=20
> SN> patches to the kernel, tho.  My approach is to describe=20
> the hardware=20
> SN> as completely and faithfully as we can (given the=20
> information in EDK),=20
> SN> and let the drivers do whatever with it that they want to.
>=20
> You'll have to correct if I'm wrong here, but from what I've=20
> been reading up about EDK and its built-in Tcl interface, one=20
> can load their system.xmp with corresponding mhs and then use=20
> Tcl to traverse the device information to create the same=20
> information as found using the generated BSP. This would=20
> allow for a Tcl system that could be setup such that a main=20
> (unchanging, or slowly changing) Tcl script that would start=20
> the EDK definition traversal and upon finding a new device it=20
> would use a registered 'handler' Tcl function. These handlers=20
> could be stored as seperate script/files and would be=20
> registered at the start of the main script either via a=20
> config file or by searching a directory and looking for tags=20
> stored at the top of the Tcl files in comments. These driver=20
> handlers would be passed the handle to the system definition=20
> and then know how to gather the information they need to=20
> create their entry in the device tree. This approach gets=20
> around the issue of losing defaults found in the mhs file.

This seems like alot of work, for relatively little benefit.  :)
Instead, I think it is safer to just describe the IP as completely as we
can up front.  Everything else should be done by the Linux driver.  Now
today there is a problem, which is that there isn't a standard way of
having a device describe itself, in the format of a device tree.  This
is something that we'll likely have to add hooks for in the future.

> I originally looked at trying to perform the same thing using=20
> just device drivers in EDK, but I think I found some pitfall.=20
>  Oh, I think it was that I would have to choose the OS for=20
> the processor and EDK wants to build the library, but there=20
> isn't anything to compile for Linux (or I wasn't compiling=20
> anything for the linux BSP) and that was adding extra time to=20
> the download.bit generation and that is already a long build process.

I can't imagine that generating this file is the bottleneck in getting
to a bitstream (or at least, I really hope it isn't!)=20

Steve