CELF Project Proposal - Device tree support for QEMU system emulation.

CELF Project Proposal - Device tree support for QEMU system emulation.

[Rob Landley]

For background of CELF project proposals, see:

  http://elinux.org/CELF_Open_Project_Proposal_2010

Summary:

Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
create board emulations on the fly (at runtime) from the same data files the 
Linux kernel uses to attach drivers to hardware.

Proposer:

Rob Landley

Description:

Currently, the QEMU is emulating system boards via hardwired .c files, which 
explicitly set up the resources for each emulation in a separate C function.  
In theory, QEMU could parse the same device tree data format the Linux kernel 
uses to set up its hardware resources,  and then pass hardware resources along 
to kernels it invokes through its built-in bootloader (I.E. with the -kernel 
option).  This could allow new boards to be added to qemu simply by supplying 
device tree files at runtime (assuming emulations for the appropriate 
peripherals had already been implemented in QEMU).

The Device Tree format is a reasonably generic data file describing hardware 
layout.  It is documented in the linux kernel source at:

  Documentation/powerpc/booting-without-of.txt

It started as the data structure Open Firmware used to describe supported 
hardware to operating systems, and was picked up by bootloaders such as u-
boot.  It allows the Linux kernel to parse a generic data structure at boot 
time to configure itself for the current hardware layout, instead of hardwiring 
board support in individual .c files.

Device trees are created using an ascii format to describe a board layout, 
which is converted into a flattened binary representation by dtc (the "device 
tree compiler", included in current linux kernel sources in scripts/dtc).  A 
bootloader supporting device trees loads the flattened device tree into memory 
as a binary blob, and passes the linux kernel a pointer to this blob in a 
register.  The kernel then uses a built-in device tree parser to understand 
the board's hardware layout and initialize itself.  (Depending on kernel 
.config, this information may also be queried from userspace via a /proc 
interface.)

At the device tree BOF at OLS in 2008, a number of developers expressed 
interest in extending device tree support to other architectures (such as arm, 
mips, and sh4).  As a result, device tree development was moved off of the 
PowerPC mailing list to its own list, for the purpose of genericizing it to 
more architectures:

  https://lists.ozlabs.org/listinfo/devicetree-discuss

The Linux MAINTAINERS entry for device tree support is:

  OPEN FIRMWARE AND FLATTENED DEVICE TREE
  M:      Grant Likely <grant.likely@secretlab.ca>
  L:      devicetree-discuss@lists.ozlabs.org
  W:      http://fdt.secretlab.ca
  S:      Maintained
  F:      drivers/of
  F:      include/linux/of*.h
  K:      of_get_property

In theory, at some future date the kernel will no longer need hardwired .c 
files describing the layout of boards for any of these architectures.  Instead 
it can have a device tree for each board, which can be statically linked into 
the kernel binary if necessary.

If the kenrnel doesn't need board support hardwared in .c files, there's no 
reason QEMU needs it either.

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds

[Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Rob Landley]

For background of CELF project proposals, see:

  http://elinux.org/CELF_Open_Project_Proposal_2010

Summary:

Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
create board emulations on the fly (at runtime) from the same data files the 
Linux kernel uses to attach drivers to hardware.

Proposer:

Rob Landley

Description:

Currently, the QEMU is emulating system boards via hardwired .c files, which 
explicitly set up the resources for each emulation in a separate C function.  
In theory, QEMU could parse the same device tree data format the Linux kernel 
uses to set up its hardware resources,  and then pass hardware resources along 
to kernels it invokes through its built-in bootloader (I.E. with the -kernel 
option).  This could allow new boards to be added to qemu simply by supplying 
device tree files at runtime (assuming emulations for the appropriate 
peripherals had already been implemented in QEMU).

The Device Tree format is a reasonably generic data file describing hardware 
layout.  It is documented in the linux kernel source at:

  Documentation/powerpc/booting-without-of.txt

It started as the data structure Open Firmware used to describe supported 
hardware to operating systems, and was picked up by bootloaders such as u-
boot.  It allows the Linux kernel to parse a generic data structure at boot 
time to configure itself for the current hardware layout, instead of hardwiring 
board support in individual .c files.

Device trees are created using an ascii format to describe a board layout, 
which is converted into a flattened binary representation by dtc (the "device 
tree compiler", included in current linux kernel sources in scripts/dtc).  A 
bootloader supporting device trees loads the flattened device tree into memory 
as a binary blob, and passes the linux kernel a pointer to this blob in a 
register.  The kernel then uses a built-in device tree parser to understand 
the board's hardware layout and initialize itself.  (Depending on kernel 
.config, this information may also be queried from userspace via a /proc 
interface.)

At the device tree BOF at OLS in 2008, a number of developers expressed 
interest in extending device tree support to other architectures (such as arm, 
mips, and sh4).  As a result, device tree development was moved off of the 
PowerPC mailing list to its own list, for the purpose of genericizing it to 
more architectures:

  https://lists.ozlabs.org/listinfo/devicetree-discuss

The Linux MAINTAINERS entry for device tree support is:

  OPEN FIRMWARE AND FLATTENED DEVICE TREE
  M:      Grant Likely <grant.likely@secretlab.ca>
  L:      devicetree-discuss@lists.ozlabs.org
  W:      http://fdt.secretlab.ca
  S:      Maintained
  F:      drivers/of
  F:      include/linux/of*.h
  K:      of_get_property

In theory, at some future date the kernel will no longer need hardwired .c 
files describing the layout of boards for any of these architectures.  Instead 
it can have a device tree for each board, which can be statically linked into 
the kernel binary if necessary.

If the kenrnel doesn't need board support hardwared in .c files, there's no 
reason QEMU needs it either.

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Anthony Liguori]

Rob Landley wrote:
> For background of CELF project proposals, see:
>
>   http://elinux.org/CELF_Open_Project_Proposal_2010
>
> Summary:
>
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
> create board emulations on the fly (at runtime) from the same data files the 
> Linux kernel uses to attach drivers to hardware.
>   

See

http://thread.gmane.org/gmane.comp.emulators.qemu/44869

I'm not sure why Paul never pushed it but I think he was able to create 
the syborg board purely from a device tree.

Regards,

Anthony Liguori

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Anthony Liguori]

Rob Landley wrote:
> For background of CELF project proposals, see:
>
>   http://elinux.org/CELF_Open_Project_Proposal_2010
>
> Summary:
>
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
> create board emulations on the fly (at runtime) from the same data files the 
> Linux kernel uses to attach drivers to hardware.
>   

See

http://thread.gmane.org/gmane.comp.emulators.qemu/44869

I'm not sure why Paul never pushed it but I think he was able to create 
the syborg board purely from a device tree.

Regards,

Anthony Liguori

Re: [Celinux-dev] CELF Project Proposal - Device tree support for QEMU system emulation.

[Tim Bird]

Rob Landley wrote:
> For background of CELF project proposals, see:
> 
>   http://elinux.org/CELF_Open_Project_Proposal_2010
> 
> Summary:
> 
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
> create board emulations on the fly (at runtime) from the same data files the 
> Linux kernel uses to attach drivers to hardware.

I've created a page for this proposal at:
http://elinux.org/CELF_Project_Proposal/Add_Device_Tree_emulation_support_to_QEMU

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

[Qemu-devel] Re: [Celinux-dev] CELF Project Proposal - Device tree support for QEMU system emulation.

[Tim Bird]

Rob Landley wrote:
> For background of CELF project proposals, see:
> 
>   http://elinux.org/CELF_Open_Project_Proposal_2010
> 
> Summary:
> 
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can 
> create board emulations on the fly (at runtime) from the same data files the 
> Linux kernel uses to attach drivers to hardware.

I've created a page for this proposal at:
http://elinux.org/CELF_Project_Proposal/Add_Device_Tree_emulation_support_to_QEMU

=============================
Tim Bird
Architecture Group Chair, CE Linux Forum
Senior Staff Engineer, Sony Corporation of America
=============================

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Blue Swirl]

On Thu, Dec 17, 2009 at 4:46 AM, Rob Landley <rob@landley.net> wrote:
> For background of CELF project proposals, see:
>
>  http://elinux.org/CELF_Open_Project_Proposal_2010
>
> Summary:
>
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can
> create board emulations on the fly (at runtime) from the same data files the
> Linux kernel uses to attach drivers to hardware.
>
> Proposer:
>
> Rob Landley
>
> Description:
>
> Currently, the QEMU is emulating system boards via hardwired .c files, which
> explicitly set up the resources for each emulation in a separate C function.
> In theory, QEMU could parse the same device tree data format the Linux kernel
> uses to set up its hardware resources,  and then pass hardware resources along
> to kernels it invokes through its built-in bootloader (I.E. with the -kernel
> option).  This could allow new boards to be added to qemu simply by supplying
> device tree files at runtime (assuming emulations for the appropriate
> peripherals had already been implemented in QEMU).

In addition to -kernel case, the device tree should be passed to
OpenBIOS, which would generate OF tree from DT. This would be useful
for OSes other than Linux.

Another interesting case is sun4v hypervisor, which uses a similar
device tree. Some DT transformation would be required in QEMU.
_______________________________________________
devicetree-discuss mailing list
devicetree-discuss@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/devicetree-discuss

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Blue Swirl]

On Thu, Dec 17, 2009 at 4:46 AM, Rob Landley <rob@landley.net> wrote:
> For background of CELF project proposals, see:
>
>  http://elinux.org/CELF_Open_Project_Proposal_2010
>
> Summary:
>
> Integrate a flattened device tree parser into the emulator QEMU, so QEMU can
> create board emulations on the fly (at runtime) from the same data files the
> Linux kernel uses to attach drivers to hardware.
>
> Proposer:
>
> Rob Landley
>
> Description:
>
> Currently, the QEMU is emulating system boards via hardwired .c files, which
> explicitly set up the resources for each emulation in a separate C function.
> In theory, QEMU could parse the same device tree data format the Linux kernel
> uses to set up its hardware resources,  and then pass hardware resources along
> to kernels it invokes through its built-in bootloader (I.E. with the -kernel
> option).  This could allow new boards to be added to qemu simply by supplying
> device tree files at runtime (assuming emulations for the appropriate
> peripherals had already been implemented in QEMU).

In addition to -kernel case, the device tree should be passed to
OpenBIOS, which would generate OF tree from DT. This would be useful
for OSes other than Linux.

Another interesting case is sun4v hypervisor, which uses a similar
device tree. Some DT transformation would be required in QEMU.

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Paul Brook]

> http://thread.gmane.org/gmane.comp.emulators.qemu/44869
> 
> I'm not sure why Paul never pushed it but I think he was able to create
> the syborg board purely from a device tree.

The patches referenced above include purely device-tree based Syborg and 
Stellaris boards.

It works fairly nicely for some embedded boards, but gets fairly hairy when 
you start considering more complex topologies/machines. I particular the PCI 
OF/FDT bindings don't really fit what we need for qemu. Obviously full qdev 
conversion is a prerequisite for device tree based configuration, and at that 
time qdev was still in its infancy.

Part of the reason for posting that patch was to demonstrate the abstraction 
layers required for the internal device model. Previous patches tended to be 
based on the pre-existing PC commandline options, which IMO is a mistake. 
Likewise if (like my implementation) you focus on embedded machines you risk 
serious problems trying to extend to server class machines.

Paul

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Paul Brook]

> http://thread.gmane.org/gmane.comp.emulators.qemu/44869
> 
> I'm not sure why Paul never pushed it but I think he was able to create
> the syborg board purely from a device tree.

The patches referenced above include purely device-tree based Syborg and 
Stellaris boards.

It works fairly nicely for some embedded boards, but gets fairly hairy when 
you start considering more complex topologies/machines. I particular the PCI 
OF/FDT bindings don't really fit what we need for qemu. Obviously full qdev 
conversion is a prerequisite for device tree based configuration, and at that 
time qdev was still in its infancy.

Part of the reason for posting that patch was to demonstrate the abstraction 
layers required for the internal device model. Previous patches tended to be 
based on the pre-existing PC commandline options, which IMO is a mistake. 
Likewise if (like my implementation) you focus on embedded machines you risk 
serious problems trying to extend to server class machines.

Paul

Re: [Qemu-devel] CELF Project Proposal - Device tree support for QEMU system emulation.

[Rob Landley]

On Thursday 17 December 2009 14:29:47 Anthony Liguori wrote:
> Rob Landley wrote:
> > For background of CELF project proposals, see:
> >
> >   http://elinux.org/CELF_Open_Project_Proposal_2010
> >
> > Summary:
> >
> > Integrate a flattened device tree parser into the emulator QEMU, so QEMU
> > can create board emulations on the fly (at runtime) from the same data
> > files the Linux kernel uses to attach drivers to hardware.
>
> See
>
> http://thread.gmane.org/gmane.comp.emulators.qemu/44869
>
> I'm not sure why Paul never pushed it but I think he was able to create
> the syborg board purely from a device tree.

He'd be a great person to sponsor to get it finished/updated and checked 
in, then.  (Or his company, code sourcry, would.)  Me, I just want to use the 
result.

By the way, device trees were a topic at this year's kernel summit, and LWN 
did an excellent write-up, as usual: http://lwn.net/Articles/357487/

(If it does get done, somebody needs to write a HOWTO...)

Rob
-- 
Latency is more important than throughput. It's that simple. - Linus Torvalds