[Qemu-devel] CMSIS SVD based peripheral definitions

[Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

CMSIS SVD
---------

The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced a new technology for implementing peripherals, based on standard CMSIS SVD definitions (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).

The SVD files are large XML files produced by the silicon vendors, and generally are considered the final hardware reference for the Cortex-M devices, so they are expected to provide the most accurate peripheral emulation.

For convenience, the original SVD files are converted to JSON files, which are generally easier to parse. 

There is one file for each sub-family; the files currently used by GNU ARM Eclipse QEMU are grouped in the devices folders:

	https://github.com/gnuarmeclipse/qemu/tree/gnuarmeclipse-dev/gnuarmeclipse/devices


Please note that some devices may include multiple instances of similar peripherals, for example multiple timers, with each instance slightly different from the others. The SVD files include all these differences, so strictly following the content of the SVD files is mandatory; extracting a definition common to all instances may seem attractive, but it is not realistic, since it may not be accurate for all instances.


Peripheral/register/bitfield
----------------------------

The basic objects used to implement peripherals are the 'registers'; a peripheral is actually an array of registers, each with its value.

For convenience, registers can be viewed as collections of bitfields; bitfield objects do not have their own values, reading a bitfield refers to the parent register, which is masked, shifted and finally returned.

Accessing bitfields is quite straightforward:

const char *enabling_bit_name = "/machine/mcu/stm32/RCC/AHB1ENR/GPIOAEN";
state->enabling_bit = OBJECT(cm_device_by_name(enabling_bit_name));
// ...
if (register_bitfield_is_non_zero(state->enabling_bit)) {
  // ...
}


Generated code
--------------

In addition to using vendor SVD files, GNU ARM Eclipse QEMU goes one step further, by generating most of the peripheral code, to the point that new peripherals can be added simply be adding the generated files to the project.

Examples of the files currently used are in sub subfolders of the devices/support folder:

	https://github.com/gnuarmeclipse/qemu/tree/gnuarmeclipse-dev/gnuarmeclipse/devices/support


As per the current STM32 implementation, to avoid redundancy, each peripheral file includes definitions for all families; adding a new device implies generating the code for the new device sub-family, and copy/paste-ing the required code in the peripheral implementation.

An example of such a peripheral implementation is the SYSFCG:

	https://github.com/gnuarmeclipse/qemu/blob/gnuarmeclipse-dev/hw/cortexm/stm32/syscfg.c



Supported devices
-----------------

GNU ARM Eclipse QEMU 2.8 supports the following boards:

  Maple                LeafLab Arduino-style STM32 microcontroller board (r5)
  NUCLEO-F103RB        ST Nucleo Development Board for STM32 F1 series
  NUCLEO-F411RE        ST Nucleo Development Board for STM32 F4 series
  NetduinoGo           Netduino GoBus Development Board with STM32F4
  NetduinoPlus2        Netduino Development Board with STM32F4
  OLIMEXINO-STM32      Olimex Maple (Arduino-like) Development Board
  STM32-E407           Olimex Development Board for STM32F407ZGT6
  STM32-H103           Olimex Header Board for STM32F103RBT6
  STM32-P103           Olimex Prototype Board for STM32F103RBT6
  STM32-P107           Olimex Prototype Board for STM32F107VCT6
  STM32F0-Discovery    ST Discovery kit for STM32F051 lines
  STM32F4-Discovery    ST Discovery kit for STM32F407/417 lines
  STM32F429I-Discovery ST Discovery kit for STM32F429/439 lines

Supported MCUs:
  STM32F051R8
  STM32F103RB
  STM32F107VC
  STM32F405RG
  STM32F407VG
  STM32F407ZG
  STM32F411RE
  STM32F429ZI

Functionally, the boards include animated LEDs and active push buttons (reset and user).


Test projects (blinky) for all supported boards are available from

	https://github.com/gnuarmeclipse/eclipse-qemu-test-projects


Conclusion
----------

Although implementing peripherals remains a challenge, using the SVD definitions, plus the tools to generate code, significantly improve and simplify the process.

Unfortunately, SVD files are available only for Cortex-M devices, but I think that, when not available, creating the JSON files and using the automated code generator is still easier than manually implementing the peripherals.


Personally I plan to use this technology to re-define the system Cortex-M devices, which, right now, are improperly implemented.


Regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 28 December 2016 at 18:49, Liviu Ionescu <ilg@livius.net> wrote:
> The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced
> a new technology for implementing peripherals, based on standard
> CMSIS SVD definitions
> (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).
>
> The SVD files are large XML files produced by the silicon vendors,
> and generally are considered the final hardware reference for
> the Cortex-M devices, so they are expected to provide the most
> accurate peripheral emulation.

This certainly seems like a good idea in principle, if there's
enough information in the SVD files to reliably create a
complete board model. The obvious issue that comes to mind
is licensing, though -- what licenses are the SVD files under,
and would those be compatible with QEMU's GPLv2 ?

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 9 Jan 2017, at 18:16, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> On 28 December 2016 at 18:49, Liviu Ionescu <ilg@livius.net> wrote:
>> The latest release of GNU ARM Eclipse QEMU (2.8.0-20161227) introduced
>> a new technology for implementing peripherals, based on standard
>> CMSIS SVD definitions
>> (http://www.keil.com/pack/doc/CMSIS/SVD/html/index.html).
>> 
>> The SVD files are large XML files produced by the silicon vendors,
>> and generally are considered the final hardware reference for
>> the Cortex-M devices, so they are expected to provide the most
>> accurate peripheral emulation.
> 
> This certainly seems like a good idea in principle, if there's
> enough information in the SVD files to reliably create a
> complete board model.

the general answer is that this information should be there.

in practice, it mostly is. however, the SVD specs are quite lax and each vendor understands them slightly differently.

to compensate for these differences, my workflow added an extra step of patching the original content. the patch files are minimal compared to the original files, and the effort to create them is perfectly acceptable.

> The obvious issue that comes to mind
> is licensing, though -- what licenses are the SVD files under,
> and would those be compatible with QEMU's GPLv2 ?

that's a common question with cmsis files. I could not find any specific license applying to svd files, so, since these files are provided as part of the arm cmsis packs, I would consider the generic license, which, starting with the latest cmsis release, was changed to Apache.

anyway, in my opinion, after experimenting both with classical (switch based) implementations and table based (peripheral/register/bitfield) implementations, if I would have to deal with a device which does not have the vendor SVD files, I would probably prefer to write the JSON file by hand and allow the automated tools to handle the rest.

actually I'll have to do this when I'll rewrite the Cortex-M system peripherals, not included in the vendor provided SVDs.


regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 9 January 2017 at 16:51, Liviu Ionescu <ilg@livius.net> wrote:
>> On 9 Jan 2017, at 18:16, Peter Maydell <peter.maydell@linaro.org> wrote:
>> The obvious issue that comes to mind
>> is licensing, though -- what licenses are the SVD files under,
>> and would those be compatible with QEMU's GPLv2 ?
>
> that's a common question with cmsis files. I could not find
> any specific license applying to svd files, so, since these
> files are provided as part of the arm cmsis packs, I would
> consider the generic license, which, starting with the latest
> cmsis release, was changed to Apache.

The FSF's view is that Apache 2.0 is not compatible with GPLv2:
https://www.gnu.org/licenses/license-list.html#apache2

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 9 Jan 2017, at 19:31, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> FSF's view is that Apache 2.0 is not compatible with GPLv2:

I can't dispute this, but does't the GPLv2 requirements apply to the qemu executable only? my JSON files are not compiled into the binary, but are data files, read by the executable when required. and anyway, the original license covers the original XML files, mine are not only JSON, but are also slightly different.


regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 9 January 2017 at 17:47, Liviu Ionescu <ilg@livius.net> wrote:
> I can't dispute this, but does't the GPLv2 requirements apply to
> the qemu executable only? my JSON files are not compiled into the
> binary, but are data files, read by the executable when required.
> and anyway, the original license covers the original XML files,
> mine are not only JSON, but are also slightly different.

I'm not a lawyer, and this isn't legal advice, but merely
my own opinion:
 (1) if you're reading in the data files at runtime then there's
a spectrum between "just reading data provided by the user which
didn't ship with QEMU, and QEMU works OK without this particular
data" (clearly OK) and "reading data that shipped with QEMU
and where QEMU won't work at all without it" (the data files
are clearly part of the program as a whole and the GPL
applies to them even if they don't happen to be binaries);
where you stand legally probably depends on where on the
spectrum you are.

 (2) your JSON files are a derived work from the original
XML, so the license of those XML files (whatever it is) still
applies to them. Making modifications or a mechanical
transformation of the original files is not sufficient to
allow you to discard the license...

With my project maintainer hat on, I prefer to avoid
grey areas, because it avoids potential trouble later on.

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 16 Jan 2017, at 20:17, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> "reading data that shipped with QEMU
> and where QEMU won't work at all without it" (the data files
> are clearly part of the program as a whole and the GPL
> applies to them even if they don't happen to be binaries);

I'm not a lawyer either, but isn't this case similar to packing binary modules with the Linux kernel? 

Regards,

Liviu

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 16 January 2017 at 18:59, Liviu Ionescu <ilg@livius.net> wrote:
>
>> On 16 Jan 2017, at 20:17, Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> "reading data that shipped with QEMU
>> and where QEMU won't work at all without it" (the data files
>> are clearly part of the program as a whole and the GPL
>> applies to them even if they don't happen to be binaries);
>
> I'm not a lawyer either, but isn't this case similar to packing
> binary modules with the Linux kernel?

Yes, and doing that is somewhere between "grey area" and
"GPL violation". Views of the kernel authors on that
topic if you want to dig into the detail:
http://www.tldp.org/HOWTO/Module-HOWTO/copyright.html
http://yarchive.net/comp/linux/gpl_modules.html

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 16 Jan 2017, at 21:23, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> doing that is somewhere between "grey area" and
> "GPL violation".

too bad for GPL. :-(

the fact is that I'll continue to improve support for Cortex-M in my branch of QEMU, and the SVD definitions are part of it.

if you are not comfortable with it, no problem, it will remain a separate branch; by all means I'm not trying to force you to accept merging it into the mainstream.

if you don't mind, I'll continue to post messages regarding the progress of GNU ARM Eclipse QEMU to inform the other developers on the new solutions, to possibly save some duplicate efforts, and, hopefully, get some feedback.


regards,

Liviu



 

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Dr. David Alan Gilbert]

* Peter Maydell (peter.maydell@linaro.org) wrote:
> On 9 January 2017 at 17:47, Liviu Ionescu <ilg@livius.net> wrote:
> > I can't dispute this, but does't the GPLv2 requirements apply to
> > the qemu executable only? my JSON files are not compiled into the
> > binary, but are data files, read by the executable when required.
> > and anyway, the original license covers the original XML files,
> > mine are not only JSON, but are also slightly different.
> 
> I'm not a lawyer, and this isn't legal advice, but merely
> my own opinion:
>  (1) if you're reading in the data files at runtime then there's
> a spectrum between "just reading data provided by the user which
> didn't ship with QEMU, and QEMU works OK without this particular
> data" (clearly OK) and "reading data that shipped with QEMU
> and where QEMU won't work at all without it" (the data files
> are clearly part of the program as a whole and the GPL
> applies to them even if they don't happen to be binaries);
> where you stand legally probably depends on where on the
> spectrum you are.
> 
>  (2) your JSON files are a derived work from the original
> XML, so the license of those XML files (whatever it is) still
> applies to them. Making modifications or a mechanical
> transformation of the original files is not sufficient to
> allow you to discard the license...
> 
> With my project maintainer hat on, I prefer to avoid
> grey areas, because it avoids potential trouble later on.

Being able to read a machine description from a file at run time
sounds reasonable, as do tools to manipulate those descriptions.
I don't see why such a description file would be any different
from a firmware file or config file, especially if you were to think
of QEMU in this case more like a simulation environment
which I think is what Liviu is trying to use it as.

I agree actually including/distributing those description files is a completely
different issue if they're derived from someone elses files;
but that wouldn't stop you creating some simple examples (e.g.
the existing related machine types) from scratch.

Dave

> thanks
> -- PMM
> 
--
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Peter Maydell]

On 17 January 2017 at 10:42, Dr. David Alan Gilbert <dgilbert@redhat.com> wrote:
> Being able to read a machine description from a file at run time
> sounds reasonable, as do tools to manipulate those descriptions.
> I don't see why such a description file would be any different
> from a firmware file or config file, especially if you were to think
> of QEMU in this case more like a simulation environment
> which I think is what Liviu is trying to use it as.

Yes, I agree that's fine and I'd be happy with that. That
means reading the XML file at runtime that the user provides...

> I agree actually including/distributing those description files is a completely
> different issue if they're derived from someone elses files;

...not preprocessing the XML into JSON, losing the license
info and attribution and shipping the JSON with QEMU. It's
those aspects that I think drop into the 'grey area'.

thanks
-- PMM

Re: [Qemu-devel] CMSIS SVD based peripheral definitions

[Liviu Ionescu]

> On 17 Jan 2017, at 13:09, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> losing the license info and attribution

inside the SVD files there is no license info or attribution.

the files themselves were created by the silicon vendors, but are distributed by ARM, inside the CMSIS Packs.

the original CMSIS license stated:

---
1.1 CMSIS DELIVERABLES
ARM hereby grants to you, subject to the terms and conditions of this Licence, a non-exclusive, non-transferable licence, to use and copy the CMSIS Deliverables for the purpose of:

(i) subject to clause 1.5(i), developing, having developed, manufacturing, having manufactured, offering to sell, selling, supplying or otherwise distributing products that comply with the CMSIS Deliverables; ...

1.5 CONDITIONS ON REDISTRIBUTION.

(i) If you distribute (directly or through your customers and authorised distributors) the products you have created pursuant to Clause 1.1 (i) you agree: (a) not to use ARM’s name, logo or trademarks to market any or all of the products created under Clause 1.1 (i); (b) to preserve any copyright notices included in the CMSIS Deliverables; and (c) to ensure your customers and authorised distributors comply with this Clause 1.5(i).
---

starting with CMSIS 5, ARM switched to Apache 2, exactly to relax licensing. as far as I can tell, there are no "patent, trademark, and attribution notices from the Source form of the Work" related to the SVD files.

if ARM wants these files used largely, but FSF refuses them, I think FSF has a problem.


regards,

Liviu