[PATCH 0/2] Syntax highlighting

[PATCH 0/2] Syntax highlighting

[Stewart Smith]

Since http://pygments.org/ (what Sphinx uses for syntax highlighting)
doesn't do DTS yet, implement DTS syntax highlighting and then use it
(and the standard C source highlighting).

I'll be submitting the DTS highlighting upstream too, but having it in
tree will ensure it works for people not running a future version of
pygments.

Stewart Smith (2):
  Enable syntax highlighting of device tree source
  Syntax highlight C and DTS

 source/conf.py                |  6 +++
 source/devicenodes.rst        | 18 ++++-----
 source/devicetree-basics.rst  | 14 ++++---
 source/extensions/DtsLexer.py | 89 +++++++++++++++++++++++++++++++++++++++++++
 source/flattened-format.rst   |  6 +--
 5 files changed, 115 insertions(+), 18 deletions(-)
 create mode 100644 source/extensions/DtsLexer.py

-- 
2.7.4

[PATCH 1/2] Enable syntax highlighting of device tree source

[Stewart Smith]

Signed-off-by: Stewart Smith <stewart-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 source/conf.py                |  6 +++
 source/extensions/DtsLexer.py | 89 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 95 insertions(+)
 create mode 100644 source/extensions/DtsLexer.py

diff --git a/source/conf.py b/source/conf.py
index 67edcb8..8f71117 100644
--- a/source/conf.py
+++ b/source/conf.py
@@ -23,6 +23,12 @@ import time
 
 sys.path.append(os.path.abspath('extensions'))
 
+from DtsLexer import DtsLexer
+
+def setup(app):
+    from sphinx.highlighting import lexers
+    lexers['dts'] = DtsLexer()
+
 
 # -- General configuration ------------------------------------------------
 
diff --git a/source/extensions/DtsLexer.py b/source/extensions/DtsLexer.py
new file mode 100644
index 0000000..eb0e7d6
--- /dev/null
+++ b/source/extensions/DtsLexer.py
@@ -0,0 +1,89 @@
+#!/usr/bin/python
+#
+# Contributors Listed Below - COPYRIGHT 2016
+# [+] International Business Machines Corp.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions are
+# met:
+#
+# * Redistributions of source code must retain the above copyright
+#   notice, this list of conditions and the following disclaimer.
+#
+# * Redistributions in binary form must reproduce the above copyright
+#   notice, this list of conditions and the following disclaimer in the
+#  documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+# "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+# LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+# A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+# OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+# SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+# LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+# DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+# THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+from pygments.lexer import RegexLexer, bygroups, include
+from pygments.token import *
+
+__all__ = ['DtsLexer']
+
+class DtsLexer(RegexLexer):
+    name = 'DTS'
+    aliases = ['dts', 'device-tree']
+    filenames = ['*.dts']
+
+    tokens = {
+        'root': [
+            include('comments'),
+            (r'/memreserve/', Keyword, ('memreserve')),
+            (r'^/dts-v1/;$', Keyword),
+            (r'\s+', Text),
+            include('node'),
+        ],
+        'comments': [
+            (r'//(\n|(.|\n)*?[^\\]\n)', Comment.Single),
+            (r'/(\\\n)?[*](.|\n)*?[*](\\\n)?/', Comment.Multiline),
+        ],
+        'memreserve': [
+            include('comments'),
+            include('integers'),
+            (r'\s+', Text),
+            (r';', Punctuation, '#pop')
+        ],
+        'integers': [
+            (r'0x[0-9a-fA-F]+', Number.Hex),
+            (r'0[0-7]+', Number.Oct),
+            (r'\d+', Number.Integer),
+        ],
+        'node': [
+            (r'\s+', Text),
+            (r'((?:[0-9a-zA-Z,._+-]+):)?(\s*)([0-9a-zA-Z,._+-]+)(@[0-9a-zA-Z,._+-]+)?(\s*)({)', bygroups(Name.Label, Text, Name.Class, Name.Function, Text, Punctuation), ('node-content')),
+            (r'(/)(\s+)({)', bygroups(Keyword, Text, Punctuation) , ('node-content')),
+        ],
+        'node-content': [
+            include('comments'),
+            include('node'),
+            (r'\s+', Text),
+            (r'([0-9a-zA-Z,._+-]+)(:)', bygroups(Name.Label, Punctuation)),
+            (r'([\#0-9a-zA-Z,._+-]+)(\s*)(=)(\s*)', bygroups(Name.Function, Text, Operator, Text), ('value')),
+            (r'([\#0-9a-zA-Z,._+-]+)(;)', bygroups(Name.Function, Punctuation)),
+            (r'};', Punctuation, ('#pop')),
+        ],
+        'value': [
+            include('integers'),
+            include('comments'),
+            (r'(\&)([a-zA-Z0-9_-]+)', bygroups(Operator, Text)),
+            (r'<', Punctuation, '#push'),
+            (r'>', Punctuation, '#pop'),
+            (r'\[', Punctuation, '#push'),
+            (r'\]', Punctuation, '#pop'),
+            (r'".*"', String),
+            (r';', Punctuation, '#pop'),
+            (r'\s+', Text),
+            (r',', Punctuation)
+        ],
+    }
-- 
2.7.4

[PATCH 2/2] Syntax highlight C and DTS

[Stewart Smith]

Signed-off-by: Stewart Smith <stewart-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org>
---
 source/devicenodes.rst       | 18 +++++++++---------
 source/devicetree-basics.rst | 14 ++++++++------
 source/flattened-format.rst  |  6 +++---
 3 files changed, 20 insertions(+), 18 deletions(-)

diff --git a/source/devicenodes.rst b/source/devicenodes.rst
index e12aa95..78b5bdb 100644
--- a/source/devicenodes.rst
+++ b/source/devicenodes.rst
@@ -91,12 +91,12 @@ considering a string as a device path, shall detect and use the alias.
 
 **Example**
 
-::
+.. code-block:: dts
 
     aliases {
         serial0 = "/simple-bus@fe000000/serial@llc500";
         ethernet0 = "/simple-bus@fe000000/ethernet@31c000";
-    }
+    };
 
 Given the alias serial0, a client program can look at the aliases node
 and determine the alias refers to the device path
@@ -168,7 +168,7 @@ and ``#size-cells = <2>``.
 
 **Example #1**
 
-::
+.. code-block:: dts
 
     memory@0 {
         device_type = "memory";
@@ -178,7 +178,7 @@ and ``#size-cells = <2>``.
 
 **Example #2**
 
-::
+.. code-block:: dts
 
     memory@0 {
         device_type = "memory";
@@ -234,7 +234,7 @@ time. It shall be a child of the root node.
 
 **Example**
 
-::
+.. code-block:: dts
 
     chosen {
         bootargs = "root=/dev/nfs rw nfsroot=192.168.1.1 console=ttyS0,115200";
@@ -618,7 +618,7 @@ Example
 
 Here is an example of a ``/cpus`` node with one child cpu node:
 
-::
+.. code-block:: dts
 
     cpus {
         #address-cells = <1>;
@@ -680,7 +680,7 @@ Example
 See the following example of a devicetree representation of two CPUs,
 each with their own on-chip L2 and a shared L3.
 
-::
+.. code-block:: dts
 
     cpus {
         #address-cells = <1>;
@@ -709,7 +709,7 @@ each with their own on-chip L2 and a shared L3.
                     cache-unified;
                     cache-size = <0x40000>; // 256 KB
                     cache-sets = <0x400>; // 1024
-                    cache-block-size =
+                    cache-block-size = <32>;
                     cache-level = <3>;
                 };
             };
@@ -730,7 +730,7 @@ each with their own on-chip L2 and a shared L3.
                 cache-unified;
                 cache-size = <0x40000>; // 256 KB
                 cache-sets = <0x400>; // 1024
-                cache-line-size = <32> // 32 bytes
+                cache-line-size = <32>; // 32 bytes
                 next-level-cache = <&L3>; // phandle to L3
             };
         };
diff --git a/source/devicetree-basics.rst b/source/devicetree-basics.rst
index ba5281c..be80af3 100644
--- a/source/devicetree-basics.rst
+++ b/source/devicetree-basics.rst
@@ -463,7 +463,7 @@ Example:
 
    See the following devicetree excerpt:
 
-   ::
+   .. code-block:: dts
 
       pic@10000000 {
          phandle = <1>;
@@ -473,9 +473,11 @@ Example:
    A *phandle* value of 1 is defined. Another device node could reference
    the pic node with a phandle value of 1:
 
-   ::
+   .. code-block:: dts
 
-      interrupt-parent = <1>;
+      another-device-node {
+        interrupt-parent = <1>;
+      };
 
 .. note:: Older versions of devicetrees may be encountered that contain a
    deprecated form of this property called ``linux,phandle``. For
@@ -557,7 +559,7 @@ Example:
 
    See the following devicetree excerpt:
 
-   ::
+   .. code-block:: dts
 
       soc {
          #address-cells = <1>;
@@ -671,7 +673,7 @@ Description:
 
 Address Translation Example:
 
-   ::
+   .. code-block:: dts
 
        soc {
           compatible = "simple-bus";
@@ -1084,7 +1086,7 @@ interrupt controller.
 
 .. _example-interrupt-mapping:
 
-::
+.. code-block:: dts
 
    soc {
       compatible = "simple-bus";
diff --git a/source/flattened-format.rst b/source/flattened-format.rst
index ba68f8a..ccac933 100644
--- a/source/flattened-format.rst
+++ b/source/flattened-format.rst
@@ -70,7 +70,7 @@ big-endian format.
 
 **Flattened Devicetree Header Fields**
 
-::
+.. code-block:: c
 
         struct fdt_header {
             uint32_t magic;
@@ -208,7 +208,7 @@ The memory reservation block consists of a list of pairs of 64-bit
 big-endian integers, each pair being represented by the following C
 structure.
 
-::
+.. code-block:: c
 
     struct fdt_reserve_entry {
         uint64_t address;
@@ -273,7 +273,7 @@ The five token types are as follows:
    describing the property. This data consists first of the property’s
    length and name represented as the following C structure:
 
-   ::
+   .. code-block:: c
 
       struct {
           uint32_t len;
-- 
2.7.4

Re: [PATCH 0/2] Syntax highlighting

[Rob Herring]

On Fri, Oct 28, 2016 at 1:12 AM, Stewart Smith
<stewart-23VcF4HTsmIX0ybBhKVfKdBPR1lH4CV8@public.gmane.org> wrote:
> Since http://pygments.org/ (what Sphinx uses for syntax highlighting)
> doesn't do DTS yet, implement DTS syntax highlighting and then use it
> (and the standard C source highlighting).
>
> I'll be submitting the DTS highlighting upstream too, but having it in
> tree will ensure it works for people not running a future version of
> pygments.
>
> Stewart Smith (2):
>   Enable syntax highlighting of device tree source
>   Syntax highlight C and DTS

Sorry for the delay. Looks nice and I've applied both. Thanks.

Rob