[PATCH] doc: document test command

[PATCH] doc: document test command

[Rasmus Villemoes]

Add documentation for the test command, including the newly added =~
operator and some gotchas wrt. the numeric comparisons.

Signed-off-by: Rasmus Villemoes <ravi@prevas.dk>
---

This should be considered on top of
https://lore.kernel.org/u-boot/20250506141035.385756-1-ravi@prevas.dk/
, but everything other than the regex-related stuff could stand on its
own.

My rst-fu is quite weak.

 doc/usage/cmd/setexpr.rst |  5 ++-
 doc/usage/cmd/test.rst    | 95 +++++++++++++++++++++++++++++++++++++++
 doc/usage/index.rst       |  1 +
 3 files changed, 99 insertions(+), 2 deletions(-)
 create mode 100644 doc/usage/cmd/test.rst

diff --git a/doc/usage/cmd/setexpr.rst b/doc/usage/cmd/setexpr.rst
index 593a0ea91e1..5bc37ae50fc 100644
--- a/doc/usage/cmd/setexpr.rst
+++ b/doc/usage/cmd/setexpr.rst
@@ -144,8 +144,9 @@ Configuration
 
 * The *setexpr* command is only available if CMD_SETEXPR=y.
 * The *setexpr fmt* sub-command is only available if CMD_SETEXPR_FMT=y.
-* The *setexpr gsub* and *setexpr sub* sub-commands are only available if
-  CONFIG_REGEX=y.
+* The *setexpr gsub* and *setexpr sub* sub-commands are only available
+  if CONFIG_REGEX=y. For an overview of the supported regex syntax,
+  see :doc:`test`.
 
 Return value
 ------------
diff --git a/doc/usage/cmd/test.rst b/doc/usage/cmd/test.rst
new file mode 100644
index 00000000000..74ee4173639
--- /dev/null
+++ b/doc/usage/cmd/test.rst
@@ -0,0 +1,95 @@
+.. SPDX-License-Identifier: GPL-2.0-or-later
+
+.. index::
+   single: test (command)
+
+test command
+============
+
+Synopsis
+--------
+
+::
+
+    test <str-op> <s>
+    test <s1> <str-cmp> <s2>
+    test <n1> <num-cmp> <n2>
+    test ! <expr>
+    test <expr1> -o <expr2>
+    test <expr1> -a <expr2>
+    test -e <interface> <dev[:part]> <path>
+    test <s> =~ <re>
+
+Description
+-----------
+
+The ``test`` command is similar to the ordinary shell built-in by the
+same name. Unlike in ordinary shells, it cannot be spelled ``[``.
+
+Strings
+~~~~~~~
+
+The string tests ``-n`` and ``-z``, and string comparison operators
+``=``, ``!=``, ``<`` and ``>``, work exactly as in ordinary shells.
+
+Numbers
+~~~~~~~
+
+The number comparison operators ``-lt``, ``-le``, ``-gt``, ``-gt``,
+``-eq`` and ``-ne`` work as in ordinary shells.
+
+*Note*, however, that the numbers are parsed with ``simple_strtol(,
+0)``, meaning that they are treated as decimal unless there is a 0x
+prefix, any errors in parsing are ignored, and parsing stops as soon
+as a non-digit (for the selected base) is encountered. And most U-Boot
+commands that generate "numeric" environment variables store them as
+hexadecimal *without* a 0x prefix.
+
+For example, this is not a correct way of testing whether a given file
+has a size less then 4KiB::
+
+  # Assuming readme.txt exists, sets 'filesize' environment variable
+  $ size mmc 0:1 readme.txt
+  $ if test "$filesize" -lt 4096 ; then ...
+
+If the file size is actually 8000 (decimal), its hexadecimal
+representation, and thus the value of ``$filesize``, is ``1f40``, so
+the comparison that is done ends up being "1 < 4096".
+
+Logic
+~~~~~
+
+The ``!`` operator negates the sense of the test of the expression
+``<expr>``. The ``-o`` and ``-a`` operators perform logical or and
+logical and, respectively, of the two expressions.
+
+File existence
+~~~~~~~~~~~~~~
+
+Like ordinary shells, the ``-e`` operator can be used to test for
+existence of a file. However, the U-Boot version takes three
+arguments: The interface and device number (possibly including a
+partition specification), in addition to the usual path.
+
+Regular expressions
+~~~~~~~~~~~~~~~~~~~
+
+When ``CONFIG_REGEX`` is enabled, an additional operator ``=~`` is
+available. This is similar to the same operator available with bash's
+extended test command ``[[ ]]``. The left operand is a string which is
+matched against the regular expression described by the right operand.
+
+The regular expression engine supports these features:
+
+- Anchoring ``^`` and ``$``, matching at the beginning/end of the
+  string.
+- Matching any single character (including whitespace) using ``.``.
+- Character classes ``[ ]``, including ranges ``[0-9]`` and negation
+  ``[^ /.]``.
+- Grouping ``( )``.
+- Alternation ``|``.
+- Postfix qualifiers ``*``, ``+`` and ``?`` and their non-greedy
+  variants ``*?``, ``+?`` and ``??``
+
+For extracting the parts matching a capture group and/or performing
+substitutions, including back references, see :doc:`setexpr`.
diff --git a/doc/usage/index.rst b/doc/usage/index.rst
index 372ef56c967..c5b45fd9290 100644
--- a/doc/usage/index.rst
+++ b/doc/usage/index.rst
@@ -123,6 +123,7 @@ Shell commands
    cmd/source
    cmd/tcpm
    cmd/temperature
+   cmd/test
    cmd/tftpput
    cmd/trace
    cmd/true
-- 
2.49.0

Re: [PATCH] doc: document test command

[Simon Glass]

On Tue, 6 May 2025 at 22:55, Rasmus Villemoes <ravi@prevas.dk> wrote:
>
> Add documentation for the test command, including the newly added =~
> operator and some gotchas wrt. the numeric comparisons.
>
> Signed-off-by: Rasmus Villemoes <ravi@prevas.dk>
> ---
>
> This should be considered on top of
> https://lore.kernel.org/u-boot/20250506141035.385756-1-ravi@prevas.dk/
> , but everything other than the regex-related stuff could stand on its
> own.
>
> My rst-fu is quite weak.
>
>  doc/usage/cmd/setexpr.rst |  5 ++-
>  doc/usage/cmd/test.rst    | 95 +++++++++++++++++++++++++++++++++++++++
>  doc/usage/index.rst       |  1 +
>  3 files changed, 99 insertions(+), 2 deletions(-)
>  create mode 100644 doc/usage/cmd/test.rst
>

Reviewed-by: Simon Glass <sjg@chromium.org>

Re: [PATCH] doc: document test command

[Quentin Schulz]

Hi Rasmus,

On 5/6/25 10:54 PM, Rasmus Villemoes wrote:
> Add documentation for the test command, including the newly added =~
> operator and some gotchas wrt. the numeric comparisons.
> 
> Signed-off-by: Rasmus Villemoes <ravi@prevas.dk>
> ---
> 
> This should be considered on top of
> https://lore.kernel.org/u-boot/20250506141035.385756-1-ravi@prevas.dk/
> , but everything other than the regex-related stuff could stand on its
> own.
> 
> My rst-fu is quite weak.
> 
>   doc/usage/cmd/setexpr.rst |  5 ++-
>   doc/usage/cmd/test.rst    | 95 +++++++++++++++++++++++++++++++++++++++
>   doc/usage/index.rst       |  1 +
>   3 files changed, 99 insertions(+), 2 deletions(-)
>   create mode 100644 doc/usage/cmd/test.rst
> 
> diff --git a/doc/usage/cmd/setexpr.rst b/doc/usage/cmd/setexpr.rst
> index 593a0ea91e1..5bc37ae50fc 100644
> --- a/doc/usage/cmd/setexpr.rst
> +++ b/doc/usage/cmd/setexpr.rst
> @@ -144,8 +144,9 @@ Configuration
>   
>   * The *setexpr* command is only available if CMD_SETEXPR=y.
>   * The *setexpr fmt* sub-command is only available if CMD_SETEXPR_FMT=y.
> -* The *setexpr gsub* and *setexpr sub* sub-commands are only available if
> -  CONFIG_REGEX=y.
> +* The *setexpr gsub* and *setexpr sub* sub-commands are only available
> +  if CONFIG_REGEX=y. For an overview of the supported regex syntax,
> +  see :doc:`test`.
>   
>   Return value
>   ------------
> diff --git a/doc/usage/cmd/test.rst b/doc/usage/cmd/test.rst
> new file mode 100644
> index 00000000000..74ee4173639
> --- /dev/null
> +++ b/doc/usage/cmd/test.rst
> @@ -0,0 +1,95 @@
> +.. SPDX-License-Identifier: GPL-2.0-or-later
> +
> +.. index::
> +   single: test (command)
> +
> +test command
> +============
> +
> +Synopsis
> +--------
> +
> +::
> +
> +    test <str-op> <s>
> +    test <s1> <str-cmp> <s2>
> +    test <n1> <num-cmp> <n2>
> +    test ! <expr>
> +    test <expr1> -o <expr2>
> +    test <expr1> -a <expr2>
> +    test -e <interface> <dev[:part]> <path>
> +    test <s> =~ <re>
> +
> +Description
> +-----------
> +
> +The ``test`` command is similar to the ordinary shell built-in by the
> +same name. Unlike in ordinary shells, it cannot be spelled ``[``.
> +
> +Strings
> +~~~~~~~
> +
> +The string tests ``-n`` and ``-z``, and string comparison operators
> +``=``, ``!=``, ``<`` and ``>``, work exactly as in ordinary shells.
> +
> +Numbers
> +~~~~~~~
> +
> +The number comparison operators ``-lt``, ``-le``, ``-gt``, ``-gt``,
> +``-eq`` and ``-ne`` work as in ordinary shells.
> +
> +*Note*, however, that the numbers are parsed with ``simple_strtol(,

Maybe use an admonition here?

Something like

.. note:: Numbers are parsed with ``simple_strtol(,0)`` ...

so it's (hopefully) properly highlighted?

> +0)``, meaning that they are treated as decimal unless there is a 0x

``0x`` ?

> +prefix, any errors in parsing are ignored, and parsing stops as soon
> +as a non-digit (for the selected base) is encountered. And most U-Boot
> +commands that generate "numeric" environment variables store them as
> +hexadecimal *without* a 0x prefix.
> +

``0x`` ?

> +For example, this is not a correct way of testing whether a given file
> +has a size less then 4KiB::
> +

s/then/than/

> +  # Assuming readme.txt exists, sets 'filesize' environment variable
> +  $ size mmc 0:1 readme.txt
> +  $ if test "$filesize" -lt 4096 ; then ...
> +
> +If the file size is actually 8000 (decimal), its hexadecimal
> +representation, and thus the value of ``$filesize``, is ``1f40``, so
> +the comparison that is done ends up being "1 < 4096".
> +

All nicely explained! Can you provide an example on how to do this 
properly though?

I assume something like:

if test "0x$filesize" -lt 4096 ; then ...

maybe?

> +Logic
> +~~~~~
> +
> +The ``!`` operator negates the sense of the test of the expression
> +``<expr>``. The ``-o`` and ``-a`` operators perform logical or and

logical OR and logical AND (and maybe even highlight them with backticks 
like ``OR``?) to make them stand out of the sentence since those are 
typical English words it may make it harder to parse the sentence if 
left looking like all other words?

> +logical and, respectively, of the two expressions.
> +

I would have the second sentence on a newline (with a blank link between 
the first and second sentence so that they make separate paragraphs in rST).

> +File existence
> +~~~~~~~~~~~~~~
> +
> +Like ordinary shells, the ``-e`` operator can be used to test for
> +existence of a file. However, the U-Boot version takes three
> +arguments: The interface and device number (possibly including a
> +partition specification), in addition to the usual path.
> +

This could be a list, e.g.:

arguments:

- the interface (e.g. `mmc`)
- the device number (possibly including a partition specification)
- the typical path to the file

> +Regular expressions
> +~~~~~~~~~~~~~~~~~~~
> +
> +When ``CONFIG_REGEX`` is enabled, an additional operator ``=~`` is
> +available. This is similar to the same operator available with bash's
> +extended test command ``[[ ]]``. The left operand is a string which is
> +matched against the regular expression described by the right operand.
> +
> +The regular expression engine supports these features:
> +
> +- Anchoring ``^`` and ``$``, matching at the beginning/end of the
> +  string.
> +- Matching any single character (including whitespace) using ``.``.
> +- Character classes ``[ ]``, including ranges ``[0-9]`` and negation
> +  ``[^ /.]``.
> +- Grouping ``( )``.
> +- Alternation ``|``.
> +- Postfix qualifiers ``*``, ``+`` and ``?`` and their non-greedy
> +  variants ``*?``, ``+?`` and ``??``
> +
> +For extracting the parts matching a capture group and/or performing
> +substitutions, including back references, see :doc:`setexpr`.

Question: Should we provide an example use case for each test subcommand?

Looking good otherwise!
Cheers,
Quentin

Re: [PATCH] doc: document test command

[Rasmus Villemoes]

On Mon, May 12 2025, Quentin Schulz <quentin.schulz@cherry.de> wrote:

> Hi Rasmus,
>

Hi Quentin

Thanks for the review.

> On 5/6/25 10:54 PM, Rasmus Villemoes wrote:
>> +
>> +Numbers
>> +~~~~~~~
>> +
>> +The number comparison operators ``-lt``, ``-le``, ``-gt``, ``-gt``,
>> +``-eq`` and ``-ne`` work as in ordinary shells.
>> +
>> +*Note*, however, that the numbers are parsed with ``simple_strtol(,
>
> Maybe use an admonition here?
>
> Something like
>
> .. note:: Numbers are parsed with ``simple_strtol(,0)`` ...
>
> so it's (hopefully) properly highlighted?

Good idea, as I said I'm not really an rst expert so I mostly just
copy-paste.

>> +0)``, meaning that they are treated as decimal unless there is a 0x
>
> ``0x`` ?
>
>> +prefix, any errors in parsing are ignored, and parsing stops as soon
>> +as a non-digit (for the selected base) is encountered. And most U-Boot
>> +commands that generate "numeric" environment variables store them as
>> +hexadecimal *without* a 0x prefix.
>> +
>
> ``0x`` ?
>

Hm, perhaps, I'm not sure if "code" highlighting is the best for those
0x, but I agree that some highlighting is in order.

>> +For example, this is not a correct way of testing whether a given file
>> +has a size less then 4KiB::
>> +
>
> s/then/than/

Ack.

>> +  # Assuming readme.txt exists, sets 'filesize' environment variable
>> +  $ size mmc 0:1 readme.txt
>> +  $ if test "$filesize" -lt 4096 ; then ...
>> +
>> +If the file size is actually 8000 (decimal), its hexadecimal
>> +representation, and thus the value of ``$filesize``, is ``1f40``, so
>> +the comparison that is done ends up being "1 < 4096".
>> +
>
> All nicely explained! Can you provide an example on how to do this
> properly though?
>
> I assume something like:
>
> if test "0x$filesize" -lt 4096 ; then ...
>
> maybe?

Yes, I think that would be the proper way - but it's really messy,
because if you ever have some variable that happens to contain a hex
number with a 0x prefix, adding another 0x in front would end up
yielding 0 when parsed. So I don't want to give the impression that one
should always blindly prefix 0x (also, sometimes one really does have a
decimal value). So I'm actually leaning on leaving a "positive" example
out, to force a developer to think.

>> +Logic
>> +~~~~~
>> +
>> +The ``!`` operator negates the sense of the test of the expression
>> +``<expr>``. The ``-o`` and ``-a`` operators perform logical or and
>
> logical OR and logical AND (and maybe even highlight them with
> backticks like ``OR``?) to make them stand out of the sentence since
> those are typical English words it may make it harder to parse the
> sentence if left looking like all other words?

Ah yes, ack, I did think about the "logical or and ..." reading quite
oddly, but didn't think of a good way of making them stand
out. Capitalizing is probably a good fix.

>> +logical and, respectively, of the two expressions.
>> +
>
> I would have the second sentence on a newline (with a blank link
> between the first and second sentence so that they make separate
> paragraphs in rST).

Ack.

>
>> +File existence
>> +~~~~~~~~~~~~~~
>> +
>> +Like ordinary shells, the ``-e`` operator can be used to test for
>> +existence of a file. However, the U-Boot version takes three
>> +arguments: The interface and device number (possibly including a
>> +partition specification), in addition to the usual path.
>> +
>
> This could be a list, e.g.:
>
> arguments:
>
> - the interface (e.g. `mmc`)
> - the device number (possibly including a partition specification)
> - the typical path to the file
>

Ack.

>> +Regular expressions
>> +~~~~~~~~~~~~~~~~~~~
>> +
>> +When ``CONFIG_REGEX`` is enabled, an additional operator ``=~`` is
>> +available. This is similar to the same operator available with bash's
>> +extended test command ``[[ ]]``. The left operand is a string which is
>> +matched against the regular expression described by the right operand.
>> +
>> +The regular expression engine supports these features:
>> +
>> +- Anchoring ``^`` and ``$``, matching at the beginning/end of the
>> +  string.
>> +- Matching any single character (including whitespace) using ``.``.
>> +- Character classes ``[ ]``, including ranges ``[0-9]`` and negation
>> +  ``[^ /.]``.
>> +- Grouping ``( )``.
>> +- Alternation ``|``.
>> +- Postfix qualifiers ``*``, ``+`` and ``?`` and their non-greedy
>> +  variants ``*?``, ``+?`` and ``??``
>> +
>> +For extracting the parts matching a capture group and/or performing
>> +substitutions, including back references, see :doc:`setexpr`.
>
> Question: Should we provide an example use case for each test subcommand?

I don't think so. I'd assume any U-Boot developer to be reasonably
familiar with the ordinary shell test, so I think it's better to just
emphasize the differences and gotchas than providing "shell 101"
material.

I'll fold in your suggestions, then resend the whole regex series with
this one properly included as patch 12.

Thanks,
Rasmus

Re: [PATCH] doc: document test command

[Quentin Schulz]

Hi Rasmus,

On 5/12/25 11:31 AM, Rasmus Villemoes wrote:
> On Mon, May 12 2025, Quentin Schulz <quentin.schulz@cherry.de> wrote:
> 
>> Hi Rasmus,
>>
> 
> Hi Quentin
> 
> Thanks for the review.
> 
>> On 5/6/25 10:54 PM, Rasmus Villemoes wrote:
>>> +
>>> +Numbers
>>> +~~~~~~~
>>> +
>>> +The number comparison operators ``-lt``, ``-le``, ``-gt``, ``-gt``,
>>> +``-eq`` and ``-ne`` work as in ordinary shells.
>>> +
>>> +*Note*, however, that the numbers are parsed with ``simple_strtol(,
>>
>> Maybe use an admonition here?
>>
>> Something like
>>
>> .. note:: Numbers are parsed with ``simple_strtol(,0)`` ...
>>
>> so it's (hopefully) properly highlighted?
> 

And re-reading myself, we probably want:

.. note::
    Numbers are parsed with ``simple_strtol(,0)`` ...

here :)

[...]

>>> +prefix, any errors in parsing are ignored, and parsing stops as soon
>>> +as a non-digit (for the selected base) is encountered. And most U-Boot
>>> +commands that generate "numeric" environment variables store them as
>>> +hexadecimal *without* a 0x prefix.
>>> +
>>
>> ``0x`` ?
>>
> 
> Hm, perhaps, I'm not sure if "code" highlighting is the best for those
> 0x, but I agree that some highlighting is in order.
> 

Fair, could be also `0x` if you prefer how this renders, I don't think 
U-Boot is too picky about those.

[...]

>>> +  # Assuming readme.txt exists, sets 'filesize' environment variable
>>> +  $ size mmc 0:1 readme.txt
>>> +  $ if test "$filesize" -lt 4096 ; then ...
>>> +
>>> +If the file size is actually 8000 (decimal), its hexadecimal
>>> +representation, and thus the value of ``$filesize``, is ``1f40``, so
>>> +the comparison that is done ends up being "1 < 4096".
>>> +
>>
>> All nicely explained! Can you provide an example on how to do this
>> properly though?
>>
>> I assume something like:
>>
>> if test "0x$filesize" -lt 4096 ; then ...
>>
>> maybe?
> 
> Yes, I think that would be the proper way - but it's really messy,
> because if you ever have some variable that happens to contain a hex
> number with a 0x prefix, adding another 0x in front would end up
> yielding 0 when parsed. So I don't want to give the impression that one
> should always blindly prefix 0x (also, sometimes one really does have a
> decimal value). So I'm actually leaning on leaving a "positive" example
> out, to force a developer to think.
> 

This is terrible :/

We now need to figure out if a command modifies environment variables to 
contain hex values, returns hex values, or simply decimal values.

Nothing that is coming from your doing in this patch though :)

[...]

>>> +Regular expressions
>>> +~~~~~~~~~~~~~~~~~~~
>>> +
>>> +When ``CONFIG_REGEX`` is enabled, an additional operator ``=~`` is
>>> +available. This is similar to the same operator available with bash's
>>> +extended test command ``[[ ]]``. The left operand is a string which is
>>> +matched against the regular expression described by the right operand.
>>> +
>>> +The regular expression engine supports these features:
>>> +
>>> +- Anchoring ``^`` and ``$``, matching at the beginning/end of the
>>> +  string.
>>> +- Matching any single character (including whitespace) using ``.``.
>>> +- Character classes ``[ ]``, including ranges ``[0-9]`` and negation
>>> +  ``[^ /.]``.
>>> +- Grouping ``( )``.
>>> +- Alternation ``|``.
>>> +- Postfix qualifiers ``*``, ``+`` and ``?`` and their non-greedy
>>> +  variants ``*?``, ``+?`` and ``??``
>>> +
>>> +For extracting the parts matching a capture group and/or performing
>>> +substitutions, including back references, see :doc:`setexpr`.
>>
>> Question: Should we provide an example use case for each test subcommand?
> 
> I don't think so. I'd assume any U-Boot developer to be reasonably
> familiar with the ordinary shell test, so I think it's better to just
> emphasize the differences and gotchas than providing "shell 101"
> material.
> 

Fair enough!

Quentin