public inbox for dtrace@lists.linux.dev
 help / color / mirror / Atom feed
* [PATCH v2] lexer: support // comments
@ 2025-07-23  5:11 Kris Van Hees
  2025-07-23  9:45 ` Alan Maguire
  0 siblings, 1 reply; 3+ messages in thread
From: Kris Van Hees @ 2025-07-23  5:11 UTC (permalink / raw)
  To: dtrace, dtrace-devel

Suggested-by: Ruud van der Pas <ruud.vanderpas@oracle.com>
Suggested-by: Alan Maguire <alan.maguire@oracle.com>
Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
---
 libdtrace/dt_lex.l                            | 11 ++++++++++-
 test/unittest/lexer/tst.boc-in-line-comment.d | 16 ++++++++++++++++
 test/unittest/lexer/tst.boc-in-line-comment.r |  5 +++++
 test/unittest/lexer/tst.eoc-in-line-comment.d | 16 ++++++++++++++++
 test/unittest/lexer/tst.eoc-in-line-comment.r |  5 +++++
 test/unittest/lexer/tst.eof-in-line-comment.d | 16 ++++++++++++++++
 test/unittest/lexer/tst.eof-in-line-comment.r |  5 +++++
 test/unittest/lexer/tst.lc-in-line-comment.d  | 16 ++++++++++++++++
 test/unittest/lexer/tst.lc-in-line-comment.r  |  5 +++++
 test/unittest/lexer/tst.line-comment.d        | 16 ++++++++++++++++
 test/unittest/lexer/tst.line-comment.r        |  5 +++++
 11 files changed, 115 insertions(+), 1 deletion(-)
 create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.d
 create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.r
 create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.d
 create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.r
 create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.d
 create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.r
 create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.d
 create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.r
 create mode 100644 test/unittest/lexer/tst.line-comment.d
 create mode 100644 test/unittest/lexer/tst.line-comment.r

diff --git a/libdtrace/dt_lex.l b/libdtrace/dt_lex.l
index 9d502912..e8f3bc8c 100644
--- a/libdtrace/dt_lex.l
+++ b/libdtrace/dt_lex.l
@@ -35,6 +35,7 @@ int yydebug;
  * S2 - D program outer scope (probe specifiers and declarations)
  * S3 - D control line parsing (i.e. after ^# is seen but before \n)
  * S4 - D control line scan (locate control directives only and invoke S3)
+ * S5 - D line comments (i.e. skip everything until end of line)
  * SIDENT - identifiers and comments only (after -> and .).  (We switch to
  *          SIDENT only from state S0: changing this would require new code
  *          to track the state to switch back to.)
@@ -46,7 +47,7 @@ int yydebug;
 %n 600		/* maximum states */
 %option yylineno
 
-%s S0 S1 S2 S3 S4 SIDENT
+%s S0 S1 S2 S3 S4 S5 SIDENT
 
 RGX_AGG		"@"[a-zA-Z_][0-9a-zA-Z_]*
 RGX_PSPEC	[-$:a-zA-Z_.?*\\\[\]!][-$:0-9a-zA-Z_.`?*\\\[\]!]*
@@ -408,6 +409,11 @@ if (yypcb->pcb_token != 0) {
 			BEGIN(S1);
 		}
 
+<S0,S2,SIDENT>"//"	{
+			yypcb->pcb_cstate = (YYSTATE);
+			BEGIN(S5);
+		}
+
 <S0>^{RGX_INTERP} |
 <S2>^{RGX_INTERP} ;	/* discard any #! lines */
 
@@ -548,6 +554,9 @@ if (yypcb->pcb_token != 0) {
 <S1>.|\n	; /* discard */
 <S1><<EOF>>	yyerror("end-of-file encountered before matching */\n");
 
+<S5>\n		BEGIN(yypcb->pcb_cstate);
+<S5>.		; /* discard */
+
 <S2>{RGX_PSPEC}	{
 			/*
 			 * S2 has an ambiguity because RGX_PSPEC includes '*'
diff --git a/test/unittest/lexer/tst.boc-in-line-comment.d b/test/unittest/lexer/tst.boc-in-line-comment.d
new file mode 100644
index 00000000..61f5961f
--- /dev/null
+++ b/test/unittest/lexer/tst.boc-in-line-comment.d
@@ -0,0 +1,16 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: Line comments cannot contain begin-of-comment markers.
+ */
+
+// Comment /*
+BEGIN
+{
+	exit(0);
+}
diff --git a/test/unittest/lexer/tst.boc-in-line-comment.r b/test/unittest/lexer/tst.boc-in-line-comment.r
new file mode 100644
index 00000000..d57f017a
--- /dev/null
+++ b/test/unittest/lexer/tst.boc-in-line-comment.r
@@ -0,0 +1,5 @@
+                   FUNCTION:NAME
+                          :BEGIN 
+
+-- @@stderr --
+dtrace: script 'test/unittest/lexer/tst.boc-in-line-comment.d' matched 1 probe
diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.d b/test/unittest/lexer/tst.eoc-in-line-comment.d
new file mode 100644
index 00000000..8fe2ff04
--- /dev/null
+++ b/test/unittest/lexer/tst.eoc-in-line-comment.d
@@ -0,0 +1,16 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: Line comments cannot contain end-of-comment markers.
+ */
+
+// Comment */
+BEGIN
+{
+	exit(0);
+}
diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.r b/test/unittest/lexer/tst.eoc-in-line-comment.r
new file mode 100644
index 00000000..884cd554
--- /dev/null
+++ b/test/unittest/lexer/tst.eoc-in-line-comment.r
@@ -0,0 +1,5 @@
+                   FUNCTION:NAME
+                          :BEGIN 
+
+-- @@stderr --
+dtrace: script 'test/unittest/lexer/tst.eoc-in-line-comment.d' matched 1 probe
diff --git a/test/unittest/lexer/tst.eof-in-line-comment.d b/test/unittest/lexer/tst.eof-in-line-comment.d
new file mode 100644
index 00000000..a9207693
--- /dev/null
+++ b/test/unittest/lexer/tst.eof-in-line-comment.d
@@ -0,0 +1,16 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: End-of-file in a line comment is an error.
+ */
+
+BEGIN
+{
+	exit(0);
+}
+// Comment
\ No newline at end of file
diff --git a/test/unittest/lexer/tst.eof-in-line-comment.r b/test/unittest/lexer/tst.eof-in-line-comment.r
new file mode 100644
index 00000000..fdf259e0
--- /dev/null
+++ b/test/unittest/lexer/tst.eof-in-line-comment.r
@@ -0,0 +1,5 @@
+                   FUNCTION:NAME
+                          :BEGIN 
+
+-- @@stderr --
+dtrace: script 'test/unittest/lexer/tst.eof-in-line-comment.d' matched 1 probe
diff --git a/test/unittest/lexer/tst.lc-in-line-comment.d b/test/unittest/lexer/tst.lc-in-line-comment.d
new file mode 100644
index 00000000..0332e1ae
--- /dev/null
+++ b/test/unittest/lexer/tst.lc-in-line-comment.d
@@ -0,0 +1,16 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: Line comments cannot contain a line commens marker.
+ */
+
+// Comment //
+BEGIN
+{
+	exit(0);
+}
diff --git a/test/unittest/lexer/tst.lc-in-line-comment.r b/test/unittest/lexer/tst.lc-in-line-comment.r
new file mode 100644
index 00000000..3075b987
--- /dev/null
+++ b/test/unittest/lexer/tst.lc-in-line-comment.r
@@ -0,0 +1,5 @@
+                   FUNCTION:NAME
+                          :BEGIN 
+
+-- @@stderr --
+dtrace: script 'test/unittest/lexer/tst.lc-in-line-comment.d' matched 1 probe
diff --git a/test/unittest/lexer/tst.line-comment.d b/test/unittest/lexer/tst.line-comment.d
new file mode 100644
index 00000000..4deb66f4
--- /dev/null
+++ b/test/unittest/lexer/tst.line-comment.d
@@ -0,0 +1,16 @@
+/*
+ * Oracle Linux DTrace.
+ * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
+ * Licensed under the Universal Permissive License v 1.0 as shown at
+ * http://oss.oracle.com/licenses/upl.
+ */
+
+/*
+ * ASSERTION: // comments are supported.
+ */
+
+// exit(1);
+BEGIN // exit(1);
+{ // exit(1);
+	exit(0); // exit(1);
+} // exit(1);
diff --git a/test/unittest/lexer/tst.line-comment.r b/test/unittest/lexer/tst.line-comment.r
new file mode 100644
index 00000000..d4e4c325
--- /dev/null
+++ b/test/unittest/lexer/tst.line-comment.r
@@ -0,0 +1,5 @@
+                   FUNCTION:NAME
+                          :BEGIN 
+
+-- @@stderr --
+dtrace: script 'test/unittest/lexer/tst.line-comment.d' matched 1 probe
-- 
2.45.2


^ permalink raw reply related	[flat|nested] 3+ messages in thread
* Re: [PATCH v2] lexer: support // comments
  2025-07-23  5:11 [PATCH v2] lexer: support // comments Kris Van Hees
@ 2025-07-23  9:45 ` Alan Maguire
  2025-07-23 12:57   ` Kris Van Hees
  0 siblings, 1 reply; 3+ messages in thread
From: Alan Maguire @ 2025-07-23  9:45 UTC (permalink / raw)
  To: Kris Van Hees, dtrace, dtrace-devel

On 23/07/2025 06:11, Kris Van Hees wrote:
> Suggested-by: Ruud van der Pas <ruud.vanderpas@oracle.com>
> Suggested-by: Alan Maguire <alan.maguire@oracle.com>
> Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>

Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
Tested-by: Alan Maguire <alan.maguire@oracle.com>

a few small things below regarding assertions as these are all positive
tests now I think (allowing /* after // etc), so the assertions need
updating, but aside from that looks great!

> ---
>  libdtrace/dt_lex.l                            | 11 ++++++++++-
>  test/unittest/lexer/tst.boc-in-line-comment.d | 16 ++++++++++++++++
>  test/unittest/lexer/tst.boc-in-line-comment.r |  5 +++++
>  test/unittest/lexer/tst.eoc-in-line-comment.d | 16 ++++++++++++++++
>  test/unittest/lexer/tst.eoc-in-line-comment.r |  5 +++++
>  test/unittest/lexer/tst.eof-in-line-comment.d | 16 ++++++++++++++++
>  test/unittest/lexer/tst.eof-in-line-comment.r |  5 +++++
>  test/unittest/lexer/tst.lc-in-line-comment.d  | 16 ++++++++++++++++
>  test/unittest/lexer/tst.lc-in-line-comment.r  |  5 +++++
>  test/unittest/lexer/tst.line-comment.d        | 16 ++++++++++++++++
>  test/unittest/lexer/tst.line-comment.r        |  5 +++++
>  11 files changed, 115 insertions(+), 1 deletion(-)
>  create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.d
>  create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.r
>  create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.d
>  create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.r
>  create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.d
>  create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.r
>  create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.d
>  create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.r
>  create mode 100644 test/unittest/lexer/tst.line-comment.d
>  create mode 100644 test/unittest/lexer/tst.line-comment.r
> 
> diff --git a/libdtrace/dt_lex.l b/libdtrace/dt_lex.l
> index 9d502912..e8f3bc8c 100644
> --- a/libdtrace/dt_lex.l
> +++ b/libdtrace/dt_lex.l
> @@ -35,6 +35,7 @@ int yydebug;
>   * S2 - D program outer scope (probe specifiers and declarations)
>   * S3 - D control line parsing (i.e. after ^# is seen but before \n)
>   * S4 - D control line scan (locate control directives only and invoke S3)
> + * S5 - D line comments (i.e. skip everything until end of line)
>   * SIDENT - identifiers and comments only (after -> and .).  (We switch to
>   *          SIDENT only from state S0: changing this would require new code
>   *          to track the state to switch back to.)
> @@ -46,7 +47,7 @@ int yydebug;
>  %n 600		/* maximum states */
>  %option yylineno
>  
> -%s S0 S1 S2 S3 S4 SIDENT
> +%s S0 S1 S2 S3 S4 S5 SIDENT
>  
>  RGX_AGG		"@"[a-zA-Z_][0-9a-zA-Z_]*
>  RGX_PSPEC	[-$:a-zA-Z_.?*\\\[\]!][-$:0-9a-zA-Z_.`?*\\\[\]!]*
> @@ -408,6 +409,11 @@ if (yypcb->pcb_token != 0) {
>  			BEGIN(S1);
>  		}
>  
> +<S0,S2,SIDENT>"//"	{
> +			yypcb->pcb_cstate = (YYSTATE);
> +			BEGIN(S5);
> +		}
> +
>  <S0>^{RGX_INTERP} |
>  <S2>^{RGX_INTERP} ;	/* discard any #! lines */
>  
> @@ -548,6 +554,9 @@ if (yypcb->pcb_token != 0) {
>  <S1>.|\n	; /* discard */
>  <S1><<EOF>>	yyerror("end-of-file encountered before matching */\n");
>  
> +<S5>\n		BEGIN(yypcb->pcb_cstate);
> +<S5>.		; /* discard */
> +
>  <S2>{RGX_PSPEC}	{
>  			/*
>  			 * S2 has an ambiguity because RGX_PSPEC includes '*'
> diff --git a/test/unittest/lexer/tst.boc-in-line-comment.d b/test/unittest/lexer/tst.boc-in-line-comment.d
> new file mode 100644
> index 00000000..61f5961f
> --- /dev/null
> +++ b/test/unittest/lexer/tst.boc-in-line-comment.d
> @@ -0,0 +1,16 @@
> +/*
> + * Oracle Linux DTrace.
> + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> + * Licensed under the Universal Permissive License v 1.0 as shown at
> + * http://oss.oracle.com/licenses/upl.
> + */
> +
> +/*
> + * ASSERTION: Line comments cannot contain begin-of-comment markers.
> + */

this needs to be updated I think? i.e. Like C, line comments can contain
begin-of-comment markers.

> +
> +// Comment /*
> +BEGIN
> +{
> +	exit(0);
> +}
> diff --git a/test/unittest/lexer/tst.boc-in-line-comment.r b/test/unittest/lexer/tst.boc-in-line-comment.r
> new file mode 100644
> index 00000000..d57f017a
> --- /dev/null
> +++ b/test/unittest/lexer/tst.boc-in-line-comment.r
> @@ -0,0 +1,5 @@
> +                   FUNCTION:NAME
> +                          :BEGIN 
> +
> +-- @@stderr --
> +dtrace: script 'test/unittest/lexer/tst.boc-in-line-comment.d' matched 1 probe
> diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.d b/test/unittest/lexer/tst.eoc-in-line-comment.d
> new file mode 100644
> index 00000000..8fe2ff04
> --- /dev/null
> +++ b/test/unittest/lexer/tst.eoc-in-line-comment.d
> @@ -0,0 +1,16 @@
> +/*
> + * Oracle Linux DTrace.
> + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> + * Licensed under the Universal Permissive License v 1.0 as shown at
> + * http://oss.oracle.com/licenses/upl.
> + */
> +
> +/*
> + * ASSERTION: Line comments cannot contain end-of-comment markers.

same here.

> + */
> +
> +// Comment */
> +BEGIN
> +{
> +	exit(0);
> +}
> diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.r b/test/unittest/lexer/tst.eoc-in-line-comment.r
> new file mode 100644
> index 00000000..884cd554
> --- /dev/null
> +++ b/test/unittest/lexer/tst.eoc-in-line-comment.r
> @@ -0,0 +1,5 @@
> +                   FUNCTION:NAME
> +                          :BEGIN 
> +
> +-- @@stderr --
> +dtrace: script 'test/unittest/lexer/tst.eoc-in-line-comment.d' matched 1 probe
> diff --git a/test/unittest/lexer/tst.eof-in-line-comment.d b/test/unittest/lexer/tst.eof-in-line-comment.d
> new file mode 100644
> index 00000000..a9207693
> --- /dev/null
> +++ b/test/unittest/lexer/tst.eof-in-line-comment.d
> @@ -0,0 +1,16 @@
> +/*
> + * Oracle Linux DTrace.
> + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> + * Licensed under the Universal Permissive License v 1.0 as shown at
> + * http://oss.oracle.com/licenses/upl.
> + */
> +
> +/*
> + * ASSERTION: End-of-file in a line comment is an error.

again this doesn't seem like an error test?

> + */
> +
> +BEGIN
> +{
> +	exit(0);
> +}
> +// Comment
> \ No newline at end of file
> diff --git a/test/unittest/lexer/tst.eof-in-line-comment.r b/test/unittest/lexer/tst.eof-in-line-comment.r
> new file mode 100644
> index 00000000..fdf259e0
> --- /dev/null
> +++ b/test/unittest/lexer/tst.eof-in-line-comment.r
> @@ -0,0 +1,5 @@
> +                   FUNCTION:NAME
> +                          :BEGIN 
> +
> +-- @@stderr --
> +dtrace: script 'test/unittest/lexer/tst.eof-in-line-comment.d' matched 1 probe
> diff --git a/test/unittest/lexer/tst.lc-in-line-comment.d b/test/unittest/lexer/tst.lc-in-line-comment.d
> new file mode 100644
> index 00000000..0332e1ae
> --- /dev/null
> +++ b/test/unittest/lexer/tst.lc-in-line-comment.d
> @@ -0,0 +1,16 @@
> +/*
> + * Oracle Linux DTrace.
> + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> + * Licensed under the Universal Permissive License v 1.0 as shown at
> + * http://oss.oracle.com/licenses/upl.
> + */
> +
> +/*
> + * ASSERTION: Line comments cannot contain a line commens marker.

same here

> + */
> +
> +// Comment //
> +BEGIN
> +{
> +	exit(0);
> +}
> diff --git a/test/unittest/lexer/tst.lc-in-line-comment.r b/test/unittest/lexer/tst.lc-in-line-comment.r
> new file mode 100644
> index 00000000..3075b987
> --- /dev/null
> +++ b/test/unittest/lexer/tst.lc-in-line-comment.r
> @@ -0,0 +1,5 @@
> +                   FUNCTION:NAME
> +                          :BEGIN 
> +
> +-- @@stderr --
> +dtrace: script 'test/unittest/lexer/tst.lc-in-line-comment.d' matched 1 probe
> diff --git a/test/unittest/lexer/tst.line-comment.d b/test/unittest/lexer/tst.line-comment.d
> new file mode 100644
> index 00000000..4deb66f4
> --- /dev/null
> +++ b/test/unittest/lexer/tst.line-comment.d
> @@ -0,0 +1,16 @@
> +/*
> + * Oracle Linux DTrace.
> + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> + * Licensed under the Universal Permissive License v 1.0 as shown at
> + * http://oss.oracle.com/licenses/upl.
> + */
> +
> +/*
> + * ASSERTION: // comments are supported.
> + */
> +
> +// exit(1);
> +BEGIN // exit(1);
> +{ // exit(1);
> +	exit(0); // exit(1);
> +} // exit(1);
> diff --git a/test/unittest/lexer/tst.line-comment.r b/test/unittest/lexer/tst.line-comment.r
> new file mode 100644
> index 00000000..d4e4c325
> --- /dev/null
> +++ b/test/unittest/lexer/tst.line-comment.r
> @@ -0,0 +1,5 @@
> +                   FUNCTION:NAME
> +                          :BEGIN 
> +
> +-- @@stderr --
> +dtrace: script 'test/unittest/lexer/tst.line-comment.d' matched 1 probe


^ permalink raw reply	[flat|nested] 3+ messages in thread
* Re: [PATCH v2] lexer: support // comments
  2025-07-23  9:45 ` Alan Maguire
@ 2025-07-23 12:57   ` Kris Van Hees
  0 siblings, 0 replies; 3+ messages in thread
From: Kris Van Hees @ 2025-07-23 12:57 UTC (permalink / raw)
  To: Alan Maguire; +Cc: Kris Van Hees, dtrace, dtrace-devel

On Wed, Jul 23, 2025 at 10:45:54AM +0100, Alan Maguire wrote:
> On 23/07/2025 06:11, Kris Van Hees wrote:
> > Suggested-by: Ruud van der Pas <ruud.vanderpas@oracle.com>
> > Suggested-by: Alan Maguire <alan.maguire@oracle.com>
> > Signed-off-by: Kris Van Hees <kris.van.hees@oracle.com>
> 
> Reviewed-by: Alan Maguire <alan.maguire@oracle.com>
> Tested-by: Alan Maguire <alan.maguire@oracle.com>
> 
> a few small things below regarding assertions as these are all positive
> tests now I think (allowing /* after // etc), so the assertions need
> updating, but aside from that looks great!

Oops, yes, was a bit hasty changing them.

(The purist in me still feels that the ODX people did the right thing in being
more pedantic on what is reported as an error in //-comments, but I can see
how that can be annoying to people - and we want to be be nice :))

> > ---
> >  libdtrace/dt_lex.l                            | 11 ++++++++++-
> >  test/unittest/lexer/tst.boc-in-line-comment.d | 16 ++++++++++++++++
> >  test/unittest/lexer/tst.boc-in-line-comment.r |  5 +++++
> >  test/unittest/lexer/tst.eoc-in-line-comment.d | 16 ++++++++++++++++
> >  test/unittest/lexer/tst.eoc-in-line-comment.r |  5 +++++
> >  test/unittest/lexer/tst.eof-in-line-comment.d | 16 ++++++++++++++++
> >  test/unittest/lexer/tst.eof-in-line-comment.r |  5 +++++
> >  test/unittest/lexer/tst.lc-in-line-comment.d  | 16 ++++++++++++++++
> >  test/unittest/lexer/tst.lc-in-line-comment.r  |  5 +++++
> >  test/unittest/lexer/tst.line-comment.d        | 16 ++++++++++++++++
> >  test/unittest/lexer/tst.line-comment.r        |  5 +++++
> >  11 files changed, 115 insertions(+), 1 deletion(-)
> >  create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.d
> >  create mode 100644 test/unittest/lexer/tst.boc-in-line-comment.r
> >  create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.d
> >  create mode 100644 test/unittest/lexer/tst.eoc-in-line-comment.r
> >  create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.d
> >  create mode 100644 test/unittest/lexer/tst.eof-in-line-comment.r
> >  create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.d
> >  create mode 100644 test/unittest/lexer/tst.lc-in-line-comment.r
> >  create mode 100644 test/unittest/lexer/tst.line-comment.d
> >  create mode 100644 test/unittest/lexer/tst.line-comment.r
> > 
> > diff --git a/libdtrace/dt_lex.l b/libdtrace/dt_lex.l
> > index 9d502912..e8f3bc8c 100644
> > --- a/libdtrace/dt_lex.l
> > +++ b/libdtrace/dt_lex.l
> > @@ -35,6 +35,7 @@ int yydebug;
> >   * S2 - D program outer scope (probe specifiers and declarations)
> >   * S3 - D control line parsing (i.e. after ^# is seen but before \n)
> >   * S4 - D control line scan (locate control directives only and invoke S3)
> > + * S5 - D line comments (i.e. skip everything until end of line)
> >   * SIDENT - identifiers and comments only (after -> and .).  (We switch to
> >   *          SIDENT only from state S0: changing this would require new code
> >   *          to track the state to switch back to.)
> > @@ -46,7 +47,7 @@ int yydebug;
> >  %n 600		/* maximum states */
> >  %option yylineno
> >  
> > -%s S0 S1 S2 S3 S4 SIDENT
> > +%s S0 S1 S2 S3 S4 S5 SIDENT
> >  
> >  RGX_AGG		"@"[a-zA-Z_][0-9a-zA-Z_]*
> >  RGX_PSPEC	[-$:a-zA-Z_.?*\\\[\]!][-$:0-9a-zA-Z_.`?*\\\[\]!]*
> > @@ -408,6 +409,11 @@ if (yypcb->pcb_token != 0) {
> >  			BEGIN(S1);
> >  		}
> >  
> > +<S0,S2,SIDENT>"//"	{
> > +			yypcb->pcb_cstate = (YYSTATE);
> > +			BEGIN(S5);
> > +		}
> > +
> >  <S0>^{RGX_INTERP} |
> >  <S2>^{RGX_INTERP} ;	/* discard any #! lines */
> >  
> > @@ -548,6 +554,9 @@ if (yypcb->pcb_token != 0) {
> >  <S1>.|\n	; /* discard */
> >  <S1><<EOF>>	yyerror("end-of-file encountered before matching */\n");
> >  
> > +<S5>\n		BEGIN(yypcb->pcb_cstate);
> > +<S5>.		; /* discard */
> > +
> >  <S2>{RGX_PSPEC}	{
> >  			/*
> >  			 * S2 has an ambiguity because RGX_PSPEC includes '*'
> > diff --git a/test/unittest/lexer/tst.boc-in-line-comment.d b/test/unittest/lexer/tst.boc-in-line-comment.d
> > new file mode 100644
> > index 00000000..61f5961f
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.boc-in-line-comment.d
> > @@ -0,0 +1,16 @@
> > +/*
> > + * Oracle Linux DTrace.
> > + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> > + * Licensed under the Universal Permissive License v 1.0 as shown at
> > + * http://oss.oracle.com/licenses/upl.
> > + */
> > +
> > +/*
> > + * ASSERTION: Line comments cannot contain begin-of-comment markers.
> > + */
> 
> this needs to be updated I think? i.e. Like C, line comments can contain
> begin-of-comment markers.
> 
> > +
> > +// Comment /*
> > +BEGIN
> > +{
> > +	exit(0);
> > +}
> > diff --git a/test/unittest/lexer/tst.boc-in-line-comment.r b/test/unittest/lexer/tst.boc-in-line-comment.r
> > new file mode 100644
> > index 00000000..d57f017a
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.boc-in-line-comment.r
> > @@ -0,0 +1,5 @@
> > +                   FUNCTION:NAME
> > +                          :BEGIN 
> > +
> > +-- @@stderr --
> > +dtrace: script 'test/unittest/lexer/tst.boc-in-line-comment.d' matched 1 probe
> > diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.d b/test/unittest/lexer/tst.eoc-in-line-comment.d
> > new file mode 100644
> > index 00000000..8fe2ff04
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.eoc-in-line-comment.d
> > @@ -0,0 +1,16 @@
> > +/*
> > + * Oracle Linux DTrace.
> > + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> > + * Licensed under the Universal Permissive License v 1.0 as shown at
> > + * http://oss.oracle.com/licenses/upl.
> > + */
> > +
> > +/*
> > + * ASSERTION: Line comments cannot contain end-of-comment markers.
> 
> same here.
> 
> > + */
> > +
> > +// Comment */
> > +BEGIN
> > +{
> > +	exit(0);
> > +}
> > diff --git a/test/unittest/lexer/tst.eoc-in-line-comment.r b/test/unittest/lexer/tst.eoc-in-line-comment.r
> > new file mode 100644
> > index 00000000..884cd554
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.eoc-in-line-comment.r
> > @@ -0,0 +1,5 @@
> > +                   FUNCTION:NAME
> > +                          :BEGIN 
> > +
> > +-- @@stderr --
> > +dtrace: script 'test/unittest/lexer/tst.eoc-in-line-comment.d' matched 1 probe
> > diff --git a/test/unittest/lexer/tst.eof-in-line-comment.d b/test/unittest/lexer/tst.eof-in-line-comment.d
> > new file mode 100644
> > index 00000000..a9207693
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.eof-in-line-comment.d
> > @@ -0,0 +1,16 @@
> > +/*
> > + * Oracle Linux DTrace.
> > + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> > + * Licensed under the Universal Permissive License v 1.0 as shown at
> > + * http://oss.oracle.com/licenses/upl.
> > + */
> > +
> > +/*
> > + * ASSERTION: End-of-file in a line comment is an error.
> 
> again this doesn't seem like an error test?
> 
> > + */
> > +
> > +BEGIN
> > +{
> > +	exit(0);
> > +}
> > +// Comment
> > \ No newline at end of file
> > diff --git a/test/unittest/lexer/tst.eof-in-line-comment.r b/test/unittest/lexer/tst.eof-in-line-comment.r
> > new file mode 100644
> > index 00000000..fdf259e0
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.eof-in-line-comment.r
> > @@ -0,0 +1,5 @@
> > +                   FUNCTION:NAME
> > +                          :BEGIN 
> > +
> > +-- @@stderr --
> > +dtrace: script 'test/unittest/lexer/tst.eof-in-line-comment.d' matched 1 probe
> > diff --git a/test/unittest/lexer/tst.lc-in-line-comment.d b/test/unittest/lexer/tst.lc-in-line-comment.d
> > new file mode 100644
> > index 00000000..0332e1ae
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.lc-in-line-comment.d
> > @@ -0,0 +1,16 @@
> > +/*
> > + * Oracle Linux DTrace.
> > + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> > + * Licensed under the Universal Permissive License v 1.0 as shown at
> > + * http://oss.oracle.com/licenses/upl.
> > + */
> > +
> > +/*
> > + * ASSERTION: Line comments cannot contain a line commens marker.
> 
> same here
> 
> > + */
> > +
> > +// Comment //
> > +BEGIN
> > +{
> > +	exit(0);
> > +}
> > diff --git a/test/unittest/lexer/tst.lc-in-line-comment.r b/test/unittest/lexer/tst.lc-in-line-comment.r
> > new file mode 100644
> > index 00000000..3075b987
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.lc-in-line-comment.r
> > @@ -0,0 +1,5 @@
> > +                   FUNCTION:NAME
> > +                          :BEGIN 
> > +
> > +-- @@stderr --
> > +dtrace: script 'test/unittest/lexer/tst.lc-in-line-comment.d' matched 1 probe
> > diff --git a/test/unittest/lexer/tst.line-comment.d b/test/unittest/lexer/tst.line-comment.d
> > new file mode 100644
> > index 00000000..4deb66f4
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.line-comment.d
> > @@ -0,0 +1,16 @@
> > +/*
> > + * Oracle Linux DTrace.
> > + * Copyright (c) 2025, Oracle and/or its affiliates. All rights reserved.
> > + * Licensed under the Universal Permissive License v 1.0 as shown at
> > + * http://oss.oracle.com/licenses/upl.
> > + */
> > +
> > +/*
> > + * ASSERTION: // comments are supported.
> > + */
> > +
> > +// exit(1);
> > +BEGIN // exit(1);
> > +{ // exit(1);
> > +	exit(0); // exit(1);
> > +} // exit(1);
> > diff --git a/test/unittest/lexer/tst.line-comment.r b/test/unittest/lexer/tst.line-comment.r
> > new file mode 100644
> > index 00000000..d4e4c325
> > --- /dev/null
> > +++ b/test/unittest/lexer/tst.line-comment.r
> > @@ -0,0 +1,5 @@
> > +                   FUNCTION:NAME
> > +                          :BEGIN 
> > +
> > +-- @@stderr --
> > +dtrace: script 'test/unittest/lexer/tst.line-comment.d' matched 1 probe

^ permalink raw reply	[flat|nested] 3+ messages in thread
end of thread, other threads:[~2025-07-23 12:57 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-23  5:11 [PATCH v2] lexer: support // comments Kris Van Hees
2025-07-23  9:45 ` Alan Maguire
2025-07-23 12:57   ` Kris Van Hees

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox