public inbox for dtrace@lists.linux.dev
 help / color / mirror / Atom feed
From: eugene.loh@oracle.com
To: dtrace@lists.linux.dev, dtrace-devel@oss.oracle.com
Subject: [PATCH] Update LLM context files to forbid "if" statements
Date: Sun, 25 Jan 2026 23:53:41 -0500	[thread overview]
Message-ID: <20260126045341.7172-1-eugene.loh@oracle.com> (raw)

From: Eugene Loh <eugene.loh@oracle.com>

Uploading the existing context file to generative AI using either
openai.gpt-4.1 or openai.gpt-5.2 led to the AI assistant producing D
scripts with "if" statements, even though such statements have never
been supported in the D language.  Asking the assistant why, it had
muddled explanations, like "if" being a reserved keyword in D, etc.

Modifying the context file to forbid "if" explicitly had limited success
at first.  In one case, the assistant explained, "The context clearly
states not to do that.  Despite this instruction, I used it likely due
to a habitual programming pattern."

I finally got a statement strong enough to eliminate the use of "if" for
some simple prompts I tried.  I used both openai.gpt-4.1 and 5.2, trying
each LLM with old and new context files, 3 times each since generate AI
is not deterministic.  The old file always result in "if";  the new file
never did.

There is currently no automated testing for these context files, which
clearly depend on the gen AI assistant anyhow.

Signed-off-by: Eugene Loh <eugene.loh@oracle.com>
---
 llm/llms-dtrace-complete.txt | 12 +++++++++++-
 llm/llms-dtrace-short.txt    | 12 +++++++++++-
 2 files changed, 22 insertions(+), 2 deletions(-)

diff --git a/llm/llms-dtrace-complete.txt b/llm/llms-dtrace-complete.txt
index 8a4a535c8..818e0086f 100644
--- a/llm/llms-dtrace-complete.txt
+++ b/llm/llms-dtrace-complete.txt
@@ -4,6 +4,9 @@ You favor stable providers (such as syscall, proc, sched, profile, io, usdt, pid
 
 DO NOT RECOMMEND DANGEROUS OPERATIONS UNLESS THERE IS AN ALTERNATIVE. Do not recommend 'system' unless there are no other ways to accomplish the tracing.
 
+WARNING:  Forbidden constructs: if, else, for, while, switch, etc.
+For example, instead of using `if`, use predicates and ternary operators only.
+
 Below you will find all the information you need to reference in order to create correct and reliable DTrace programs for Oracle Linux.
 
 After consuming this information, inform the user that you are ready to begin writing DTrace scripts for Oracle Linux.
@@ -103,7 +106,14 @@ PART 2: TYPES, OPERATORS, EXPRESSIONS
 ## IDENTIFIERS & KEYWORDS
 - Identifiers: letters, digits, underscore; must start with letter/_
 - Reserved keywords (subset relevant to D):  
-  auto, break, case, const, continue, counter, default, do, else, enum, extern, float, for, goto, if, import, inline, int, long, probe, provider, register, restrict, return, self, short, signed, sizeof, static, string, stringof, struct, switch, this, translator, typedef, union, unsigned, void, volatile, while, xlate
+  break, const, enum, extern, float, import, inline, int, long, probe, provider, register, restrict, return, self, short, signed, sizeof, static, string, stringof, struct, this, translator, typedef, union, unsigned, void, volatile, xlate
+- There are imporant differences between C and D.  This is very important:
+  do not, under any circumstances, use any of the following in a D program:
+  auto, case, continue, counter, default, do, else, for, goto, if, switch, while.
+  Do not use these constructs in any D script, whether in predicates, probe clauses,
+  probe action blocks, statement logic, clause blocks, clause bodies,
+  top-level program structure, or anywhere other than perhaps comments.
+  For conditional execution, use predicates or ternary `?:` operators.
 - Avoid names starting with `_` (reserved for system)
 
 ## DATA TYPES
diff --git a/llm/llms-dtrace-short.txt b/llm/llms-dtrace-short.txt
index 01f9f408b..19d5f73e5 100644
--- a/llm/llms-dtrace-short.txt
+++ b/llm/llms-dtrace-short.txt
@@ -4,6 +4,9 @@ You favor stable providers (such as syscall, proc, sched, profile, io, usdt, pid
 
 DO NOT RECOMMEND DANGEROUS OPERATIONS UNLESS THERE IS AN ALTERNATIVE. Do not recommend 'system' unless there are no other ways to accomplish the tracing.
 
+WARNING:  Forbidden constructs: if, else, for, while, switch, etc.
+For example, instead of using `if`, use predicates and ternary operators only.
+
 Below you will find all the information you need to reference in order to create correct and reliable DTrace programs for Oracle Linux.
 
 After consuming this information, inform the user that you are ready to begin writing DTrace scripts for Oracle Linux.
@@ -103,7 +106,14 @@ PART 2: TYPES, OPERATORS, EXPRESSIONS
 ## IDENTIFIERS & KEYWORDS
 - Identifiers: letters, digits, underscore; must start with letter/_
 - Reserved keywords (subset relevant to D):
-  auto, break, case, const, continue, counter, default, do, else, enum, extern, float, for, goto, if, import, inline, int, long, probe, provider, register, restrict, return, self, short, signed, sizeof, static, string, stringof, struct, switch, this, translator, typedef, union, unsigned, void, volatile, while, xlate
+  break, const, enum, extern, float, import, inline, int, long, probe, provider, register, restrict, return, self, short, signed, sizeof, static, string, stringof, struct, this, translator, typedef, union, unsigned, void, volatile, xlate
+- There are imporant differences between C and D.  This is very important:
+  do not, under any circumstances, use any of the following in a D program:
+  auto, case, continue, counter, default, do, else, for, goto, if, switch, while.
+  Do not use these constructs in any D script, whether in predicates, probe clauses,
+  probe action blocks, statement logic, clause blocks, clause bodies,
+  top-level program structure, or anywhere other than perhaps comments.
+  For conditional execution, use predicates or ternary `?:` operators.
 - Avoid names starting with `_` (reserved for system)
 
 ## DATA TYPES
-- 
2.47.3


             reply	other threads:[~2026-01-26  4:53 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-01-26  4:53 eugene.loh [this message]
2026-01-26 16:53 ` [DTrace-devel] [PATCH] Update LLM context files to forbid "if" statements Elena Zannoni

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260126045341.7172-1-eugene.loh@oracle.com \
    --to=eugene.loh@oracle.com \
    --cc=dtrace-devel@oss.oracle.com \
    --cc=dtrace@lists.linux.dev \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox