[PATCH] db: Add Intel NMI-source reporting feature flag

[PATCH] db: Add Intel NMI-source reporting feature flag

[Sohil Mehta]

NMI-source is documented starting with FRED specification v6. It is
enumerated with the feature flag:
CPUID.(EAX=7,ECX=1):EAX.NMI_SRC[bit20]

Link: https://www.intel.com/content/www/us/en/content-details/819481/flexible-return-and-event-delivery-fred-specification.html

Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
---
 db/xml/leaf_07.xml | 1 +
 1 file changed, 1 insertion(+)

diff --git a/db/xml/leaf_07.xml b/db/xml/leaf_07.xml
index 415524e..5d15015 100644
--- a/db/xml/leaf_07.xml
+++ b/db/xml/leaf_07.xml
@@ -566,6 +566,7 @@
       <bit19 len="1"  id="wrmsrns"                 desc="WRMSRNS instr (WRMSR-non-serializing)">
         <xen          feature="true"               attrs="S" />
       </bit19>
+      <bit20 len="1"  id="nmi_src"                 desc="NMI-source reporting with FRED event data" />
       <bit21 len="1"  id="amx_fp16"                desc="AMX-FP16: FP16 tile operations">
         <linux        feature="true"               proc="false" />
       </bit21>
-- 
2.34.1

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Sohil Mehta]

Hi Ahmed,

On 9/23/2024 3:32 PM, Sohil Mehta wrote:
> +      <bit20 len="1"  id="nmi_src"                 desc="NMI-source reporting with FRED event data" />

I am starting to work on a new x86 feature and submitting this in
preparation for that.

>        <bit21 len="1"  id="amx_fp16"                desc="AMX-FP16: FP16 tile operations">
>          <linux        feature="true"               proc="false" />

I couldn't find much documentation for the <linux> tag. When is it
expected to be included?

Also, are the following linux hint attributes documented somewhere? I
couldn't decipher them from their name itself.

>   <xs:complexType name="linuxType">
>     <xs:attribute type="xs:boolean"      name="feature"     use="required" />
>     <xs:attribute type="xs:boolean"      name="proc"        use="required" />
>     <xs:attribute type="xs:string"       name="altid"       use="optional" />
>     <xs:attribute type="xs:string"       name="procid"      use="optional" />
>   </xs:complexType>
> 

Does feature="true" mean the feature is intended for linux or it is
merged into the linux kernel?

Thanks,
Sohil

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Ahmed S. Darwish]

Hi Sohil,

On Mon, 23 Sep 2024, Sohil Mehta wrote:
>
> >  <bit21 len="1"  id="amx_fp16"   desc="AMX-FP16: FP16 tile operations">
> >    <linux        feature="true"  proc="false" />
>
> I couldn't find much documentation for the <linux> tag. When is it
> expected to be included?
>
> Also, are the following linux hint attributes documented somewhere? I
> couldn't decipher them from their name itself.
>
> >   <xs:complexType name="linuxType">
> >     <xs:attribute type="xs:boolean" name="feature" use="required" />
> >     <xs:attribute type="xs:boolean" name="proc"    use="required" />
> >     <xs:attribute type="xs:string"  name="altid"   use="optional" />
> >     <xs:attribute type="xs:string"  name="procid"  use="optional" />
> >   </xs:complexType>
> >

Yeah, I should've documented the schema more, either in-place or in a
separate file.

The <linux> tag means that this bit is also represented inside the Linux
Kernel x86 tree in various means.

Here are two nice examples from leaf_01.xml:

  <bit2  len="1" id="dtes64"    desc="64-bit DS save area">
    <vendors>
      <vendor>Intel</vendor>
    </vendors>
    <linux	 feature="true"  proc="true" />
    <xen         feature="true"  attrs="" />
  </bit2>
  ...
  <bit0  len="1" id="sse3"      desc="Streaming SIMD Extensions 3 (SSE3)">
    <vendors>
      <vendor>Intel</vendor>
      <vendor>AMD</vendor>
    </vendors>
    <linux       feature="true"	proc="true" altid="xmm3" procid="pni" />
    <xen         feature="true"	attrs="A" />
  </bit0>

<linux> "feature=true" attribute means that the feature bit is also an
X86_FEATURE_* flag.  So, in the "dtes64" case, there's indeed an
X86_FEATURE_DTES64 defined flag in the Linux kernel x86 tree.

<linux> "feature=true altid=NAME" means there's an X86_FEATURE_* flag,
but the name is slightly different from the database attribute id name
(for various reasons).  Thus, in the SSE3 feature flag case, the Linux
feature flag is actually called X86_FEATURE_XMM3 (altid="xmm3".)

<linux> "proc=true" means that the feature bit is also displayed at
Linux's /proc/cpuinfo if available on the CPU.  So, in the "dtes64"
case, you'll indeed see dtes64 flag at /proc/cpuinfo on most Intel CPUs.

<linux> "proc=true procid=PROCNAME' means that the feature bit is
displayed at /proc/cpuinfo, but the flag name there is different from
the database attribute id name (for various reasons.)  Thus, in the
"sse3" case, "procid=pni" because that's how Linux displays SSE3 feature
flag at /proc/cpuinfo.

So far, that <linux> tag is not used by any of the XSLT transformers.
But, we plan to use this information as we go deeper into the Linux x86
tree refactoring(s).  For now, the goal was to cover all known names for
the cpuid feature bits, especially when they differ; i.e. canonical name
vs. Linux X86_FEATURE name vs. Linux /proc/cpuinfo name.

Thanks for the question! I'll indeed need to document that part of the
schema before the v2.0 release.

All the best,

--
Ahmed S. Darwish
Linutronix GmbH

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Andrew Cooper]

On 26/09/2024 2:12 pm, Ahmed S. Darwish wrote:
> Hi Sohil,
>
> On Mon, 23 Sep 2024, Sohil Mehta wrote:
>>>  <bit21 len="1"  id="amx_fp16"   desc="AMX-FP16: FP16 tile operations">
>>>    <linux        feature="true"  proc="false" />
>> I couldn't find much documentation for the <linux> tag. When is it
>> expected to be included?
>>
>> Also, are the following linux hint attributes documented somewhere? I
>> couldn't decipher them from their name itself.
>>
>>>   <xs:complexType name="linuxType">
>>>     <xs:attribute type="xs:boolean" name="feature" use="required" />
>>>     <xs:attribute type="xs:boolean" name="proc"    use="required" />
>>>     <xs:attribute type="xs:string"  name="altid"   use="optional" />
>>>     <xs:attribute type="xs:string"  name="procid"  use="optional" />
>>>   </xs:complexType>
>>>
> Yeah, I should've documented the schema more, either in-place or in a
> separate file.
>
> The <linux> tag means that this bit is also represented inside the Linux
> Kernel x86 tree in various means.
>
> Here are two nice examples from leaf_01.xml:
>
>   <bit2  len="1" id="dtes64"    desc="64-bit DS save area">
>     <vendors>
>       <vendor>Intel</vendor>
>     </vendors>
>     <linux	 feature="true"  proc="true" />
>     <xen         feature="true"  attrs="" />
>   </bit2>
>   ...
>   <bit0  len="1" id="sse3"      desc="Streaming SIMD Extensions 3 (SSE3)">
>     <vendors>
>       <vendor>Intel</vendor>
>       <vendor>AMD</vendor>
>     </vendors>
>     <linux       feature="true"	proc="true" altid="xmm3" procid="pni" />
>     <xen         feature="true"	attrs="A" />
>   </bit0>
>
> <linux> "feature=true" attribute means that the feature bit is also an
> X86_FEATURE_* flag.  So, in the "dtes64" case, there's indeed an
> X86_FEATURE_DTES64 defined flag in the Linux kernel x86 tree.
>
> <linux> "feature=true altid=NAME" means there's an X86_FEATURE_* flag,
> but the name is slightly different from the database attribute id name
> (for various reasons).  Thus, in the SSE3 feature flag case, the Linux
> feature flag is actually called X86_FEATURE_XMM3 (altid="xmm3".)
>
> <linux> "proc=true" means that the feature bit is also displayed at
> Linux's /proc/cpuinfo if available on the CPU.  So, in the "dtes64"
> case, you'll indeed see dtes64 flag at /proc/cpuinfo on most Intel CPUs.
>
> <linux> "proc=true procid=PROCNAME' means that the feature bit is
> displayed at /proc/cpuinfo, but the flag name there is different from
> the database attribute id name (for various reasons.)  Thus, in the
> "sse3" case, "procid=pni" because that's how Linux displays SSE3 feature
> flag at /proc/cpuinfo.
>
> So far, that <linux> tag is not used by any of the XSLT transformers.
> But, we plan to use this information as we go deeper into the Linux x86
> tree refactoring(s).  For now, the goal was to cover all known names for
> the cpuid feature bits, especially when they differ; i.e. canonical name
> vs. Linux X86_FEATURE name vs. Linux /proc/cpuinfo name.
>
> Thanks for the question! I'll indeed need to document that part of the
> schema before the v2.0 release.

I guess I need to start getting back to this...

The Xen 'attrs' need stripping. More specifically, they need to live in
Xen itself, because they are different by release, and are used in the
derivation of what is safe to offer to guests.

But the same is true for 'feature' too.  It's one think to know what the
bit means, but it's something else to be making a statement about what
is supported in an external project.

When this is used in earnest, regenerating the headers will be needed in
each stable tree to get e.g. the new speculation (anti)features, and
doing so needs to not change the other features="true" for bits only
supported in newer branches.

~Andrew

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Ahmed S. Darwish]

On Thu, 26 Sep 2024, Andrew Cooper wrote:
>
> I guess I need to start getting back to this...
>
> The Xen 'attrs' need stripping. More specifically, they need to live in
> Xen itself, because they are different by release, and are used in the
> derivation of what is safe to offer to guests.
>
> But the same is true for 'feature' too.  It's one think to know what the
> bit means, but it's something else to be making a statement about what
> is supported in an external project.
>
> When this is used in earnest, regenerating the headers will be needed in
> each stable tree to get e.g. the new speculation (anti)features, and
> doing so needs to not change the other features="true" for bits only
> supported in newer branches.
>

Yeah, correct.

I still remember our discussion on how the <xen> tag (and its
attributes) were not fitting Xen's model.  So, for now, it's indeed a
placeholder until work starts earnestly on that part.

Thanks,
Ahmed

--
Linutronix GmbH

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Sohil Mehta]

On 9/26/2024 6:12 AM, Ahmed S. Darwish wrote:

> 
> The <linux> tag means that this bit is also represented inside the Linux
> Kernel x86 tree in various means.
> 

Got it. But this would mean that the entry needs to be submitted after
the linux usage is established.

> For now, the goal was to cover all known names for
> the cpuid feature bits, especially when they differ; i.e. canonical name
> vs. Linux X86_FEATURE name vs. Linux /proc/cpuinfo name.
> 

Also, the various cpuid bit usages in the kernel might happen in stages.

Are you planning to have scripts that automatically pull this data from
the kernel as and when these usages get merged? I believe asking
developers to submit multiple edits to the cpuid-db might be
inconvenient and error prone.

> Thanks for the question! I'll indeed need to document that part of the
> schema before the v2.0 release.
> 

Thanks! That would be very useful as more folks start contributing to
the project.

-Sohil

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[H. Peter Anvin]

On September 27, 2024 10:47:56 AM PDT, Sohil Mehta <sohil.mehta@intel.com> wrote:
>On 9/26/2024 6:12 AM, Ahmed S. Darwish wrote:
>
>> 
>> The <linux> tag means that this bit is also represented inside the Linux
>> Kernel x86 tree in various means.
>> 
>
>Got it. But this would mean that the entry needs to be submitted after
>the linux usage is established.
>
>> For now, the goal was to cover all known names for
>> the cpuid feature bits, especially when they differ; i.e. canonical name
>> vs. Linux X86_FEATURE name vs. Linux /proc/cpuinfo name.
>> 
>
>Also, the various cpuid bit usages in the kernel might happen in stages.
>
>Are you planning to have scripts that automatically pull this data from
>the kernel as and when these usages get merged? I believe asking
>developers to submit multiple edits to the cpuid-db might be
>inconvenient and error prone.
>
>> Thanks for the question! I'll indeed need to document that part of the
>> schema before the v2.0 release.
>> 
>
>Thanks! That would be very useful as more folks start contributing to
>the project.
>
>-Sohil
>
>

Should we include a copy of this database in the kernel sources?

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Sohil Mehta]

> 
> Should we include a copy of this database in the kernel sources?

I believe this series does that:
https://lore.kernel.org/x86-cpuid/20240718134755.378115-1-darwi@linutronix.de/

It updates the kcpuid tool and the CSV file based on the new database.

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Ahmed S. Darwish]

Hi hpa,

On Fri, 27 Sep 2024, H. Peter Anvin wrote:
>
> Should we include a copy of this database in the kernel sources?
>

We expclitily didn't want it to make it a linux kernel build dependency.
Mainly out of a desire to use the database for other projects beside the
kernel as well, starting with Xen.

So, the tool have multiple transformers which generates "listings".  One
of these listings is now merged:

    https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/tools/arch/x86/kcpuid/cpuid.csv

The tool also generates Linux kernel C data structures:

    https://gitlab.com/x86-cpuid.org/x86-cpuid-db/-/jobs/7907817537/artifacts/download

We're working on a patch series for the x86 subsystem, where the above
data structures will be integrated in the kernel -- along with cpuid
accessors and a data model.

All geneated files also has tags on top that will allow KernelCI to
verify the integrity of these CSV or C header-file listings:

   "Tagging Conventions for Generated Files"
   https://gitlab.com/x86-cpuid.org/x86-cpuid-db/-/blob/tip/README.rst

(I was going to show this and other things at Plumbers x86 track demo,
 but sadly, BBB screen sharing misbehaved during the talk...  Next time
 I'll have a proper fallback mechanism.)

Thanks,

--
Ahmed S. Darwish
Linutronix GmbH

Re: [PATCH] db: Add Intel NMI-source reporting feature flag

[Ahmed S. Darwish]

On Mon, 23 Sep 2024, Sohil Mehta wrote:
>
> NMI-source is documented starting with FRED specification v6. It is
> enumerated with the feature flag:
> CPUID.(EAX=7,ECX=1):EAX.NMI_SRC[bit20]
>
> Link: https://www.intel.com/content/www/us/en/content-details/819481/flexible-return-and-event-delivery-fred-specification.html
>
> Signed-off-by: Sohil Mehta <sohil.mehta@intel.com>
> ---

Thanks! Now merged to tip:

    https://gitlab.com/x86-cpuid.org/x86-cpuid-db/-/commit/e35608e1f58e

All the best,
Ahmed