[ANNOUNCE] x86-cpuid-db: Release v2.0

[ANNOUNCE] x86-cpuid-db: Release v2.0

[Ahmed S. Darwish]

Hi,

I'm happy to announce release 2.0 of the x86-cpuid-db project:

  https://gitlab.com/x86-cpuid.org/x86-cpuid-db
  https://gitlab.com/x86-cpuid.org/x86-cpuid-db/-/releases/v2.0

Changelog v2.0
--------------

- Introduce a new transformer, which generates one Linux kernel C header
  file for all the cpuid leaves' bitfields.

  (A single linux kernel C header file, with version tag on top, shall be
   easier to maintain than multiple headers; Linux x86 PQ pending)

- Introduce the cpuidgen --kheaders option to invoke that transformer.

- Extend the cpuid database bitfields coverage:

  - Add Transmeta vendor tags to the appropriate bitfields at leaves
    0x0, 0x01, and from leaves 0x80000000 to 0x80000006.

  - Add the Transmeta-specific CPUID leaves 0x03, and 0x80860000 to
    0x80860007.

  - Add Centaur/Zhaoxin leaves 0xc0000000 and 0xc0000001, along with
    Zhoaxin's exclusive feature bits.

- Add some documentation in the schema for the <linux> annotation tag.

CPUID bitfields coverage
~~~~~~~~~~~~~~~~~~~~~~~~

In total, this release increases the x86-cpuid database bitfield
coverage to:

    CPUID leaves:        63 leaves
    CPUID bitfields:     907 entries

Changes to generated files styling
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Beside the new cpuid leaves and bitfields, the generated Linux kernel C
structures has changed from:

    /*
     * CPUID leaf 0
     */

    struct leaf0_sl0 {
        ...;
    };

    /*
     * CPUID leaf 7
     */

    struct leaf7_sl0 {
        ...;
    };

    struct leaf7_sl1 {
        ...;
    };

to:

    /*
     * Leaf 0x0
     * Maximum standard leaf number + CPU vendor string
     */

    struct leaf_0x0_0 {
        ...;
    };

    /*
     * Leaf 0x7
     * Extended CPU features enumeration
     */

    struct leaf_0x7_0 {
        ...;
    };

    struct leaf_0x7_1 {
        ...;
    };

Overall, this removes ambiguity for IDs like 0x16, and improves kernel
code greppability (x86 PQ pending).

Finally, the generated CSV and C header files can be quickly checked in
full from the release artifacts at:

  https://gitlab.com/x86-cpuid.org/x86-cpuid-db/-/releases/v2.0

Thanks,

--
Ahmed S. Darwish
Linutronix GmbH

Re: [ANNOUNCE] x86-cpuid-db: Release v2.0

[Andrew Cooper]

On 08/10/2024 2:31 pm, Ahmed S. Darwish wrote:
> to:
>
>     /*
>      * Leaf 0x0
>      * Maximum standard leaf number + CPU vendor string
>      */
>
>     struct leaf_0x0_0 {
>         ...;
>     };
>
>     /*
>      * Leaf 0x7
>      * Extended CPU features enumeration
>      */
>
>     struct leaf_0x7_0 {
>         ...;
>     };
>
>     struct leaf_0x7_1 {
>         ...;
>     };
>
> Overall, this removes ambiguity for IDs like 0x16, and improves kernel
> code greppability (x86 PQ pending).

So what does leaf 0xd with 62 subleaves look like.  Do we really have
one number in hex, and one in decimal?

~Andrew

Re: [ANNOUNCE] x86-cpuid-db: Release v2.0

[Ahmed S. Darwish]

Hi Andrew,

On Tue, 08 Oct 2024, Andrew Cooper wrote:
>
> On 08/10/2024 2:31 pm, Ahmed S. Darwish wrote:
> > to:
> >
> >     /*
> >      * Leaf 0x0
> >      * Maximum standard leaf number + CPU vendor string
> >      */
> >
> >     struct leaf_0x0_0 {
> >         ...;
> >     };
> >
> >     /*
> >      * Leaf 0x7
> >      * Extended CPU features enumeration
> >      */
> >
> >     struct leaf_0x7_0 {
> >         ...;
> >     };
> >
> >     struct leaf_0x7_1 {
> >         ...;
> >     };
> >
> > Overall, this removes ambiguity for IDs like 0x16, and improves kernel
> > code greppability (x86 PQ pending).
>
> So what does leaf 0xd with 62 subleaves look like.  Do we really have
> one number in hex, and one in decimal?

Hmmm, good point...

For now, if we grep the generated C linux header for all the structures
that are not subleaf 0, we get:

    $ grep -E 'struct leaf_0x[0-9a-f]+_[^0]' cpuid-bitfields.h

    struct leaf_0x7_1 {
    struct leaf_0x7_2 {
    struct leaf_0xd_1 {
    struct leaf_0xd_2 {
    struct leaf_0xf_1 {
    struct leaf_0x10_1 {
    struct leaf_0x10_3 {
    struct leaf_0x12_1 {
    struct leaf_0x12_2 {
    struct leaf_0x14_1 {
    struct leaf_0x17_1 {
    struct leaf_0x1d_1 {
    struct leaf_0x23_1 {
    struct leaf_0x23_3 {
    struct leaf_0x80000020_1 {
    struct leaf_0x80000020_2 {
    struct leaf_0x80000020_3 {

We do indeed have an encodings for leaves with large dynamic number of
valid subleaves, through "array=":

    darwi@lx ~/x86-cpuid-db (main)> git grep array=

    db/xml/leaf_04.xml:  <subleaf id="0" array="32">
    db/xml/leaf_0b.xml:  <subleaf id="0" array="2">
    db/xml/leaf_0d.xml:  <subleaf id="2" array="62">
    db/xml/leaf_10.xml:  <subleaf id="1" array="2">
    db/xml/leaf_12.xml:  <subleaf id="2" array="30">
    db/xml/leaf_17.xml:  <subleaf id="1" array="3">
    db/xml/leaf_18.xml:  <subleaf id="0" array="32">
    db/xml/leaf_1b.xml:  <subleaf id="0" array="32">
    db/xml/leaf_1f.xml:  <subleaf id="0" array="6">
    db/xml/leaf_8000001d.xml:  <subleaf id="0" array="32">
    db/xml/leaf_80000026.xml:  <subleaf id="0" array="4">

But the generators don't deal with that (yet), and that's why they're
also not in the generated CSV/header files.

I'm going through the Linux x86 PQ now, so it's a nice opportunity to
see how something like "struct leaf_0x0_0x0" looks within the rest of
the kernel code...  I'll also ping Thomas to see if he has any feedback
on the topic.

Thanks!

--
Ahmed S. Darwish
Linutronix GmbH