[Xen-devel] Ryzen 3xxx works with Windows

[Xen-devel] Ryzen 3xxx works with Windows

[Andreas Kinzler]

[-- Attachment #1: Type: text/plain, Size: 343 bytes --]

Hello All,

I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to 
3700X and found only very few differences. I added

cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]

to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score 
is >2050 which is more or less the bare metal value.

Regards Andreas

[-- Attachment #2: cpuid-2700X.tar.xz --]
[-- Type: application/octet-stream, Size: 6696 bytes --]

[-- Attachment #3: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[George Dunlap]

On 11/15/19 5:06 PM, Andreas Kinzler wrote:
> Hello All,
> 
> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
> 3700X and found only very few differences. I added
> 
> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
> 
> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
> is >2050 which is more or less the bare metal value.

Awesome.  Any idea what those bits do?

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Steven Haigh]

Can add weight to these findings. Tested with Xen 4.12.1 and the cpuid 
line suggested and it looks like my Windows VM has come up with 4 vCPUS.

I can't RDP in to make sure its 100% booted, but it certainly isn't 
doing the crash dump cycle - and CPU usage is consistent with being 
successfully booted.
Steven Haigh

📧 netwiz@crc.id.au     💻 https://www.crc.id.au
📞 +613 9001 6090       📱 +614 1293 5897


On Fri, Nov 15, 2019 at 18:06, Andreas Kinzler <hfp@posteo.de> wrote:
> Hello All,
> 
> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) 
> to 3700X and found only very few differences. I added
> 
> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
> 
> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 
> score is >2050 which is more or less the bare metal value.
> 
> Regards Andreas



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Jan Beulich]

On 15.11.2019 18:13, George Dunlap wrote:
> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>> Hello All,
>>
>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>> 3700X and found only very few differences. I added
>>
>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>
>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>> is >2050 which is more or less the bare metal value.
> 
> Awesome.  Any idea what those bits do?

That's the apic_id_size field (using the naming of
xen/include/xen/lib/x86/cpuid.h).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [SUSPECTED SPAM]Ryzen 3xxx works with Windows

[George Dunlap]

On 11/15/19 5:06 PM, Andreas Kinzler wrote:
> Hello All,
> 
> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
> 3700X and found only very few differences. I added
> 
> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
> 
> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
> is >2050 which is more or less the bare metal value.

So this is setting those bits to 0100, or 4.  What is it on the 3700X if
you don't change it here?

  -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [SUSPECTED SPAM]Ryzen 3xxx works with Windows

[Jan Beulich]

On 18.11.2019 12:39, George Dunlap wrote:
> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>> Hello All,
>>
>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>> 3700X and found only very few differences. I added
>>
>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>
>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>> is >2050 which is more or less the bare metal value.
> 
> So this is setting those bits to 0100, or 4.  What is it on the 3700X if
> you don't change it here?

According to the CPUID output Andreas had sent it's 4. The
current massaging in libxc (which your patch would conditionally
suppress) would double this to 8 afaict.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Jan Beulich]

On 18.11.2019 13:29, Jan Beulich wrote:
> On 18.11.2019 12:39, George Dunlap wrote:
>> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>>> Hello All,
>>>
>>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>>> 3700X and found only very few differences. I added
>>>
>>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>>
>>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>>> is >2050 which is more or less the bare metal value.
>>
>> So this is setting those bits to 0100, or 4.  What is it on the 3700X if
>> you don't change it here?
> 
> According to the CPUID output Andreas had sent it's 4. The
> current massaging in libxc (which your patch would conditionally
> suppress) would double this to 8 afaict.

Ouch, no - it would increment it (to 5), not double it.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Jan Beulich]

On 18.11.2019 12:39, George Dunlap wrote:
> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>> Hello All,
>>
>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>> 3700X and found only very few differences. I added
>>
>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>
>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>> is >2050 which is more or less the bare metal value.
> 
> So this is setting those bits to 0100, or 4.

One of the many possible problems with our incrementing of this field
could be that this results in a value that in "reserved" as per AMD's
documentation. At least for some of the Fam17 models they document
exactly which values are "legal".

This may then (further) get in the way of OSes trying to fully split
APIC IDs into the parts correlating with the topological sub-units
(thread, core, core complex, die, socket).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[George Dunlap]

On 11/18/19 12:54 PM, Jan Beulich wrote:
> On 18.11.2019 12:39, George Dunlap wrote:
>> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>>> Hello All,
>>>
>>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>>> 3700X and found only very few differences. I added
>>>
>>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>>
>>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>>> is >2050 which is more or less the bare metal value.
>>
>> So this is setting those bits to 0100, or 4.
> 
> One of the many possible problems with our incrementing of this field
> could be that this results in a value that in "reserved" as per AMD's
> documentation. At least for some of the Fam17 models they document
> exactly which values are "legal".

Do you have a reference for this?  I did manage to download one of the
AMD manuals and find the reference for ecx:80000008, but it didn't
mention any restrictions on legal values.

At any rate, this will affect the topology calculation for Linux guests
with more than 8 vcpus, so I don't think we can simply enable it across
the board, even on the affected chipsets.  If the xl.cfg cpuid override
perists across migrate, documenting it might be the best solution for now.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Jan Beulich]

On 18.11.2019 16:15, George Dunlap wrote:
> On 11/18/19 12:54 PM, Jan Beulich wrote:
>> On 18.11.2019 12:39, George Dunlap wrote:
>>> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>>>> Hello All,
>>>>
>>>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>>>> 3700X and found only very few differences. I added
>>>>
>>>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>>>
>>>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>>>> is >2050 which is more or less the bare metal value.
>>>
>>> So this is setting those bits to 0100, or 4.
>>
>> One of the many possible problems with our incrementing of this field
>> could be that this results in a value that in "reserved" as per AMD's
>> documentation. At least for some of the Fam17 models they document
>> exactly which values are "legal".
> 
> Do you have a reference for this?

Unfortunately none that I can share. But you can bet that OS vendors use
that rather than the public variant(s) for their development work.

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Andreas Kinzler]

On 15.11.2019 18:13, George Dunlap wrote:
> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>> Hello All,
>>
>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>> 3700X and found only very few differences. I added
>>
>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>
>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>> is >2050 which is more or less the bare metal value.
> Awesome.  Any idea what those bits do?

 From the AMD APM (https://www.amd.com/system/files/TechDocs/24594.pdf):

APIC ID size. The number of bits in the initial APIC20[ApicId] value 
that indicate core ID within a processor. A zero value indicates that 
legacy methods must be used to derive the maximum number of cores. The 
size of this field determines the maximum number of cores (MNC) that the 
processor could theoretically support, not the actual number of cores 
that are actually implemented or enabled on the processor, as indicated 
by CPUID Fn8000_0008_ECX[NC].
if (ApicIdCoreIdSize[3:0] == 0){
   // Used by legacy dual-core/single-core processors
   MNC = CPUID Fn8000_0008_ECX[NC] + 1;
} else {
   // use ApicIdCoreIdSize[3:0] field
   MNC = (2 ^ ApicIdCoreIdSize[3:0]);
}

The value programmed in 2700X is 4, on 3700X it is 7. See my dump in 
https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02189.html

Please note that the value is an exponent - that means MNC is programmed 
as 16 for 2700X and 128 for 3700X.

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[George Dunlap]

On 11/18/19 4:11 PM, Andreas Kinzler wrote:
> On 15.11.2019 18:13, George Dunlap wrote:
>> On 11/15/19 5:06 PM, Andreas Kinzler wrote:
>>> Hello All,
>>>
>>> I compared the CPUID listings from Ryzen 2700X (attached as tar.xz) to
>>> 3700X and found only very few differences. I added
>>>
>>> cpuid = [ "0x80000008:ecx=xxxxxxxxxxxxxxxx0100xxxxxxxxxxxx" ]
>>>
>>> to xl.cfg and then Windows runs great with 16 vCPUs. Cinebench R15 score
>>> is >2050 which is more or less the bare metal value.
>> Awesome.  Any idea what those bits do?
> 
> From the AMD APM (https://www.amd.com/system/files/TechDocs/24594.pdf):
> 
> APIC ID size. The number of bits in the initial APIC20[ApicId] value
> that indicate core ID within a processor. A zero value indicates that
> legacy methods must be used to derive the maximum number of cores. The
> size of this field determines the maximum number of cores (MNC) that the
> processor could theoretically support, not the actual number of cores
> that are actually implemented or enabled on the processor, as indicated
> by CPUID Fn8000_0008_ECX[NC].
> if (ApicIdCoreIdSize[3:0] == 0){
>   // Used by legacy dual-core/single-core processors
>   MNC = CPUID Fn8000_0008_ECX[NC] + 1;
> } else {
>   // use ApicIdCoreIdSize[3:0] field
>   MNC = (2 ^ ApicIdCoreIdSize[3:0]);
> }
> 
> The value programmed in 2700X is 4, on 3700X it is 7. See my dump in
> https://lists.xenproject.org/archives/html/xen-devel/2019-09/msg02189.html

Where were these values collected -- on a PV dom0?  Or from within the
guest?

As Jan pointed out, what actually happens here is that the domain
builder actually bumps this number by 1 for HVM guests by default.  So
if the real value is 4, then the guest will see 5; if the real value is
7, the guest will see 8.

Andy says that the modifications from the manual cpuid settings happen
*after* this adjustment; so that by setting it to 0100, you change it to 4.

Could you try this with `0111` instead?  That should set it to its
actual hardware value (although the other "adjustments" won't be modified).

Given the current practical limitations with numbers of vcpus, I'd be
tempted to say we could always just limit this value to 7.  If after a
migration, a Linux guest sees this move from 8 to 7, I suspect it will
cope just fine.

 -George

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] Ryzen 3xxx works with Windows

[Andreas Kinzler]

On 18.11.2019 17:25, George Dunlap wrote:
> Where were these values collected -- on a PV dom0?  Or from within the
> guest?

Neither. Bare metal kernel - no Xen at all.

> Could you try this with `0111` instead?

Works. '1000' crashes again. Now it is clear that 7 is the maximum 
Windows accepts.

Regards Andreas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel