Detecting whether dom0 is in a VM

Detecting whether dom0 is in a VM

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 540 bytes --]

Hey all,

The following systemd issue was brought to my attention:

https://github.com/systemd/systemd/issues/28113

I think basically, they want `systemd-detect-virt` return the following
values:

Xen on hardware, from a dom0:  `none`
Xen on hardware, from a domU: `xen`
Xen in a VM, from a dom0: (ideally the virtualization type, or `vm-other`
if not)
Xen in a VM, from a domU: `xen`

Is there a reliable set of tests which would work across all dom0 guest
types / architectures?  If not, can we expose the information somehow?

 -George

[-- Attachment #2: Type: text/html, Size: 747 bytes --]

Re: Detecting whether dom0 is in a VM

[zithro]

On 05 Jul 2023 17:51, George Dunlap wrote:
> Hey all,
> 
> The following systemd issue was brought to my attention:
> 
> https://github.com/systemd/systemd/issues/28113
> 
> I think basically, they want `systemd-detect-virt` return the following
> values:
> 
> Xen on hardware, from a dom0:  `none`
> Xen on hardware, from a domU: `xen`
> Xen in a VM, from a dom0: (ideally the virtualization type, or `vm-other`
> if not)
> Xen in a VM, from a domU: `xen`
> 
> Is there a reliable set of tests which would work across all dom0 guest
> types / architectures?  If not, can we expose the information somehow?
> 
>   -George
> 

Small follow-up, I did some more tests (AMD platforms).
systemd-detect-virt (sdv) is using "/sys/class/dmi/id/sys_vendor".

On both "baremetal" dom0s, sdv is reporting the platform manufacturer 
("MSI" or "Micro-Star International Co., Ltd." on my systems).

On a nested dom0, sdv is reporting "xen" :
   root@xen-nested:~# SYSTEMD_LOG_LEVEL=debug systemd-detect-virt
   Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
   Found container virtualization none.
   Virtualization Xen found in DMI (/sys/class/dmi/id/sys_vendor)
   Found VM virtualization xen
   xen

PS: my host "xen-nested" is not masking CPUID leaves in cfg file.

So I'm wondering, isn't that path enough for correct detection ?
I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any 
other known hypervisor), it's nested, otherwise it's on hardware ?

Is that really mandatory to use CPUID leaves ?

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 05.07.2023 18:20, zithro wrote:
> On 05 Jul 2023 17:51, George Dunlap wrote:
>> Hey all,
>>
>> The following systemd issue was brought to my attention:
>>
>> https://github.com/systemd/systemd/issues/28113
>>
>> I think basically, they want `systemd-detect-virt` return the following
>> values:
>>
>> Xen on hardware, from a dom0:  `none`
>> Xen on hardware, from a domU: `xen`
>> Xen in a VM, from a dom0: (ideally the virtualization type, or `vm-other`
>> if not)
>> Xen in a VM, from a domU: `xen`
>>
>> Is there a reliable set of tests which would work across all dom0 guest
>> types / architectures?  If not, can we expose the information somehow?
>>
>>   -George
>>
> 
> Small follow-up, I did some more tests (AMD platforms).
> systemd-detect-virt (sdv) is using "/sys/class/dmi/id/sys_vendor".
> 
> On both "baremetal" dom0s, sdv is reporting the platform manufacturer 
> ("MSI" or "Micro-Star International Co., Ltd." on my systems).
> 
> On a nested dom0, sdv is reporting "xen" :
>    root@xen-nested:~# SYSTEMD_LOG_LEVEL=debug systemd-detect-virt
>    Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy
>    Found container virtualization none.
>    Virtualization Xen found in DMI (/sys/class/dmi/id/sys_vendor)
>    Found VM virtualization xen
>    xen
> 
> PS: my host "xen-nested" is not masking CPUID leaves in cfg file.
> 
> So I'm wondering, isn't that path enough for correct detection ?
> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any 
> other known hypervisor), it's nested, otherwise it's on hardware ?
> 
> Is that really mandatory to use CPUID leaves ?

Let me ask the other way around: In user mode code under a non-nested
vs nested Xen, what would you be able to derive from CPUID? The
"hypervisor" bit is going to be set in both cases. (All assuming you
run on new enough hardware+Xen such that CPUID would be intercepted
even for PV.)

Yet relying on DMI is fragile, too: Along the lines of
https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
basically any value in there could be "inherited" from the host (i.e.
from the layer below, to be precise). The only way to be reasonably
certain is to ask Xen about its view. The raw or host featuresets
should give you this information, in the "mirror" of said respective
CPUID leave's "hypervisor" bit. But of course that still won't tell
you which _kind_ of hypervisor is the immediate next one underneath
Xen.

This then further raises the question of what use it is to know the
kind of the next level hypervisor, when multiple may be stacked on
top of one another ...

Jan

Re: Detecting whether dom0 is in a VM

[zithro]

On 06 Jul 2023 09:02, Jan Beulich wrote:
> On 05.07.2023 18:20, zithro wrote:
>> So I'm wondering, isn't that path enough for correct detection ?
>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
>> other known hypervisor), it's nested, otherwise it's on hardware ?
>>
>> Is that really mandatory to use CPUID leaves ?
> 
> Let me ask the other way around: In user mode code under a non-nested
> vs nested Xen, what would you be able to derive from CPUID? The
> "hypervisor" bit is going to be set in both cases. (All assuming you
> run on new enough hardware+Xen such that CPUID would be intercepted
> even for PV.)

I'm a bit clueless about CPUID stuff, but if I understand correctly, 
you're essentially saying that using CPUID may not be the perfect way ?
Also, I don't get why the cpuid command returns two different values, 
depending on the -k switch :
# cpuid -l 0x40000000
hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
# cpuid -k -l 0x40000000
hypervisor_id (0x40000000) = "XenVMMXenVMM"

> Yet relying on DMI is fragile, too: Along the lines of
> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
> basically any value in there could be "inherited" from the host (i.e.
> from the layer below, to be precise).

So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep 
DMI:" is also not perfect, as values can be inherited/spoofed by 
underneath hypervisor ?

> The only way to be reasonably
> certain is to ask Xen about its view. The raw or host featuresets
> should give you this information, in the "mirror" of said respective
> CPUID leave's "hypervisor" bit. 

As said above, I'm clueless, can you expand please ?

> But of course that still won't tell
> you which _kind_ of hypervisor is the immediate next one underneath
> Xen.
> 
> This then further raises the question of what use it is to know the
> kind of the next level hypervisor, when multiple may be stacked on
> top of one another ...

We need an answer from systemd guys, but the commiter expanded on the 
reasons why the change was made [1] : "the detect_vm_cpuid check was 
returning a VIRTUALIZATION_NONE result on non-nested dom0 (checking the 
log from back then I was getting No virtualization found in CPUID), but 
would report other CPUID-detectable hypervisors when dom0 was nested, so 
we still wanted to check it for this case".

Systemd is using this information to not start some services when 
nested, like SMART/smartmontools (which have 
"ConditionVirtualization=no" in their systemd unit files).
As the check now fails on baremetal dom0s, the service is not starting.

[1] https://github.com/systemd/systemd/issues/28113#issuecomment-1621559642

Re: Detecting whether dom0 is in a VM

[Yann Dirson]

On 7/6/23 17:35, zithro wrote:
> On 06 Jul 2023 09:02, Jan Beulich wrote:
>> On 05.07.2023 18:20, zithro wrote:
>>> So I'm wondering, isn't that path enough for correct detection ?
>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
>>> other known hypervisor), it's nested, otherwise it's on hardware ?
>>>
>>> Is that really mandatory to use CPUID leaves ?
>>
>> Let me ask the other way around: In user mode code under a non-nested
>> vs nested Xen, what would you be able to derive from CPUID? The
>> "hypervisor" bit is going to be set in both cases. (All assuming you
>> run on new enough hardware+Xen such that CPUID would be intercepted
>> even for PV.)
> 
> I'm a bit clueless about CPUID stuff, but if I understand correctly, 
> you're essentially saying that using CPUID may not be the perfect way ?
> Also, I don't get why the cpuid command returns two different values, 
> depending on the -k switch :
> # cpuid -l 0x40000000
> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
> # cpuid -k -l 0x40000000
> hypervisor_id (0x40000000) = "XenVMMXenVMM"
> 
>> Yet relying on DMI is fragile, too: Along the lines of
>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
>> basically any value in there could be "inherited" from the host (i.e.
>> from the layer below, to be precise).
> 
> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep 
> DMI:" is also not perfect, as values can be inherited/spoofed by 
> underneath hypervisor ?
> 
>> The only way to be reasonably
>> certain is to ask Xen about its view. The raw or host featuresets
>> should give you this information, in the "mirror" of said respective
>> CPUID leave's "hypervisor" bit. 
> 
> As said above, I'm clueless, can you expand please ?
> 
>> But of course that still won't tell
>> you which _kind_ of hypervisor is the immediate next one underneath
>> Xen.
>>
>> This then further raises the question of what use it is to know the
>> kind of the next level hypervisor, when multiple may be stacked on
>> top of one another ...
> 
> We need an answer from systemd guys, but the commiter expanded on the 
> reasons why the change was made [1] : "the detect_vm_cpuid check was 
> returning a VIRTUALIZATION_NONE result on non-nested dom0 (checking the 
> log from back then I was getting No virtualization found in CPUID), but 
> would report other CPUID-detectable hypervisors when dom0 was nested, so 
> we still wanted to check it for this case".
> 
> Systemd is using this information to not start some services when 
> nested, like SMART/smartmontools (which have 
> "ConditionVirtualization=no" in their systemd unit files).
> As the check now fails on baremetal dom0s, the service is not starting.

Is this really wise?  With any passed-through device the user likely 
wants such a service to start.  Are they not misled with this effort?

> 
> [1] https://github.com/systemd/systemd/issues/28113#issuecomment-1621559642
> 

-- 
Yann Dirson | Vates Platform Developer
XCP-ng & Xen Orchestra - Vates solutions
w: vates.tech | xcp-ng.org | xen-orchestra.com


Yann Dirson | Vates Platform Developer

XCP-ng & Xen Orchestra - Vates solutions
web: https://vates.tech

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 06.07.2023 17:35, zithro wrote:
> On 06 Jul 2023 09:02, Jan Beulich wrote:
>> On 05.07.2023 18:20, zithro wrote:
>>> So I'm wondering, isn't that path enough for correct detection ?
>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
>>> other known hypervisor), it's nested, otherwise it's on hardware ?
>>>
>>> Is that really mandatory to use CPUID leaves ?
>>
>> Let me ask the other way around: In user mode code under a non-nested
>> vs nested Xen, what would you be able to derive from CPUID? The
>> "hypervisor" bit is going to be set in both cases. (All assuming you
>> run on new enough hardware+Xen such that CPUID would be intercepted
>> even for PV.)
> 
> I'm a bit clueless about CPUID stuff, but if I understand correctly, 
> you're essentially saying that using CPUID may not be the perfect way ?
> Also, I don't get why the cpuid command returns two different values, 
> depending on the -k switch :
> # cpuid -l 0x40000000
> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
> # cpuid -k -l 0x40000000
> hypervisor_id (0x40000000) = "XenVMMXenVMM"

I'm afraid I can't comment on this without knowing what tool you're
taking about. Neither of the two systems I checked have one of this
name.

>> Yet relying on DMI is fragile, too: Along the lines of
>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
>> basically any value in there could be "inherited" from the host (i.e.
>> from the layer below, to be precise).
> 
> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep 
> DMI:" is also not perfect, as values can be inherited/spoofed by 
> underneath hypervisor ?

That's my understanding, yes.

>> The only way to be reasonably
>> certain is to ask Xen about its view. The raw or host featuresets
>> should give you this information, in the "mirror" of said respective
>> CPUID leave's "hypervisor" bit. 
> 
> As said above, I'm clueless, can you expand please ?

Xen's public interface offers access to the featuresets known / found /
used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
via xc_get_cpu_featureset().

Jan

Re: Detecting whether dom0 is in a VM

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 2523 bytes --]

On Fri, Jul 7, 2023 at 9:00 AM Jan Beulich <jbeulich@suse.com> wrote:

> On 06.07.2023 17:35, zithro wrote:
> > On 06 Jul 2023 09:02, Jan Beulich wrote:
> >> On 05.07.2023 18:20, zithro wrote:
> >>> So I'm wondering, isn't that path enough for correct detection ?
> >>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
> >>> other known hypervisor), it's nested, otherwise it's on hardware ?
> >>>
> >>> Is that really mandatory to use CPUID leaves ?
> >>
> >> Let me ask the other way around: In user mode code under a non-nested
> >> vs nested Xen, what would you be able to derive from CPUID? The
> >> "hypervisor" bit is going to be set in both cases. (All assuming you
> >> run on new enough hardware+Xen such that CPUID would be intercepted
> >> even for PV.)
> >
> > I'm a bit clueless about CPUID stuff, but if I understand correctly,
> > you're essentially saying that using CPUID may not be the perfect way ?
> > Also, I don't get why the cpuid command returns two different values,
> > depending on the -k switch :
> > # cpuid -l 0x40000000
> > hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
> > # cpuid -k -l 0x40000000
> > hypervisor_id (0x40000000) = "XenVMMXenVMM"
>
> I'm afraid I can't comment on this without knowing what tool you're
> taking about. Neither of the two systems I checked have one of this
> name.
>
> >> Yet relying on DMI is fragile, too: Along the lines of
> >> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
> >> basically any value in there could be "inherited" from the host (i.e.
> >> from the layer below, to be precise).
> >
> > So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep
> > DMI:" is also not perfect, as values can be inherited/spoofed by
> > underneath hypervisor ?
>
> That's my understanding, yes.
>
> >> The only way to be reasonably
> >> certain is to ask Xen about its view. The raw or host featuresets
> >> should give you this information, in the "mirror" of said respective
> >> CPUID leave's "hypervisor" bit.
> >
> > As said above, I'm clueless, can you expand please ?
>
> Xen's public interface offers access to the featuresets known / found /
> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
> via xc_get_cpu_featureset().
>

Are any of these exposed in dom0 via sysctl, or hypfs?  SYSCTLs are
unfortunately not stable interfaces, correct?  So it wouldn't be practical
for systemd to use them.

 -George

[-- Attachment #2: Type: text/html, Size: 3424 bytes --]

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 07.07.2023 11:52, George Dunlap wrote:
> On Fri, Jul 7, 2023 at 9:00 AM Jan Beulich <jbeulich@suse.com> wrote:
> 
>> On 06.07.2023 17:35, zithro wrote:
>>> On 06 Jul 2023 09:02, Jan Beulich wrote:
>>>> On 05.07.2023 18:20, zithro wrote:
>>>>> So I'm wondering, isn't that path enough for correct detection ?
>>>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
>>>>> other known hypervisor), it's nested, otherwise it's on hardware ?
>>>>>
>>>>> Is that really mandatory to use CPUID leaves ?
>>>>
>>>> Let me ask the other way around: In user mode code under a non-nested
>>>> vs nested Xen, what would you be able to derive from CPUID? The
>>>> "hypervisor" bit is going to be set in both cases. (All assuming you
>>>> run on new enough hardware+Xen such that CPUID would be intercepted
>>>> even for PV.)
>>>
>>> I'm a bit clueless about CPUID stuff, but if I understand correctly,
>>> you're essentially saying that using CPUID may not be the perfect way ?
>>> Also, I don't get why the cpuid command returns two different values,
>>> depending on the -k switch :
>>> # cpuid -l 0x40000000
>>> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
>>> # cpuid -k -l 0x40000000
>>> hypervisor_id (0x40000000) = "XenVMMXenVMM"
>>
>> I'm afraid I can't comment on this without knowing what tool you're
>> taking about. Neither of the two systems I checked have one of this
>> name.
>>
>>>> Yet relying on DMI is fragile, too: Along the lines of
>>>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
>>>> basically any value in there could be "inherited" from the host (i.e.
>>>> from the layer below, to be precise).
>>>
>>> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep
>>> DMI:" is also not perfect, as values can be inherited/spoofed by
>>> underneath hypervisor ?
>>
>> That's my understanding, yes.
>>
>>>> The only way to be reasonably
>>>> certain is to ask Xen about its view. The raw or host featuresets
>>>> should give you this information, in the "mirror" of said respective
>>>> CPUID leave's "hypervisor" bit.
>>>
>>> As said above, I'm clueless, can you expand please ?
>>
>> Xen's public interface offers access to the featuresets known / found /
>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>> via xc_get_cpu_featureset().
>>
> 
> Are any of these exposed in dom0 via sysctl, or hypfs?

sysctl - yes (as the quoted name also says). hypfs no, afaict.

>  SYSCTLs are
> unfortunately not stable interfaces, correct?  So it wouldn't be practical
> for systemd to use them.

Indeed, neither sysctl-s nor the libxc interfaces are stable.

Jan

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 07.07.2023 12:16, Jan Beulich wrote:
> On 07.07.2023 11:52, George Dunlap wrote:
>> On Fri, Jul 7, 2023 at 9:00 AM Jan Beulich <jbeulich@suse.com> wrote:
>>
>>> On 06.07.2023 17:35, zithro wrote:
>>>> On 06 Jul 2023 09:02, Jan Beulich wrote:
>>>>> On 05.07.2023 18:20, zithro wrote:
>>>>>> So I'm wondering, isn't that path enough for correct detection ?
>>>>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
>>>>>> other known hypervisor), it's nested, otherwise it's on hardware ?
>>>>>>
>>>>>> Is that really mandatory to use CPUID leaves ?
>>>>>
>>>>> Let me ask the other way around: In user mode code under a non-nested
>>>>> vs nested Xen, what would you be able to derive from CPUID? The
>>>>> "hypervisor" bit is going to be set in both cases. (All assuming you
>>>>> run on new enough hardware+Xen such that CPUID would be intercepted
>>>>> even for PV.)
>>>>
>>>> I'm a bit clueless about CPUID stuff, but if I understand correctly,
>>>> you're essentially saying that using CPUID may not be the perfect way ?
>>>> Also, I don't get why the cpuid command returns two different values,
>>>> depending on the -k switch :
>>>> # cpuid -l 0x40000000
>>>> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
>>>> # cpuid -k -l 0x40000000
>>>> hypervisor_id (0x40000000) = "XenVMMXenVMM"
>>>
>>> I'm afraid I can't comment on this without knowing what tool you're
>>> taking about. Neither of the two systems I checked have one of this
>>> name.
>>>
>>>>> Yet relying on DMI is fragile, too: Along the lines of
>>>>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
>>>>> basically any value in there could be "inherited" from the host (i.e.
>>>>> from the layer below, to be precise).
>>>>
>>>> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep
>>>> DMI:" is also not perfect, as values can be inherited/spoofed by
>>>> underneath hypervisor ?
>>>
>>> That's my understanding, yes.
>>>
>>>>> The only way to be reasonably
>>>>> certain is to ask Xen about its view. The raw or host featuresets
>>>>> should give you this information, in the "mirror" of said respective
>>>>> CPUID leave's "hypervisor" bit.
>>>>
>>>> As said above, I'm clueless, can you expand please ?
>>>
>>> Xen's public interface offers access to the featuresets known / found /
>>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>>> via xc_get_cpu_featureset().
>>>
>>
>> Are any of these exposed in dom0 via sysctl, or hypfs?
> 
> sysctl - yes (as the quoted name also says). hypfs no, afaict.
> 
>>  SYSCTLs are
>> unfortunately not stable interfaces, correct?  So it wouldn't be practical
>> for systemd to use them.
> 
> Indeed, neither sysctl-s nor the libxc interfaces are stable.

Thinking of it, xen-cpuid is a wrapper tool around those. They may want
to look at its output (and, if they want to use it, advise distros to
also package it), which I think we try to keep reasonably stable,
albeit without providing any guarantees.

Jan

Re: Detecting whether dom0 is in a VM

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 2841 bytes --]

On Fri, Jul 7, 2023 at 11:17 AM Jan Beulich <jbeulich@suse.com> wrote:

> On 07.07.2023 11:52, George Dunlap wrote:
> > On Fri, Jul 7, 2023 at 9:00 AM Jan Beulich <jbeulich@suse.com> wrote:
> >
> >> On 06.07.2023 17:35, zithro wrote:
> >>> On 06 Jul 2023 09:02, Jan Beulich wrote:
> >>>> On 05.07.2023 18:20, zithro wrote:
> >>>>> So I'm wondering, isn't that path enough for correct detection ?
> >>>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or any
> >>>>> other known hypervisor), it's nested, otherwise it's on hardware ?
> >>>>>
> >>>>> Is that really mandatory to use CPUID leaves ?
> >>>>
> >>>> Let me ask the other way around: In user mode code under a non-nested
> >>>> vs nested Xen, what would you be able to derive from CPUID? The
> >>>> "hypervisor" bit is going to be set in both cases. (All assuming you
> >>>> run on new enough hardware+Xen such that CPUID would be intercepted
> >>>> even for PV.)
> >>>
> >>> I'm a bit clueless about CPUID stuff, but if I understand correctly,
> >>> you're essentially saying that using CPUID may not be the perfect way ?
> >>> Also, I don't get why the cpuid command returns two different values,
> >>> depending on the -k switch :
> >>> # cpuid -l 0x40000000
> >>> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
> >>> # cpuid -k -l 0x40000000
> >>> hypervisor_id (0x40000000) = "XenVMMXenVMM"
> >>
> >> I'm afraid I can't comment on this without knowing what tool you're
> >> taking about. Neither of the two systems I checked have one of this
> >> name.
> >>
> >>>> Yet relying on DMI is fragile, too: Along the lines of
> >>>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
> >>>> basically any value in there could be "inherited" from the host (i.e.
> >>>> from the layer below, to be precise).
> >>>
> >>> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep
> >>> DMI:" is also not perfect, as values can be inherited/spoofed by
> >>> underneath hypervisor ?
> >>
> >> That's my understanding, yes.
> >>
> >>>> The only way to be reasonably
> >>>> certain is to ask Xen about its view. The raw or host featuresets
> >>>> should give you this information, in the "mirror" of said respective
> >>>> CPUID leave's "hypervisor" bit.
> >>>
> >>> As said above, I'm clueless, can you expand please ?
> >>
> >> Xen's public interface offers access to the featuresets known / found /
> >> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
> >> via xc_get_cpu_featureset().
> >>
> >
> > Are any of these exposed in dom0 via sysctl, or hypfs?
>
> sysctl - yes (as the quoted name also says). hypfs no, afaict.
>

Sorry, that was a typo; I meant Linux's sysfs.  But of course if it's a
sysctl, I expect that to be a "no".

 -George

[-- Attachment #2: Type: text/html, Size: 4164 bytes --]

Re: Detecting whether dom0 is in a VM

[George Dunlap]

[-- Attachment #1: Type: text/plain, Size: 4287 bytes --]

On Fri, Jul 7, 2023 at 12:45 PM Jan Beulich <jbeulich@suse.com> wrote:

> On 07.07.2023 12:16, Jan Beulich wrote:
> > On 07.07.2023 11:52, George Dunlap wrote:
> >> On Fri, Jul 7, 2023 at 9:00 AM Jan Beulich <jbeulich@suse.com> wrote:
> >>
> >>> On 06.07.2023 17:35, zithro wrote:
> >>>> On 06 Jul 2023 09:02, Jan Beulich wrote:
> >>>>> On 05.07.2023 18:20, zithro wrote:
> >>>>>> So I'm wondering, isn't that path enough for correct detection ?
> >>>>>> I mean, if "/sys/class/dmi/id/sys_vendor" reports Xen (or KVM, or
> any
> >>>>>> other known hypervisor), it's nested, otherwise it's on hardware ?
> >>>>>>
> >>>>>> Is that really mandatory to use CPUID leaves ?
> >>>>>
> >>>>> Let me ask the other way around: In user mode code under a non-nested
> >>>>> vs nested Xen, what would you be able to derive from CPUID? The
> >>>>> "hypervisor" bit is going to be set in both cases. (All assuming you
> >>>>> run on new enough hardware+Xen such that CPUID would be intercepted
> >>>>> even for PV.)
> >>>>
> >>>> I'm a bit clueless about CPUID stuff, but if I understand correctly,
> >>>> you're essentially saying that using CPUID may not be the perfect way
> ?
> >>>> Also, I don't get why the cpuid command returns two different values,
> >>>> depending on the -k switch :
> >>>> # cpuid -l 0x40000000
> >>>> hypervisor_id (0x40000000) = "\0\0\0\0\0\0\0\0\0\0\0\0"
> >>>> # cpuid -k -l 0x40000000
> >>>> hypervisor_id (0x40000000) = "XenVMMXenVMM"
> >>>
> >>> I'm afraid I can't comment on this without knowing what tool you're
> >>> taking about. Neither of the two systems I checked have one of this
> >>> name.
> >>>
> >>>>> Yet relying on DMI is fragile, too: Along the lines of
> >>>>> https://lists.xen.org/archives/html/xen-devel/2022-01/msg00604.html
> >>>>> basically any value in there could be "inherited" from the host (i.e.
> >>>>> from the layer below, to be precise).
> >>>>
> >>>> So using "/sys/class/dmi/id/sys_vendor", or simply doing "dmesg | grep
> >>>> DMI:" is also not perfect, as values can be inherited/spoofed by
> >>>> underneath hypervisor ?
> >>>
> >>> That's my understanding, yes.
> >>>
> >>>>> The only way to be reasonably
> >>>>> certain is to ask Xen about its view. The raw or host featuresets
> >>>>> should give you this information, in the "mirror" of said respective
> >>>>> CPUID leave's "hypervisor" bit.
> >>>>
> >>>> As said above, I'm clueless, can you expand please ?
> >>>
> >>> Xen's public interface offers access to the featuresets known / found /
> >>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
> >>> via xc_get_cpu_featureset().
> >>>
> >>
> >> Are any of these exposed in dom0 via sysctl, or hypfs?
> >
> > sysctl - yes (as the quoted name also says). hypfs no, afaict.
> >
> >>  SYSCTLs are
> >> unfortunately not stable interfaces, correct?  So it wouldn't be
> practical
> >> for systemd to use them.
> >
> > Indeed, neither sysctl-s nor the libxc interfaces are stable.
>
> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
> to look at its output (and, if they want to use it, advise distros to
> also package it), which I think we try to keep reasonably stable,
> albeit without providing any guarantees.
>

We haven't had any clear guidance yet on what the systemd team want in the
<xen in a VM, systemd in a dom0> question; I just sort of assumed they
wanted the L-1 virtualization *if possible*.  It sounds like `vm-other`
would be acceptable, particularly as a fall-back output if there's no way
to get Xen's picture of the cpuid.

It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old
Virt SIG CentOS packages; so I'd expect most packages to follow suit.
That's a place to start.

Just to take the discussion all the way to its conclusion:

- Supposing xen-cpuid isn't available, is there any other way to tell if
Xen is running in a VM from dom0?

- Would it make sense to expose that information somewhere, either in sysfs
or in hypfs (or both?), so that eventually even systems which may not get
the memo about packaging xen-cpuid will get support (or if the systemd guys
would rather avoid executing another process if possible)?

 -George

[-- Attachment #2: Type: text/html, Size: 5932 bytes --]

Re: Detecting whether dom0 is in a VM

[George Dunlap]

On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
>> >>> Xen's public interface offers access to the featuresets known / found /
>> >>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>> >>> via xc_get_cpu_featureset().
>> >>>
>> >>
>> >> Are any of these exposed in dom0 via sysctl, or hypfs?
>> >
>> > sysctl - yes (as the quoted name also says). hypfs no, afaict.
>> >
>> >>  SYSCTLs are
>> >> unfortunately not stable interfaces, correct?  So it wouldn't be practical
>> >> for systemd to use them.
>> >
>> > Indeed, neither sysctl-s nor the libxc interfaces are stable.
>>
>> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
>> to look at its output (and, if they want to use it, advise distros to
>> also package it), which I think we try to keep reasonably stable,
>> albeit without providing any guarantees.
>
>
> We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
>
> It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
>
> Just to take the discussion all the way to its conclusion:
>
> - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
>
> - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?

Resurrecting this thread.

To recap:

Currently, `systemd-detect-virt` has the following  input / output table:

1. Xen on real hardware, domU: xen
2. Xen on real hardware, dom0: vm-other
3. Xen in a VM, domU: xen
4. Xen in aVM, dom0: vm-other

It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
those to be `none` and `vm-other` (or the actual value, like `xen` or
`kvm`).

It looks like ATM, /proc/xen/capabilities will contain `control_d` if
it's a dom0.  Simply unilaterally returning `none` if
/proc/xen/capabilities contains `control_d` would correct the vast
majority of instances (since the number of instances of Xen on real
hardware is presumably higher than the number running virtualized).

Is /proc/xen/capabilities expected to stay around?  I don't see
anything equivalent in /dev/xen.

Other than adding a new interface to Xen, is there any way to tell if
Xen is running in a VM?  If we do need to expose an interface, what
would be the best way to do that?

 -George

Re: Detecting whether dom0 is in a VM

[George Dunlap]

On Fri, Apr 19, 2024 at 4:29 PM George Dunlap <george.dunlap@cloud.com> wrote:
>
> On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
> >> >>> Xen's public interface offers access to the featuresets known / found /
> >> >>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
> >> >>> via xc_get_cpu_featureset().
> >> >>>
> >> >>
> >> >> Are any of these exposed in dom0 via sysctl, or hypfs?
> >> >
> >> > sysctl - yes (as the quoted name also says). hypfs no, afaict.
> >> >
> >> >>  SYSCTLs are
> >> >> unfortunately not stable interfaces, correct?  So it wouldn't be practical
> >> >> for systemd to use them.
> >> >
> >> > Indeed, neither sysctl-s nor the libxc interfaces are stable.
> >>
> >> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
> >> to look at its output (and, if they want to use it, advise distros to
> >> also package it), which I think we try to keep reasonably stable,
> >> albeit without providing any guarantees.
> >
> >
> > We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
> >
> > It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
> >
> > Just to take the discussion all the way to its conclusion:
> >
> > - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
> >
> > - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?
>
> Resurrecting this thread.
>
> To recap:
>
> Currently, `systemd-detect-virt` has the following  input / output table:
>
> 1. Xen on real hardware, domU: xen
> 2. Xen on real hardware, dom0: vm-other
> 3. Xen in a VM, domU: xen
> 4. Xen in aVM, dom0: vm-other
>
> It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
> those to be `none` and `vm-other` (or the actual value, like `xen` or
> `kvm`).
>
> It looks like ATM, /proc/xen/capabilities will contain `control_d` if
> it's a dom0.  Simply unilaterally returning `none` if
> /proc/xen/capabilities contains `control_d` would correct the vast
> majority of instances (since the number of instances of Xen on real
> hardware is presumably higher than the number running virtualized).
>
> Is /proc/xen/capabilities expected to stay around?  I don't see
> anything equivalent in /dev/xen.
>
> Other than adding a new interface to Xen, is there any way to tell if
> Xen is running in a VM?  If we do need to expose an interface, what
> would be the best way to do that?

Actually, the entire code for detection is here:

https://github.com/systemd/systemd/blob/f5fefec786e35ef7606e2b14e2530878b74ca879/src/basic/virt.c

The core issue seems to be this bit:

        /* Detect from CPUID */
        v = detect_vm_cpuid();
        if (v < 0)
                return v;
        if (v == VIRTUALIZATION_VM_OTHER)
                other = true;
        else if (v != VIRTUALIZATION_NONE)
                goto finish;

        /* If we are in Dom0 and have not yet finished, finish with
the result of detect_vm_cpuid */
        if (xen_dom0 > 0)
                goto finish;

Basically, the code expects that dom0 will be able to read the raw
virtualization CPUID leaves, and that the result will be
VIRTUALIZATION_NONE on real hardware.

But in fact, the result appears to be VIRTUALIZATION_VM_OTHER -- which
would mean that 1) the CPUID instruction for reading CPUID leaf 0x1
succeeded, 2) the 'hypervisor' bit was set, but 3) the signature at
the hypervisor information leaf (0x40000000) didn't match any
hypervisor (including XenVMMXenVMM, which it's looking for).

What do we do in regard to these for dom0?

 -George

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 19.04.2024 17:51, George Dunlap wrote:
> On Fri, Apr 19, 2024 at 4:29 PM George Dunlap <george.dunlap@cloud.com> wrote:
>>
>> On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
>>>>>>> Xen's public interface offers access to the featuresets known / found /
>>>>>>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>>>>>>> via xc_get_cpu_featureset().
>>>>>>>
>>>>>>
>>>>>> Are any of these exposed in dom0 via sysctl, or hypfs?
>>>>>
>>>>> sysctl - yes (as the quoted name also says). hypfs no, afaict.
>>>>>
>>>>>>  SYSCTLs are
>>>>>> unfortunately not stable interfaces, correct?  So it wouldn't be practical
>>>>>> for systemd to use them.
>>>>>
>>>>> Indeed, neither sysctl-s nor the libxc interfaces are stable.
>>>>
>>>> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
>>>> to look at its output (and, if they want to use it, advise distros to
>>>> also package it), which I think we try to keep reasonably stable,
>>>> albeit without providing any guarantees.
>>>
>>>
>>> We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
>>>
>>> It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
>>>
>>> Just to take the discussion all the way to its conclusion:
>>>
>>> - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
>>>
>>> - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?
>>
>> Resurrecting this thread.
>>
>> To recap:
>>
>> Currently, `systemd-detect-virt` has the following  input / output table:
>>
>> 1. Xen on real hardware, domU: xen
>> 2. Xen on real hardware, dom0: vm-other
>> 3. Xen in a VM, domU: xen
>> 4. Xen in aVM, dom0: vm-other
>>
>> It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
>> those to be `none` and `vm-other` (or the actual value, like `xen` or
>> `kvm`).
>>
>> It looks like ATM, /proc/xen/capabilities will contain `control_d` if
>> it's a dom0.  Simply unilaterally returning `none` if
>> /proc/xen/capabilities contains `control_d` would correct the vast
>> majority of instances (since the number of instances of Xen on real
>> hardware is presumably higher than the number running virtualized).
>>
>> Is /proc/xen/capabilities expected to stay around?  I don't see
>> anything equivalent in /dev/xen.
>>
>> Other than adding a new interface to Xen, is there any way to tell if
>> Xen is running in a VM?  If we do need to expose an interface, what
>> would be the best way to do that?
> 
> Actually, the entire code for detection is here:
> 
> https://github.com/systemd/systemd/blob/f5fefec786e35ef7606e2b14e2530878b74ca879/src/basic/virt.c
> 
> The core issue seems to be this bit:
> 
>         /* Detect from CPUID */
>         v = detect_vm_cpuid();
>         if (v < 0)
>                 return v;
>         if (v == VIRTUALIZATION_VM_OTHER)
>                 other = true;
>         else if (v != VIRTUALIZATION_NONE)
>                 goto finish;
> 
>         /* If we are in Dom0 and have not yet finished, finish with
> the result of detect_vm_cpuid */
>         if (xen_dom0 > 0)
>                 goto finish;
> 
> Basically, the code expects that dom0 will be able to read the raw
> virtualization CPUID leaves, and that the result will be
> VIRTUALIZATION_NONE on real hardware.
> 
> But in fact, the result appears to be VIRTUALIZATION_VM_OTHER -- which
> would mean that 1) the CPUID instruction for reading CPUID leaf 0x1
> succeeded, 2) the 'hypervisor' bit was set, but 3) the signature at
> the hypervisor information leaf (0x40000000) didn't match any
> hypervisor (including XenVMMXenVMM, which it's looking for).

Which in turn suggests that they (a) issue plain CPUID (not the "Xen
escaped" form of it) and (b) there's no CPUID faulting available in the
underlying hardware (or the Xen version used is too old). Since (b) is
nothing userspace can do anything about, using the "Xen escaped" form
of CPUID may be the way to go in order to actually see the hypervisor
leaves. Beyond that ...

> What do we do in regard to these for dom0?

... I'm afraid I have no good idea at the moment.

Jan

Re: Detecting whether dom0 is in a VM

[Jan Beulich]

On 19.04.2024 17:29, George Dunlap wrote:
> On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
>>>>>> Xen's public interface offers access to the featuresets known / found /
>>>>>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>>>>>> via xc_get_cpu_featureset().
>>>>>>
>>>>>
>>>>> Are any of these exposed in dom0 via sysctl, or hypfs?
>>>>
>>>> sysctl - yes (as the quoted name also says). hypfs no, afaict.
>>>>
>>>>>  SYSCTLs are
>>>>> unfortunately not stable interfaces, correct?  So it wouldn't be practical
>>>>> for systemd to use them.
>>>>
>>>> Indeed, neither sysctl-s nor the libxc interfaces are stable.
>>>
>>> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
>>> to look at its output (and, if they want to use it, advise distros to
>>> also package it), which I think we try to keep reasonably stable,
>>> albeit without providing any guarantees.
>>
>>
>> We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
>>
>> It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
>>
>> Just to take the discussion all the way to its conclusion:
>>
>> - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
>>
>> - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?
> 
> Resurrecting this thread.
> 
> To recap:
> 
> Currently, `systemd-detect-virt` has the following  input / output table:
> 
> 1. Xen on real hardware, domU: xen
> 2. Xen on real hardware, dom0: vm-other
> 3. Xen in a VM, domU: xen
> 4. Xen in aVM, dom0: vm-other
> 
> It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
> those to be `none` and `vm-other` (or the actual value, like `xen` or
> `kvm`).
> 
> It looks like ATM, /proc/xen/capabilities will contain `control_d` if
> it's a dom0.  Simply unilaterally returning `none` if
> /proc/xen/capabilities contains `control_d` would correct the vast
> majority of instances (since the number of instances of Xen on real
> hardware is presumably higher than the number running virtualized).
> 
> Is /proc/xen/capabilities expected to stay around?  I don't see
> anything equivalent in /dev/xen.

I believe it's intended to stay around, but a definitive answer can only
come from Linux folks. Jürgen, Stefano?

Jan

> Other than adding a new interface to Xen, is there any way to tell if
> Xen is running in a VM?  If we do need to expose an interface, what
> would be the best way to do that?
> 
>  -George

Re: Detecting whether dom0 is in a VM

[Stefano Stabellini]

[-- Attachment #1: Type: text/plain, Size: 3127 bytes --]

On Mon, 22 Apr 2024, Jan Beulich wrote:
> On 19.04.2024 17:29, George Dunlap wrote:
> > On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
> >>>>>> Xen's public interface offers access to the featuresets known / found /
> >>>>>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
> >>>>>> via xc_get_cpu_featureset().
> >>>>>>
> >>>>>
> >>>>> Are any of these exposed in dom0 via sysctl, or hypfs?
> >>>>
> >>>> sysctl - yes (as the quoted name also says). hypfs no, afaict.
> >>>>
> >>>>>  SYSCTLs are
> >>>>> unfortunately not stable interfaces, correct?  So it wouldn't be practical
> >>>>> for systemd to use them.
> >>>>
> >>>> Indeed, neither sysctl-s nor the libxc interfaces are stable.
> >>>
> >>> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
> >>> to look at its output (and, if they want to use it, advise distros to
> >>> also package it), which I think we try to keep reasonably stable,
> >>> albeit without providing any guarantees.
> >>
> >>
> >> We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
> >>
> >> It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
> >>
> >> Just to take the discussion all the way to its conclusion:
> >>
> >> - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
> >>
> >> - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?
> > 
> > Resurrecting this thread.
> > 
> > To recap:
> > 
> > Currently, `systemd-detect-virt` has the following  input / output table:
> > 
> > 1. Xen on real hardware, domU: xen
> > 2. Xen on real hardware, dom0: vm-other
> > 3. Xen in a VM, domU: xen
> > 4. Xen in aVM, dom0: vm-other
> > 
> > It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
> > those to be `none` and `vm-other` (or the actual value, like `xen` or
> > `kvm`).
> > 
> > It looks like ATM, /proc/xen/capabilities will contain `control_d` if
> > it's a dom0.  Simply unilaterally returning `none` if
> > /proc/xen/capabilities contains `control_d` would correct the vast
> > majority of instances (since the number of instances of Xen on real
> > hardware is presumably higher than the number running virtualized).
> > 
> > Is /proc/xen/capabilities expected to stay around?  I don't see
> > anything equivalent in /dev/xen.
> 
> I believe it's intended to stay around, but a definitive answer can only
> come from Linux folks. Jürgen, Stefano?

No plans to get rid of /proc/xen/capabilities that I am aware

Re: Detecting whether dom0 is in a VM

[Jürgen Groß]

On 22.04.24 09:12, Jan Beulich wrote:
> On 19.04.2024 17:29, George Dunlap wrote:
>> On Fri, Jul 7, 2023 at 3:56 PM George Dunlap <george.dunlap@cloud.com> wrote:
>>>>>>> Xen's public interface offers access to the featuresets known / found /
>>>>>>> used by the hypervisor. See XEN_SYSCTL_get_cpu_featureset, accessible
>>>>>>> via xc_get_cpu_featureset().
>>>>>>>
>>>>>>
>>>>>> Are any of these exposed in dom0 via sysctl, or hypfs?
>>>>>
>>>>> sysctl - yes (as the quoted name also says). hypfs no, afaict.
>>>>>
>>>>>>   SYSCTLs are
>>>>>> unfortunately not stable interfaces, correct?  So it wouldn't be practical
>>>>>> for systemd to use them.
>>>>>
>>>>> Indeed, neither sysctl-s nor the libxc interfaces are stable.
>>>>
>>>> Thinking of it, xen-cpuid is a wrapper tool around those. They may want
>>>> to look at its output (and, if they want to use it, advise distros to
>>>> also package it), which I think we try to keep reasonably stable,
>>>> albeit without providing any guarantees.
>>>
>>>
>>> We haven't had any clear guidance yet on what the systemd team want in the <xen in a VM, systemd in a dom0> question; I just sort of assumed they wanted the L-1 virtualization *if possible*.  It sounds like `vm-other` would be acceptable, particularly as a fall-back output if there's no way to get Xen's picture of the cpuid.
>>>
>>> It looks like xen-cpuid is available on Fedora, Debian, Ubuntu, and the old Virt SIG CentOS packages; so I'd expect most packages to follow suit.  That's a place to start.
>>>
>>> Just to take the discussion all the way to its conclusion:
>>>
>>> - Supposing xen-cpuid isn't available, is there any other way to tell if Xen is running in a VM from dom0?
>>>
>>> - Would it make sense to expose that information somewhere, either in sysfs or in hypfs (or both?), so that eventually even systems which may not get the memo about packaging xen-cpuid will get support (or if the systemd guys would rather avoid executing another process if possible)?
>>
>> Resurrecting this thread.
>>
>> To recap:
>>
>> Currently, `systemd-detect-virt` has the following  input / output table:
>>
>> 1. Xen on real hardware, domU: xen
>> 2. Xen on real hardware, dom0: vm-other
>> 3. Xen in a VM, domU: xen
>> 4. Xen in aVM, dom0: vm-other
>>
>> It's the dom0 lines (2 and 4) which are problematic; we'd ideally like
>> those to be `none` and `vm-other` (or the actual value, like `xen` or
>> `kvm`).
>>
>> It looks like ATM, /proc/xen/capabilities will contain `control_d` if
>> it's a dom0.  Simply unilaterally returning `none` if
>> /proc/xen/capabilities contains `control_d` would correct the vast
>> majority of instances (since the number of instances of Xen on real
>> hardware is presumably higher than the number running virtualized).
>>
>> Is /proc/xen/capabilities expected to stay around?  I don't see
>> anything equivalent in /dev/xen.
> 
> I believe it's intended to stay around, but a definitive answer can only
> come from Linux folks. Jürgen, Stefano?

I have no plans to remove it.


Juergen

> 
> Jan
> 
>> Other than adding a new interface to Xen, is there any way to tell if
>> Xen is running in a VM?  If we do need to expose an interface, what
>> would be the best way to do that?
>>
>>   -George
>