Command line options of dubious use

Command line options of dubious use

[Andrew Cooper]

Hello,

As I've spent a while staring at the command line docs recently, I've
come to the conclusion that we should probably remove these:

* ro-hpet

I'm afraid that I didn't spot this one going in, and would have objected
to it if I'd found it.  dom0 (either PV, or PVH) cannot use the HPET
safely, even if it is restricted to just read-only access.  Dom0 must
under no circumstance interact directly with the hardware HPET, as it is
a direct interrupt source.  A related problem is that Linux has chipset
quirks for missing HPET ACPI tables, and on some systems can manage to
program the HPET behind Xen's back, resulting in chaos.  The default
MMIO locations of these devices are standard nowadays, so we should
probably blacklist mapping attempts completely.

If there does happen to be something else adjacent to the HPET in the
same page, the only safe way to handle the 4k frame as emulated MMIO,
and forward accesses to the latter 3072 bytes to hardware.

* tbuf_size and tevt_mask

The command line documentation for these refers to the xentrace(8)
documentation, and AFAICT, selecting these on the hypervisor command
line only serves to prevent xentrace's ability to set them at runtime. 
Given that xentrace can set them at runtime, and it is debugging
functionality, I don't see a plausible use the command line options at all.

* vga = ask

The single piece of keyboard interaction we have in Xen is the 16bit
assembly code menu to display the graphics adapter modes.  This clearly
isn't used in production due to it blocking for an answer, but does
anyone use it in development?  At the point that you can edit the boot
command line to ask for the right mode, a suitable mode is already
available in the bootloader.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Olaf Hering]

[-- Attachment #1.1: Type: text/plain, Size: 359 bytes --]

On Mon, Dec 31, Andrew Cooper wrote:

> * tbuf_size and tevt_mask
> Given that xentrace can set them at runtime, and it is debugging
> functionality, I don't see a plausible use the command line options at all.

'tbuf_size=-1 dom0_mem=N' collects events during boot.
xenalyze is (or was) unable to properly parse the result, but thats a
separate issue.

Olaf

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 195 bytes --]

[-- Attachment #2: Type: text/plain, Size: 157 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Andrew Cooper]

On 02/01/2019 16:56, Olaf Hering wrote:
> On Mon, Dec 31, Andrew Cooper wrote:
>
>> * tbuf_size and tevt_mask
>> Given that xentrace can set them at runtime, and it is debugging
>> functionality, I don't see a plausible use the command line options at all.
> 'tbuf_size=-1 dom0_mem=N' collects events during boot.
> xenalyze is (or was) unable to properly parse the result, but thats a
> separate issue.

Ooh - I'd not considered that as a usecase at all.  That probably is
worthwhile keeping.

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Andrii Anisov]

On 02.01.19 19:15, Andrew Cooper wrote:
> On 02/01/2019 16:56, Olaf Hering wrote:
>> On Mon, Dec 31, Andrew Cooper wrote:
>>
>>> * tbuf_size and tevt_mask
>>> Given that xentrace can set them at runtime, and it is debugging
>>> functionality, I don't see a plausible use the command line options at all.
>> 'tbuf_size=-1 dom0_mem=N' collects events during boot.
>> xenalyze is (or was) unable to properly parse the result, but thats a
>> separate issue.
> 
> Ooh - I'd not considered that as a usecase at all.  That probably is
> worthwhile keeping.

IMO it also worth to be documented as a good practice.

-- 
Sincerely,
Andrii Anisov.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Jan Beulich]

>>> On 31.12.18 at 19:46, <andrew.cooper3@citrix.com> wrote:
> As I've spent a while staring at the command line docs recently, I've
> come to the conclusion that we should probably remove these:
> 
> * ro-hpet
> 
> I'm afraid that I didn't spot this one going in, and would have objected
> to it if I'd found it.

Not how the commit introducing this actually tightened things,
rather than (like you make it sound) unduly relaxing them. The
option was introduced as a safe guard in case people run into
problems.

>  dom0 (either PV, or PVH) cannot use the HPET
> safely, even if it is restricted to just read-only access.  Dom0 must
> under no circumstance interact directly with the hardware HPET, as it is
> a direct interrupt source.

But reads don't cause interrupts, do they?

Just like with the IO-APIC, the main problem here isn't going to be
the Dom0 kernel, but ACPI methods accessing certain pieces of
hardware. That's the price we pay for the split brain model we use
for dealing with ACPI. For this reason I'm afraid I would object to
any attempt to remove the option, despite the care that's needed
when wanting/needing to make use of it.

>  A related problem is that Linux has chipset
> quirks for missing HPET ACPI tables, and on some systems can manage to
> program the HPET behind Xen's back, resulting in chaos.  The default
> MMIO locations of these devices are standard nowadays, so we should
> probably blacklist mapping attempts completely.
> 
> If there does happen to be something else adjacent to the HPET in the
> same page, the only safe way to handle the 4k frame as emulated MMIO,
> and forward accesses to the latter 3072 bytes to hardware.

Right; we didn't want to implement this until actually running into a
system in need of it.

> * vga = ask
> 
> The single piece of keyboard interaction we have in Xen is the 16bit
> assembly code menu to display the graphics adapter modes.  This clearly
> isn't used in production due to it blocking for an answer, but does
> anyone use it in development?

It's been less than half a year ago that I had to use it.

>  At the point that you can edit the boot
> command line to ask for the right mode, a suitable mode is already
> available in the bootloader.

But that's just one, not necessarily the one you'd like to use.
On the system that I needed to use it, the set of modes usable
at boot time (as reported by the VESA BIOS) was different from
the set reported at runtime (with X already active) by hwinfo or
some such, so picking a _reliably valid_ mode from the list
provided there was not possible.

Jan



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Andrew Cooper]

On 04/01/2019 14:04, Jan Beulich wrote:
>>  dom0 (either PV, or PVH) cannot use the HPET
>> safely, even if it is restricted to just read-only access.  Dom0 must
>> under no circumstance interact directly with the hardware HPET, as it is
>> a direct interrupt source.
> But reads don't cause interrupts, do they?

No, but they do need to be performed under lock, and excluded against
Xen actions.

> Just like with the IO-APIC, the main problem here isn't going to be
> the Dom0 kernel, but ACPI methods accessing certain pieces of
> hardware. That's the price we pay for the split brain model we use
> for dealing with ACPI. For this reason I'm afraid I would object to
> any attempt to remove the option, despite the care that's needed
> when wanting/needing to make use of it.

There can only be a single driver per HPET in the system.  In our case,
that must be Xen, even if dom0 can see the HPET.

AML cannot use an HPET which is exposed to the OS, because it can't
reasonably know whether the main counter is enabled or not, certainly
can't play with the enablement of the main counter behind the OS's back,
and can't safely use it in a read-only capacity because of spliced reads.

AML could in principle have a full HPET driver (as a timesource, if not
an interrupt source), if it hid the HPET entirely from the OS, but in
this case, it will explode on a read-only mapping.

>>  A related problem is that Linux has chipset
>> quirks for missing HPET ACPI tables, and on some systems can manage to
>> program the HPET behind Xen's back, resulting in chaos.  The default
>> MMIO locations of these devices are standard nowadays, so we should
>> probably blacklist mapping attempts completely.
>>
>> If there does happen to be something else adjacent to the HPET in the
>> same page, the only safe way to handle the 4k frame as emulated MMIO,
>> and forward accesses to the latter 3072 bytes to hardware.
> Right; we didn't want to implement this until actually running into a
> system in need of it.

So we opted for subtle timing bugs over obvious crashes, on the
unrealistic offchance that AML is reading the HPET?

The mapping should either be non-existent (so we actually spot attempts
to use it), or have a fully emulated HPET in it, with the rest of the 4k
frame with emulated and forwarded.

>
>> * vga = ask
>>
>> The single piece of keyboard interaction we have in Xen is the 16bit
>> assembly code menu to display the graphics adapter modes.  This clearly
>> isn't used in production due to it blocking for an answer, but does
>> anyone use it in development?
> It's been less than half a year ago that I had to use it.
>
>>  At the point that you can edit the boot
>> command line to ask for the right mode, a suitable mode is already
>> available in the bootloader.
> But that's just one, not necessarily the one you'd like to use.
> On the system that I needed to use it, the set of modes usable
> at boot time (as reported by the VESA BIOS) was different from
> the set reported at runtime (with X already active) by hwinfo or
> some such, so picking a _reliably valid_ mode from the list
> provided there was not possible.

So what you are saying is that, despite attempting to use the option, it
didn't actually work usefully?

~Andrew

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: Command line options of dubious use

[Jan Beulich]

>>> On 14.01.19 at 12:05, <andrew.cooper3@citrix.com> wrote:
> On 04/01/2019 14:04, Jan Beulich wrote:
>>>  dom0 (either PV, or PVH) cannot use the HPET
>>> safely, even if it is restricted to just read-only access.  Dom0 must
>>> under no circumstance interact directly with the hardware HPET, as it is
>>> a direct interrupt source.
>> But reads don't cause interrupts, do they?
> 
> No, but they do need to be performed under lock, and excluded against
> Xen actions.

That's for the result to be sensible, i.e. affecting Dom0. There's then
still no harm to Xen afaict.

>> Just like with the IO-APIC, the main problem here isn't going to be
>> the Dom0 kernel, but ACPI methods accessing certain pieces of
>> hardware. That's the price we pay for the split brain model we use
>> for dealing with ACPI. For this reason I'm afraid I would object to
>> any attempt to remove the option, despite the care that's needed
>> when wanting/needing to make use of it.
> 
> There can only be a single driver per HPET in the system.  In our case,
> that must be Xen, even if dom0 can see the HPET.
> 
> AML cannot use an HPET which is exposed to the OS, because it can't
> reasonably know whether the main counter is enabled or not,

It can read the config register.

> certainly
> can't play with the enablement of the main counter behind the OS's back,
> and can't safely use it in a read-only capacity because of spliced reads.
> 
> AML could in principle have a full HPET driver (as a timesource, if not
> an interrupt source), if it hid the HPET entirely from the OS, but in
> this case, it will explode on a read-only mapping.
> 
>>>  A related problem is that Linux has chipset
>>> quirks for missing HPET ACPI tables, and on some systems can manage to
>>> program the HPET behind Xen's back, resulting in chaos.  The default
>>> MMIO locations of these devices are standard nowadays, so we should
>>> probably blacklist mapping attempts completely.
>>>
>>> If there does happen to be something else adjacent to the HPET in the
>>> same page, the only safe way to handle the 4k frame as emulated MMIO,
>>> and forward accesses to the latter 3072 bytes to hardware.
>> Right; we didn't want to implement this until actually running into a
>> system in need of it.
> 
> So we opted for subtle timing bugs over obvious crashes, on the
> unrealistic offchance that AML is reading the HPET?

You take an unfortunate perspective here. In the similar IO-APIC
case, the reads or RTEs were - as far as I was able to tell from
analyzing two or three variants of affected AML code - completely
useless. Presumably just some debugging code left somewhere.
It seems quite desirable to be able to boot on such systems without
any oopses in Dom0.

> The mapping should either be non-existent (so we actually spot attempts
> to use it), or have a fully emulated HPET in it, with the rest of the 4k
> frame with emulated and forwarded.

That would indeed be best, in an ideal world.

>>> * vga = ask
>>>
>>> The single piece of keyboard interaction we have in Xen is the 16bit
>>> assembly code menu to display the graphics adapter modes.  This clearly
>>> isn't used in production due to it blocking for an answer, but does
>>> anyone use it in development?
>> It's been less than half a year ago that I had to use it.
>>
>>>  At the point that you can edit the boot
>>> command line to ask for the right mode, a suitable mode is already
>>> available in the bootloader.
>> But that's just one, not necessarily the one you'd like to use.
>> On the system that I needed to use it, the set of modes usable
>> at boot time (as reported by the VESA BIOS) was different from
>> the set reported at runtime (with X already active) by hwinfo or
>> some such, so picking a _reliably valid_ mode from the list
>> provided there was not possible.
> 
> So what you are saying is that, despite attempting to use the option, it
> didn't actually work usefully?

Sort of - I did obtain the desired mode when the system was fully up
(a 1280x1024 one), just to find that during boot it wouldn't work. I
tried one or two more (with different color depths) until I finally used
the "vga=ask" option to see what the BIOS reports valid at that
point. To my surprise, only 1024x768 modes were listed (on a pretty
recent nVidia card with 6GB of framebuffer).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel