Re: [RFC PATCH 0/6] virtio: Solution to restrict memory access under Xen using xen-virtio DMA ops layer

Re: [RFC PATCH 0/6] virtio: Solution to restrict memory access under Xen using xen-virtio DMA ops layer

[Christoph Hellwig]

I can only see three out of 6 patches on the linux-arm-kernel list,
which makes reviewing this impossible.  Also please Cc me directly
on any series doing crazy things with dma ops.  Thanks!
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 0/6] virtio: Solution to restrict memory access under Xen using xen-virtio DMA ops layer

[Michael S. Tsirkin]

On Thu, Apr 14, 2022 at 10:19:27PM +0300, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Hello all.
> 
> The purpose of this RFC patch series is to add support for restricting memory access under Xen using specific
> grant table based DMA ops layer. Patch series is based on Juergen Gross’ initial work [1] which implies using
> grant references instead of raw guest physical addresses (GPA) for the virtio communications (some kind of
> the software IOMMU).
> 
> The high level idea is to create new Xen’s grant table based DMA ops layer for the guest Linux whose main
> purpose is to provide a special 64-bit DMA address which is formed by using the grant reference (for a page
> to be shared with the backend) with offset and setting the highest address bit (this is for the backend to
> be able to distinguish grant ref based DMA address from normal GPA). For this to work we need the ability
> to allocate contiguous (consecutive) grant references for multi-page allocations. And the backend then needs
> to offer VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 feature bits (it must support virtio-mmio modern
> transport for 64-bit addresses in the virtqueue).

I'm not enough of a xen expert to review this, and I didn't get
all patches, but I'm very happy to see that approach being
taken. VIRTIO_F_ACCESS_PLATFORM and VIRTIO_F_VERSION_1 are
exactly the way to declare not all of memory is accessible.
Thanks!

> Xen's grant mapping mechanism is the secure and safe solution to share pages between domains which proven
> to work and works for years (in the context of traditional Xen PV drivers for example). So far, the foreign
> mapping is used for the virtio backend to map and access guest memory. With the foreign mapping, the backend
> is able to map arbitrary pages from the guest memory (or even from Dom0 memory). And as the result, the malicious
> backend which runs in a non-trusted domain can take advantage of this. Instead, with the grant mapping
> the backend is only allowed to map pages which were explicitly granted by the guest before and nothing else. 
> According to the discussions in various mainline threads this solution would likely be welcome because it
> perfectly fits in the security model Xen provides. 
> 
> What is more, the grant table based solution requires zero changes to the Xen hypervisor itself at least
> with virtio-mmio and DT (in comparison, for example, with "foreign mapping + virtio-iommu" solution which would
> require the whole new complex emulator in hypervisor in addition to new functionality/hypercall to pass IOVA
> from the virtio backend running elsewhere to the hypervisor and translate it to the GPA before mapping into
> P2M or denying the foreign mapping request if no corresponding IOVA-GPA mapping present in the IOMMU page table
> for that particular device). We only need to update toolstack to insert a new "xen,dev-domid" property to
> the virtio-mmio device node when creating a guest device-tree (this is an indicator for the guest to use grants
> and the ID of Xen domain where the corresponding backend resides, it is used as an argument to the grant mapping
> APIs). It worth mentioning that toolstack patch is based on non  upstreamed yet “Virtio support for toolstack
> on Arm” series which is on review now [2].
> 
> Please note the following:
> - Patch series only covers Arm and virtio-mmio (device-tree) for now. To enable the restricted memory access
>   feature on Arm the following options should be set:
>   CONFIG_XEN_VIRTIO = y
>   CONFIG_XEN_HVM_VIRTIO_GRANT = y
> - Some callbacks in xen-virtio DMA ops layer (map_sg/unmap_sg, etc) are not implemented yet as they are not
>   needed/used in the first prototype
> 
> Patch series is rebased on Linux 5.18-rc2 tag and tested on Renesas Salvator-X board + H3 ES3.0 SoC (Arm64)
> with standalone userspace (non-Qemu) virtio-mmio based virtio-disk backend running in Driver domain and Linux
> guest running on existing virtio-blk driver (frontend). No issues were observed. Guest domain 'reboot/destroy'
> use-cases work properly. I have also tested other use-cases such as assigning several virtio block devices
> or a mix of virtio and Xen PV block devices to the guest. 
> 
> 1. Xen changes located at (last patch):
> https://github.com/otyshchenko1/xen/commits/libxl_virtio_next
> 2. Linux changes located at:
> https://github.com/otyshchenko1/linux/commits/virtio_grant5
> 3. virtio-disk changes located at:
> https://github.com/otyshchenko1/virtio-disk/commits/virtio_grant
> 
> Any feedback/help would be highly appreciated.
> 
> [1] https://www.youtube.com/watch?v=IrlEdaIUDPk
> [2] https://lore.kernel.org/xen-devel/1649442065-8332-1-git-send-email-olekstysh@gmail.com/
> 
> Juergen Gross (2):
>   xen/grants: support allocating consecutive grants
>   virtio: add option to restrict memory access under Xen
> 
> Oleksandr Tyshchenko (4):
>   dt-bindings: xen: Add xen,dev-domid property description for
>     xen-virtio layer
>   virtio: Various updates to xen-virtio DMA ops layer
>   arm/xen: Introduce xen_setup_dma_ops()
>   arm/xen: Assign xen-virtio DMA ops for virtio devices in Xen guests
> 
>  .../devicetree/bindings/virtio/xen,dev-domid.yaml  |  39 +++
>  arch/arm/include/asm/xen/xen-ops.h                 |   1 +
>  arch/arm/mm/dma-mapping.c                          |   5 +-
>  arch/arm/xen/enlighten.c                           |  11 +
>  arch/arm64/include/asm/xen/xen-ops.h               |   1 +
>  arch/arm64/mm/dma-mapping.c                        |   5 +-
>  arch/x86/mm/init.c                                 |  15 +
>  arch/x86/mm/mem_encrypt.c                          |   5 -
>  arch/x86/xen/Kconfig                               |   9 +
>  drivers/xen/Kconfig                                |  20 ++
>  drivers/xen/Makefile                               |   1 +
>  drivers/xen/grant-table.c                          | 238 +++++++++++++--
>  drivers/xen/xen-virtio.c                           | 335 +++++++++++++++++++++
>  include/xen/arm/xen-ops.h                          |  20 ++
>  include/xen/grant_table.h                          |   4 +
>  include/xen/xen-ops.h                              |  13 +
>  16 files changed, 679 insertions(+), 43 deletions(-)
>  create mode 100644 Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml
>  create mode 100644 arch/arm/include/asm/xen/xen-ops.h
>  create mode 100644 arch/arm64/include/asm/xen/xen-ops.h
>  create mode 100644 drivers/xen/xen-virtio.c
>  create mode 100644 include/xen/arm/xen-ops.h
> 
> -- 
> 2.7.4

_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization

Re: [RFC PATCH 3/6] dt-bindings: xen: Add xen,dev-domid property description for xen-virtio layer

[Stefano Stabellini]

On Thu, 14 Apr 2022, Oleksandr Tyshchenko wrote:
> From: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> 
> Introduce Xen specific binding for the virtio-mmio device to be used
> by Xen virtio support driver in a subsequent commit.
> 
> This binding specifies the ID of Xen domain where the corresponding
> device (backend) resides. This is needed for the option to restrict
> memory access using Xen grant mappings to work.
> 
> Signed-off-by: Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> ---
>  .../devicetree/bindings/virtio/xen,dev-domid.yaml  | 39 ++++++++++++++++++++++
>  1 file changed, 39 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml
> 
> diff --git a/Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml b/Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml
> new file mode 100644
> index 00000000..78be993
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/virtio/xen,dev-domid.yaml
> @@ -0,0 +1,39 @@
> +# SPDX-License-Identifier: (GPL-2.0-only or BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/virtio/xen,dev-domid.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: Xen specific binding for the virtio device
> +
> +maintainers:
> +  - Oleksandr Tyshchenko <oleksandr_tyshchenko@epam.com>
> +
> +select: true
> +
> +description:
> +  This binding specifies the ID of Xen domain where the corresponding device
> +  (backend) resides. This is needed for the option to restrict memory access
> +  using Xen grant mappings to work.
> +
> +  Note that current and generic "iommus" bindings are mutually exclusive, since
> +  the restricted memory access model on Xen behaves as a kind of software IOMMU.

I don't think that this last statement is necessary or fully accurate, so
I would remove it. Other than that, this looks good to me.


> +properties:
> +  xen,dev-domid:
> +    $ref: /schemas/types.yaml#/definitions/uint32
> +    description:
> +      Should contain the ID of device's domain.

Maybe better as:
"The domid (domain ID) of the domain where the device (backend) is running"



> +additionalProperties: true
> +
> +examples:
> +  - |
> +    virtio_block@3000 {
> +            compatible = "virtio,mmio";
> +            reg = <0x3000 0x100>;
> +            interrupts = <41>;
> +
> +            /* The device is located in Xen domain with ID 1 */
> +            xen,dev-domid = <1>;
> +    };
> -- 
> 2.7.4
> 
_______________________________________________
Virtualization mailing list
Virtualization@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/virtualization