[PATCH v1 0/6] GenieZone hypervisor drivers

[PATCH v1 0/6] GenieZone hypervisor drivers

[Yi-De Wu]

This series is based on linux-next, tag: next-20230412.

GenieZone is MediaTek proprietary hypervisor solution, and it is running
in EL2 stand alone as a type-I hypervisor. It is a pure EL2
implementation which implies it does not rely any specific host VM, and
this behavior improves GenieZone's security as it limits its interface.

To enable guest VMs running, a driver (gzvm) is provided for VMM (virtual
machine monitor) to operate. Currently, the gzvm driver supports only
crosvm.

This series supports ioctl interfaces for userspace VMM(eg., crosvm) to
operate guest VMs lifecycle, irqchip for virtual interrupt handling,
asynchronous notifcation mechanism for VMM.

Yi-De Wu (6):
  docs: geniezone: Introduce GenieZone hypervisor
  dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor
  soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support
  soc: mediatek: virt: geniezone: Introduce irqchip for virtual
    interrupt injection
  soc: mediatek: virt: geniezone: Add ioeventfd support
  soc: mediatek: virt: geniezone: Add irqfd support

 .../bindings/hypervisor/mediatek,gzvm.yaml    |  30 +
 Documentation/virt/geniezone/introduction.rst |  34 +
 arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++
 drivers/soc/mediatek/Kconfig                  |   2 +
 drivers/soc/mediatek/Makefile                 |   1 +
 drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
 drivers/soc/mediatek/virt/geniezone/Makefile  |   5 +
 drivers/soc/mediatek/virt/geniezone/gzvm.h    | 126 +++
 .../mediatek/virt/geniezone/gzvm_eventfd.c    | 749 ++++++++++++++++++
 .../soc/mediatek/virt/geniezone/gzvm_hyp.h    |  72 ++
 .../mediatek/virt/geniezone/gzvm_irqchip.c    | 107 +++
 .../soc/mediatek/virt/geniezone/gzvm_main.c   | 233 ++++++
 .../soc/mediatek/virt/geniezone/gzvm_vcpu.c   | 296 +++++++
 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 551 +++++++++++++
 include/uapi/linux/gzvm_common.h              | 291 +++++++
 15 files changed, 2593 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
 create mode 100644 Documentation/virt/geniezone/introduction.rst
 create mode 100644 arch/arm64/include/uapi/asm/gzvm_arch.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/Kconfig
 create mode 100644 drivers/soc/mediatek/virt/geniezone/Makefile
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_main.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
 create mode 100644 include/uapi/linux/gzvm_common.h

-- 
2.18.0

[PATCH v1 1/6] docs: geniezone: Introduce GenieZone hypervisor

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

GenieZone is MediaTek proprietary hypervisor solution, and it is running
in EL2 stand alone as a type-I hypervisor. It is a pure EL2
implementation which implies it does not rely any specific host VM, and
this behavior improves GenieZone's security as it limits its interface.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 Documentation/virt/geniezone/introduction.rst | 34 +++++++++++++++++++
 1 file changed, 34 insertions(+)
 create mode 100644 Documentation/virt/geniezone/introduction.rst

diff --git a/Documentation/virt/geniezone/introduction.rst b/Documentation/virt/geniezone/introduction.rst
new file mode 100644
index 000000000000..1fffd6cbb4db
--- /dev/null
+++ b/Documentation/virt/geniezone/introduction.rst
@@ -0,0 +1,34 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======================
+GenieZone Introduction
+======================
+
+
+Overview
+========
+GenieZone is MediaTek proprietary hypervisor solution, and it is running in EL2
+stand alone as a type-I hypervisor. It is a pure EL2 implementation which
+implies it does not rely any specific host VM, and this behavior improves
+GenieZone's security as it limits its interface.
+
+To enable guest VMs running, a driver (gzvm) is provided for VMM (virtual
+machine monitor) to operate. Currently, the gzvm driver supports only crosvm.
+
+
+Supported Architecture
+======================
+GenieZone now only supports MediaTek arm64 SoC.
+
+
+Platform Virtualization
+=======================
+We leverages arm64's timer virtualization and gic virtualization for timer and
+interrupts controller.
+
+
+Device Virtualizaton
+====================
+We adopts VMM's virtio devices emulations by passing io trap to VMM, and virtio
+is a well-known and widely used virtual device implementation.
+
-- 
2.18.0

[PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

Add documentation for GenieZone(gzvm) node. This node informs gzvm
driver to start probing if geniezone hypervisor is available and
able to do virtual machine operations.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 .../bindings/hypervisor/mediatek,gzvm.yaml    | 30 +++++++++++++++++++
 1 file changed, 30 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml

diff --git a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
new file mode 100644
index 000000000000..35e1e5b18e47
--- /dev/null
+++ b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
@@ -0,0 +1,30 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: MediaTek GenieZone hypervisor
+
+maintainers:
+  - Yingshiuan Pan <yingshiuan.pan@mediatek.com>
+
+description:
+  GenieZone is MediaTek proprietary hypervisor. This device node informs its
+  driver, gzvm, to probe if platform supports running virtual machines.
+
+properties:
+  compatible:
+    const: mediatek,gzvm
+
+required:
+  - compatible
+
+additionalProperties: false
+
+examples:
+  - |
+    hypervisor {
+        compatible = "mediatek,gzvm";
+        status = "okay";
+    };
-- 
2.18.0

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Krzysztof Kozlowski]

On 13/04/2023 11:07, Yi-De Wu wrote:
> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> 
> Add documentation for GenieZone(gzvm) node. This node informs gzvm
> driver to start probing if geniezone hypervisor is available and

Subject: drop second/last, redundant "binding for". The "dt-bindings"
prefix is already stating that these are bindings.

> able to do virtual machine operations.
> 
> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> ---
>  .../bindings/hypervisor/mediatek,gzvm.yaml    | 30 +++++++++++++++++++
>  1 file changed, 30 insertions(+)
>  create mode 100644 Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> 
> diff --git a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> new file mode 100644
> index 000000000000..35e1e5b18e47
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> @@ -0,0 +1,30 @@
> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> +%YAML 1.2
> +---
> +$id: http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml#
> +$schema: http://devicetree.org/meta-schemas/core.yaml#
> +
> +title: MediaTek GenieZone hypervisor
> +
> +maintainers:
> +  - Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> +
> +description:
> +  GenieZone is MediaTek proprietary hypervisor. This device node informs its
> +  driver, gzvm, to probe if platform supports running virtual machines.

Do not describe Linux, we all know how driver binding works, but
hardware/firmware/hypervisor.

I don't know if we actually want to support proprietary hypervisors.
There can be hundreds of them, one per each SoC manufacturer, and they
can come with many ridiculous ideas.

> +
> +properties:
> +  compatible:
> +    const: mediatek,gzvm
> +
> +required:
> +  - compatible
> +
> +additionalProperties: false
> +
> +examples:
> +  - |
> +    hypervisor {
> +        compatible = "mediatek,gzvm";
> +        status = "okay";

Drop status.

Best regards,
Krzysztof

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Yi-De Wu (吳一德)]

On Thu, 2023-04-13 at 15:05 +0200, Krzysztof Kozlowski wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 13/04/2023 11:07, Yi-De Wu wrote:
> > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > 
> > Add documentation for GenieZone(gzvm) node. This node informs gzvm
> > driver to start probing if geniezone hypervisor is available and
> 
> Subject: drop second/last, redundant "binding for". The "dt-bindings"
> prefix is already stating that these are bindings.
> 
Thank you for the review comments. We would remove the "binding for"
wording in the subject on the next version.

> > able to do virtual machine operations.
> > 
> > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > ---
> >  .../bindings/hypervisor/mediatek,gzvm.yaml    | 30
> > +++++++++++++++++++
> >  1 file changed, 30 insertions(+)
> >  create mode 100644
> > Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> > 
> > diff --git
> > a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> > b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> > new file mode 100644
> > index 000000000000..35e1e5b18e47
> > --- /dev/null
> > +++
> > b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> > @@ -0,0 +1,30 @@
> > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > +%YAML 1.2
> > +---
> > +$id: 
> > https://urldefense.com/v3/__http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmA3TPJSag$
> > +$schema: 
> > https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmDSXil_Qw$
> > +
> > +title: MediaTek GenieZone hypervisor
> > +
> > +maintainers:
> > +  - Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > +
> > +description:
> > +  GenieZone is MediaTek proprietary hypervisor. This device node
> > informs its
> > +  driver, gzvm, to probe if platform supports running virtual
> > machines.
> 
> Do not describe Linux, we all know how driver binding works, but
> hardware/firmware/hypervisor.
> 
Noted. We would enhance the description on next version.

> I don't know if we actually want to support proprietary hypervisors.
> There can be hundreds of them, one per each SoC manufacturer, and
> they
> can come with many ridiculous ideas.
> 
MediaTek, as a partner of Android, our GenieZone hypervisor has been
one of the backend options under Android Virtualization Framework(AVF)
now.
Thus, we'd like to donate these patches for better supporting the
Linux/Android ecosystem.

Reference link: https://crosvm.dev/book/hypervisors.html#geniezone 

> > +
> > +properties:
> > +  compatible:
> > +    const: mediatek,gzvm
> > +
> > +required:
> > +  - compatible
> > +
> > +additionalProperties: false
> > +
> > +examples:
> > +  - |
> > +    hypervisor {
> > +        compatible = "mediatek,gzvm";
> > +        status = "okay";
> 
> Drop status.
> 
> Best regards,
> Krzysztof
> 

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Krzysztof Kozlowski]

On 14/04/2023 10:35, Yi-De Wu (吳一德) wrote:
>> I don't know if we actually want to support proprietary hypervisors.
>> There can be hundreds of them, one per each SoC manufacturer, and
>> they
>> can come with many ridiculous ideas.
>>
> MediaTek, as a partner of Android, our GenieZone hypervisor has been
> one of the backend options under Android Virtualization Framework(AVF)
> now.
> Thus, we'd like to donate these patches for better supporting the
> Linux/Android ecosystem.

If it is proprietary, I don't have much interests in it. Make it
open-source. :)

Best regards,
Krzysztof

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Yi-De Wu (吳一德)]

On Fri, 2023-04-14 at 10:42 +0200, Krzysztof Kozlowski wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 14/04/2023 10:35, Yi-De Wu (吳一德) wrote:
> > > I don't know if we actually want to support proprietary
> > > hypervisors.
> > > There can be hundreds of them, one per each SoC manufacturer, and
> > > they
> > > can come with many ridiculous ideas.
> > > 
> > 
> > MediaTek, as a partner of Android, our GenieZone hypervisor has
> > been
> > one of the backend options under Android Virtualization
> > Framework(AVF)
> > now.
> > Thus, we'd like to donate these patches for better supporting the
> > Linux/Android ecosystem.
> 
> If it is proprietary, I don't have much interests in it. Make it
> open-source. :)
> 
> Best regards,
> Krzysztof
> 
I'm sorry it's hard for us to open source right away. We tend to
elaborate more on our design in the documentation and make it more
transparent to the public and reviewers.

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Matthias Brugger]

On 14/04/2023 10:35, Yi-De Wu (吳一德) wrote:
> On Thu, 2023-04-13 at 15:05 +0200, Krzysztof Kozlowski wrote:
>> External email : Please do not click links or open attachments until
>> you have verified the sender or the content.
>>
>>
>> On 13/04/2023 11:07, Yi-De Wu wrote:
>>> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
>>>
>>> Add documentation for GenieZone(gzvm) node. This node informs gzvm
>>> driver to start probing if geniezone hypervisor is available and
>>
>> Subject: drop second/last, redundant "binding for". The "dt-bindings"
>> prefix is already stating that these are bindings.
>>
> Thank you for the review comments. We would remove the "binding for"
> wording in the subject on the next version.
> 
>>> able to do virtual machine operations.
>>>
>>> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>>> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
>>> ---
>>>   .../bindings/hypervisor/mediatek,gzvm.yaml    | 30
>>> +++++++++++++++++++
>>>   1 file changed, 30 insertions(+)
>>>   create mode 100644
>>> Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
>>>
>>> diff --git
>>> a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
>>> b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
>>> new file mode 100644
>>> index 000000000000..35e1e5b18e47
>>> --- /dev/null
>>> +++
>>> b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
>>> @@ -0,0 +1,30 @@
>>> +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
>>> +%YAML 1.2
>>> +---
>>> +$id:
>>> https://urldefense.com/v3/__http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmA3TPJSag$
>>> +$schema:
>>> https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmDSXil_Qw$
>>> +
>>> +title: MediaTek GenieZone hypervisor
>>> +
>>> +maintainers:
>>> +  - Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>>> +
>>> +description:
>>> +  GenieZone is MediaTek proprietary hypervisor. This device node
>>> informs its
>>> +  driver, gzvm, to probe if platform supports running virtual
>>> machines.
>>
>> Do not describe Linux, we all know how driver binding works, but
>> hardware/firmware/hypervisor.
>>
> Noted. We would enhance the description on next version.
> 
>> I don't know if we actually want to support proprietary hypervisors.
>> There can be hundreds of them, one per each SoC manufacturer, and
>> they
>> can come with many ridiculous ideas.
>>
> MediaTek, as a partner of Android, our GenieZone hypervisor has been
> one of the backend options under Android Virtualization Framework(AVF)
> now.
> Thus, we'd like to donate these patches for better supporting the
> Linux/Android ecosystem.
> 
> Reference link: https://crosvm.dev/book/hypervisors.html#geniezone
> 

What is the difference between geniezone and gunyah? Why will we need both of 
them? Couldn't we just get one hypervisor implementation merged that includes 
all the needed features. In the end it will be used with the same VMM.

Regards,
Matthias

>>> +
>>> +properties:
>>> +  compatible:
>>> +    const: mediatek,gzvm
>>> +
>>> +required:
>>> +  - compatible
>>> +
>>> +additionalProperties: false
>>> +
>>> +examples:
>>> +  - |
>>> +    hypervisor {
>>> +        compatible = "mediatek,gzvm";
>>> +        status = "okay";
>>
>> Drop status.
>>
>> Best regards,
>> Krzysztof
>>

Re: [PATCH v1 2/6] dt-bindings: hypervisor: Add binding for MediaTek GenieZone hypervisor

[Yi-De Wu (吳一德)]

On Fri, 2023-04-14 at 17:29 +0200, Matthias Brugger wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 14/04/2023 10:35, Yi-De Wu (吳一德) wrote:
> > On Thu, 2023-04-13 at 15:05 +0200, Krzysztof Kozlowski wrote:
> > > External email : Please do not click links or open attachments
> > > until
> > > you have verified the sender or the content.
> > > 
> > > 
> > > On 13/04/2023 11:07, Yi-De Wu wrote:
> > > > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > > > 
> > > > Add documentation for GenieZone(gzvm) node. This node informs
> > > > gzvm
> > > > driver to start probing if geniezone hypervisor is available
> > > > and
> > > 
> > > Subject: drop second/last, redundant "binding for". The "dt-
> > > bindings"
> > > prefix is already stating that these are bindings.
> > > 
> > 
> > Thank you for the review comments. We would remove the "binding
> > for"
> > wording in the subject on the next version.
> > 
> > > > able to do virtual machine operations.
> > > > 
> > > > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > > > ---
> > > >   .../bindings/hypervisor/mediatek,gzvm.yaml    | 30
> > > > +++++++++++++++++++
> > > >   1 file changed, 30 insertions(+)
> > > >   create mode 100644
> > > > Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.yaml
> > > > 
> > > > diff --git
> > > > a/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.ya
> > > > ml
> > > > b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.ya
> > > > ml
> > > > new file mode 100644
> > > > index 000000000000..35e1e5b18e47
> > > > --- /dev/null
> > > > +++
> > > > b/Documentation/devicetree/bindings/hypervisor/mediatek,gzvm.ya
> > > > ml
> > > > @@ -0,0 +1,30 @@
> > > > +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
> > > > +%YAML 1.2
> > > > +---
> > > > +$id:
> > > > 
https://urldefense.com/v3/__http://devicetree.org/schemas/hypervisor/mediatek,gzvm.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmA3TPJSag$
> > > > +$schema:
> > > > 
https://urldefense.com/v3/__http://devicetree.org/meta-schemas/core.yaml*__;Iw!!CTRNKA9wMg0ARbw!lp4d4WBp26cpOeEGcKn_aHcNGfyl1Y--BYzfR8oR_0Xzr9TLvUELfOQAAdqHlLIOra1W_QdjVSJv0-HMpLesJmDSXil_Qw$
> > > > +
> > > > +title: MediaTek GenieZone hypervisor
> > > > +
> > > > +maintainers:
> > > > +  - Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > > +
> > > > +description:
> > > > +  GenieZone is MediaTek proprietary hypervisor. This device
> > > > node
> > > > informs its
> > > > +  driver, gzvm, to probe if platform supports running virtual
> > > > machines.
> > > 
> > > Do not describe Linux, we all know how driver binding works, but
> > > hardware/firmware/hypervisor.
> > > 
> > 
> > Noted. We would enhance the description on next version.
> > 
> > > I don't know if we actually want to support proprietary
> > > hypervisors.
> > > There can be hundreds of them, one per each SoC manufacturer, and
> > > they
> > > can come with many ridiculous ideas.
> > > 
> > 
> > MediaTek, as a partner of Android, our GenieZone hypervisor has
> > been
> > one of the backend options under Android Virtualization
> > Framework(AVF)
> > now.
> > Thus, we'd like to donate these patches for better supporting the
> > Linux/Android ecosystem.
> > 
> > Reference link: 
> > https://urldefense.com/v3/__https://crosvm.dev/book/hypervisors.html*geniezone__;Iw!!CTRNKA9wMg0ARbw!n1sN15eKnG1M6mlcAb7UBi4hoIYmKsw2zlfdwO21thuHZuHYHHkHnZF9WWfp3xScXVha3mlbLVCUMGIo3OM6D8eI$
> > 
> 
> What is the difference between geniezone and gunyah? Why will we need
> both of
> them? Couldn't we just get one hypervisor implementation merged that
> includes
> all the needed features. In the end it will be used with the same
> VMM.
> 
> Regards,
> Matthias
> 

We might not be able to speak for Gunyah, but Gunyah and GenieZone are
quite different hypervisor implementations in various aspects such like
VM execution flow, semantics, UAPI...etc.

Still, we're open to dicussion if there's any proposal that provide a
unified interface among all the hypervisors.

> > > > +
> > > > +properties:
> > > > +  compatible:
> > > > +    const: mediatek,gzvm
> > > > +
> > > > +required:
> > > > +  - compatible
> > > > +
> > > > +additionalProperties: false
> > > > +
> > > > +examples:
> > > > +  - |
> > > > +    hypervisor {
> > > > +        compatible = "mediatek,gzvm";
> > > > +        status = "okay";
> > > 
> > > Drop status.
> > > 
> > > Best regards,
> > > Krzysztof
> > > 

[PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

GenieZone is MediaTek proprietary hypervisor solution, and it is running
in EL2 stand alone as a type-I hypervisor. This patch exports a set of
ioctl interfaces for userspace VMM (e.g., crosvm) to operate guest VMs
lifecycle (creation, running, and destroy) on GenieZone.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
 drivers/soc/mediatek/Kconfig                  |   2 +
 drivers/soc/mediatek/Makefile                 |   1 +
 drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
 drivers/soc/mediatek/virt/geniezone/Makefile  |   5 +
 drivers/soc/mediatek/virt/geniezone/gzvm.h    | 103 ++++
 .../soc/mediatek/virt/geniezone/gzvm_hyp.h    |  72 +++
 .../soc/mediatek/virt/geniezone/gzvm_main.c   | 233 +++++++++
 .../soc/mediatek/virt/geniezone/gzvm_vcpu.c   | 266 +++++++++++
 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 444 ++++++++++++++++++
 include/uapi/linux/gzvm_common.h              | 217 +++++++++
 11 files changed, 1439 insertions(+)
 create mode 100644 arch/arm64/include/uapi/asm/gzvm_arch.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/Kconfig
 create mode 100644 drivers/soc/mediatek/virt/geniezone/Makefile
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_main.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
 create mode 100644 include/uapi/linux/gzvm_common.h

diff --git a/arch/arm64/include/uapi/asm/gzvm_arch.h b/arch/arm64/include/uapi/asm/gzvm_arch.h
new file mode 100644
index 000000000000..3714f96c832b
--- /dev/null
+++ b/arch/arm64/include/uapi/asm/gzvm_arch.h
@@ -0,0 +1,79 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#ifndef __GZVM_ARCH_H__
+#define __GZVM_ARCH_H__
+
+#include <asm/ptrace.h>
+
+/*
+ * Architecture specific registers are to be defined in arch headers and
+ * ORed with the arch identifier.
+ */
+#define GZVM_REG_ARM		0x4000000000000000ULL
+#define GZVM_REG_ARM64		0x6000000000000000ULL
+
+#define GZVM_REG_SIZE_SHIFT	52
+#define GZVM_REG_SIZE_MASK	0x00f0000000000000ULL
+#define GZVM_REG_SIZE_U8	0x0000000000000000ULL
+#define GZVM_REG_SIZE_U16	0x0010000000000000ULL
+#define GZVM_REG_SIZE_U32	0x0020000000000000ULL
+#define GZVM_REG_SIZE_U64	0x0030000000000000ULL
+#define GZVM_REG_SIZE_U128	0x0040000000000000ULL
+#define GZVM_REG_SIZE_U256	0x0050000000000000ULL
+#define GZVM_REG_SIZE_U512	0x0060000000000000ULL
+#define GZVM_REG_SIZE_U1024	0x0070000000000000ULL
+#define GZVM_REG_SIZE_U2048	0x0080000000000000ULL
+
+#define GZVM_NR_SPSR	5
+struct gzvm_regs {
+	struct user_pt_regs regs;	/* sp = sp_el0 */
+
+	__u64	sp_el1;
+	__u64	elr_el1;
+
+	__u64	spsr[GZVM_NR_SPSR];
+
+	struct user_fpsimd_state fp_regs;
+};
+
+/* If you need to interpret the index values, here is the key: */
+#define GZVM_REG_ARM_COPROC_MASK	0x000000000FFF0000
+#define GZVM_REG_ARM_COPROC_SHIFT	16
+
+/* Normal registers are mapped as coprocessor 16. */
+#define GZVM_REG_ARM_CORE		(0x0010 << GZVM_REG_ARM_COPROC_SHIFT)
+#define GZVM_REG_ARM_CORE_REG(name)	(offsetof(struct gzvm_regs, name) / sizeof(__u32))
+
+/* Some registers need more space to represent values. */
+#define GZVM_REG_ARM_DEMUX		(0x0011 << GZVM_REG_ARM_COPROC_SHIFT)
+#define GZVM_REG_ARM_DEMUX_ID_MASK	0x000000000000FF00
+#define GZVM_REG_ARM_DEMUX_ID_SHIFT	8
+#define GZVM_REG_ARM_DEMUX_ID_CCSIDR	(0x00 << GZVM_REG_ARM_DEMUX_ID_SHIFT)
+#define GZVM_REG_ARM_DEMUX_VAL_MASK	0x00000000000000FF
+#define GZVM_REG_ARM_DEMUX_VAL_SHIFT	0
+
+/* AArch64 system registers */
+#define GZVM_REG_ARM64_SYSREG		(0x0013 << GZVM_REG_ARM_COPROC_SHIFT)
+#define GZVM_REG_ARM64_SYSREG_OP0_MASK	0x000000000000c000
+#define GZVM_REG_ARM64_SYSREG_OP0_SHIFT	14
+#define GZVM_REG_ARM64_SYSREG_OP1_MASK	0x0000000000003800
+#define GZVM_REG_ARM64_SYSREG_OP1_SHIFT	11
+#define GZVM_REG_ARM64_SYSREG_CRN_MASK	0x0000000000000780
+#define GZVM_REG_ARM64_SYSREG_CRN_SHIFT	7
+#define GZVM_REG_ARM64_SYSREG_CRM_MASK	0x0000000000000078
+#define GZVM_REG_ARM64_SYSREG_CRM_SHIFT	3
+#define GZVM_REG_ARM64_SYSREG_OP2_MASK	0x0000000000000007
+#define GZVM_REG_ARM64_SYSREG_OP2_SHIFT	0
+
+/* Physical Timer EL0 Registers */
+#define GZVM_REG_ARM_PTIMER_CTL		ARM64_SYS_REG(3, 3, 14, 2, 1)
+#define GZVM_REG_ARM_PTIMER_CVAL	ARM64_SYS_REG(3, 3, 14, 2, 2)
+#define GZVM_REG_ARM_PTIMER_CNT		ARM64_SYS_REG(3, 3, 14, 0, 1)
+
+/* SVE registers */
+#define GZVM_REG_ARM64_SVE		(0x0015 << KVM_REG_ARM_COPROC_SHIFT)
+
+#endif /* __GZVM_ARCH_H__ */
diff --git a/drivers/soc/mediatek/Kconfig b/drivers/soc/mediatek/Kconfig
index a88cf04fc803..01fad024a1c1 100644
--- a/drivers/soc/mediatek/Kconfig
+++ b/drivers/soc/mediatek/Kconfig
@@ -91,4 +91,6 @@ config MTK_SVS
 	  chip process corner, temperatures and other factors. Then DVFS
 	  driver could apply SVS bank voltage to PMIC/Buck.
 
+source "drivers/soc/mediatek/virt/geniezone/Kconfig"
+
 endmenu
diff --git a/drivers/soc/mediatek/Makefile b/drivers/soc/mediatek/Makefile
index 8c0ddacbcde8..e5d7225c1d08 100644
--- a/drivers/soc/mediatek/Makefile
+++ b/drivers/soc/mediatek/Makefile
@@ -9,3 +9,4 @@ obj-$(CONFIG_MTK_SCPSYS_PM_DOMAINS) += mtk-pm-domains.o
 obj-$(CONFIG_MTK_MMSYS) += mtk-mmsys.o
 obj-$(CONFIG_MTK_MMSYS) += mtk-mutex.o
 obj-$(CONFIG_MTK_SVS) += mtk-svs.o
+obj-$(CONFIG_MTK_GZVM) += virt/geniezone/
diff --git a/drivers/soc/mediatek/virt/geniezone/Kconfig b/drivers/soc/mediatek/virt/geniezone/Kconfig
new file mode 100644
index 000000000000..6fad3c30f8d9
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/Kconfig
@@ -0,0 +1,17 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+config MTK_GZVM
+	tristate "GenieZone Hypervisor driver for guest VM operation"
+	depends on ARM64
+	depends on KVM
+	help
+	  This driver, gzvm, enables to run guest VMs on MTK GenieZone
+	  hypervisor. It exports kvm-like interfaces for VMM (e.g., crosvm) in
+	  order to operate guest VMs on GenieZone hypervisor.
+
+	  GenieZone hypervisor now only supports MediaTek SoC and arm64
+	  architecture.
+
+	  Select M if you want it be built as a module (gzvm.ko).
+
+	  If unsure, say N.
diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile
new file mode 100644
index 000000000000..e1dfbb9c568d
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/Makefile
@@ -0,0 +1,5 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o
+
+obj-$(CONFIG_MTK_GZVM) += gzvm.o
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h
new file mode 100644
index 000000000000..43f215d4b0da
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h
@@ -0,0 +1,103 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#ifndef __GZVM_H__
+#define __GZVM_H__
+
+#include <linux/srcu.h>
+#include <linux/arm-smccc.h>
+#include <linux/gzvm_common.h>
+#include "gzvm_hyp.h"
+
+#define MODULE_NAME	"gzvm"
+#define GZVM_VCPU_MMAP_SIZE  PAGE_SIZE
+#define INVALID_VM_ID   0xffff
+
+/* VM's memory slot descriptor */
+struct gzvm_memslot {
+	u64 base_gfn;			/* begin of guest page frame */
+	unsigned long npages;		/* number of pages this slot covers */
+	unsigned long userspace_addr;	/* corresponding userspace va */
+	struct vm_area_struct *vma;	/* vma related to this userspace addr */
+	u32 flags;
+	u32 slot_id;
+};
+
+/* pre-declaration for circular reference in struct gzvm */
+struct gzvm_vcpu;
+
+struct gzvm {
+	struct gzvm_vcpu *vcpus[GZVM_MAX_VCPUS];
+	struct mm_struct *mm; /* userspace tied to this vm */
+	struct gzvm_memslot memslot[GZVM_MAX_MEM_REGION];
+	struct mutex lock;
+	struct list_head vm_list;
+	struct list_head devices;
+	gzvm_id_t vm_id;
+
+	struct {
+		spinlock_t        lock;
+		struct list_head  items;
+		struct list_head  resampler_list;
+		struct mutex      resampler_lock;
+	} irqfds;
+	struct hlist_head irq_ack_notifier_list;
+	struct srcu_struct irq_srcu;
+	struct mutex irq_lock;
+};
+
+struct gzvm_vcpu {
+	struct gzvm *gzvm;
+	int vcpuid;
+	struct mutex lock;
+	struct gzvm_vcpu_run *run;
+	struct gzvm_vcpu_hwstate *hwstate;
+};
+
+/**
+ * allocate 2 pages for data sharing between driver and gz hypervisor
+ * |- page 0           -|- page 1      -|
+ * |gzvm_vcpu_run|......|hwstate|.......|
+ */
+#define GZVM_VCPU_RUN_MAP_SIZE		(PAGE_SIZE * 2)
+
+long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args);
+
+void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu);
+int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid);
+int gzvm_dev_ioctl_create_vm(unsigned long vm_type);
+
+int gzvm_arm_get_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg);
+int gzvm_arm_set_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg);
+
+int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
+			 unsigned long a3, unsigned long a4, unsigned long a5,
+			 unsigned long a6, unsigned long a7,
+			 struct arm_smccc_res *res);
+
+#define SMC_ENTITY_MTK			59
+#define GZVM_FUNCID_START		(0x1000)
+#define GZVM_HCALL_ID(func)				\
+	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,	\
+			   SMC_ENTITY_MTK, (GZVM_FUNCID_START + (func)))
+
+#define MT_HVC_GZVM_CREATE_VM		GZVM_HCALL_ID(GZVM_FUNC_CREATE_VM)
+#define MT_HVC_GZVM_DESTROY_VM		GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VM)
+#define MT_HVC_GZVM_CREATE_VCPU		GZVM_HCALL_ID(GZVM_FUNC_CREATE_VCPU)
+#define MT_HVC_GZVM_DESTROY_VCPU	GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VCPU)
+#define MT_HVC_GZVM_SET_MEMREGION	GZVM_HCALL_ID(GZVM_FUNC_SET_MEMREGION)
+#define MT_HVC_GZVM_RUN			GZVM_HCALL_ID(GZVM_FUNC_RUN)
+#define MT_HVC_GZVM_GET_REGS		GZVM_HCALL_ID(GZVM_FUNC_GET_REGS)
+#define MT_HVC_GZVM_SET_REGS		GZVM_HCALL_ID(GZVM_FUNC_SET_REGS)
+#define MT_HVC_GZVM_GET_ONE_REG		GZVM_HCALL_ID(GZVM_FUNC_GET_ONE_REG)
+#define MT_HVC_GZVM_SET_ONE_REG		GZVM_HCALL_ID(GZVM_FUNC_SET_ONE_REG)
+#define MT_HVC_GZVM_IRQ_LINE		GZVM_HCALL_ID(GZVM_FUNC_IRQ_LINE)
+#define MT_HVC_GZVM_CREATE_DEVICE	GZVM_HCALL_ID(GZVM_FUNC_CREATE_DEVICE)
+#define MT_HVC_GZVM_PROBE		GZVM_HCALL_ID(GZVM_FUNC_PROBE)
+#define MT_HVC_GZVM_ENABLE_CAP		GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP)
+
+int gz_err_to_errno(unsigned long err);
+
+#endif /* __GZVM_H__ */
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h b/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h
new file mode 100644
index 000000000000..17dd3de5c285
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h
@@ -0,0 +1,72 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#ifndef __GZ_ERR_H__
+#define __GZ_ERR_H__
+
+/**
+ * @brief Definitions of APIs between GenieZone hypervisor and driver
+ *
+ * These are not needed to be visible to uapi
+ */
+
+/* We need GenieZone specific error code in order to map to Linux errno */
+#define NO_ERROR                (0)
+#define ERR_NO_MEMORY           (-5)
+#define ERR_NOT_SUPPORTED       (-24)
+#define ERR_NOT_IMPLEMENTED     (-27)
+#define ERR_FAULT               (-40)
+
+static inline unsigned int
+assemble_vm_vcpu_tuple(gzvm_id_t vmid, gzvm_vcpu_id_t vcpuid)
+{
+	return ((unsigned int)vmid << 16 | vcpuid);
+}
+
+static inline gzvm_id_t get_vmid_from_tuple(unsigned int tuple)
+{
+	return (gzvm_id_t)(tuple >> 16);
+}
+
+static inline gzvm_vcpu_id_t get_vcpuid_from_tuple(unsigned int tuple)
+{
+	return (gzvm_vcpu_id_t) (tuple & 0xffff);
+}
+
+static inline void
+disassemble_vm_vcpu_tuple(unsigned int tuple, gzvm_id_t *vmid,
+			  gzvm_vcpu_id_t *vcpuid)
+{
+	*vmid = get_vmid_from_tuple(tuple);
+	*vcpuid = get_vcpuid_from_tuple(tuple);
+}
+
+/*
+ * The following data structures are for data transferring between driver and
+ * hypervisor
+ */
+
+/* align hypervisor definitions */
+#define GZVM_MAX_VCPUS		 8
+#define GZVM_MAX_MEM_REGION	10
+
+/* identical to ffa memory constituent */
+struct mem_region_addr_range {
+	/* The base IPA of the constituent memory region, aligned to 4 kiB */
+	__u64 address;
+	/* The number of 4 kiB pages in the constituent memory region. */
+	__u32 pg_cnt;
+	__u32 reserved;
+};
+
+struct gzvm_memory_region_ranges {
+	__u32 slot;
+	__u32 constituent_cnt;
+	__u64 total_pages;
+	__u64 gpa;
+	struct mem_region_addr_range constituents[];
+};
+
+#endif /* __GZ_ERR_H__ */
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_main.c b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c
new file mode 100644
index 000000000000..1fabe4a579da
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c
@@ -0,0 +1,233 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/arm-smccc.h>
+#include <linux/device.h>
+#include <linux/file.h>
+#include <linux/kdev_t.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/slab.h>
+#include "gzvm.h"
+
+static void (*invoke_gzvm_fn)(unsigned long, unsigned long, unsigned long,
+			      unsigned long, unsigned long, unsigned long,
+			      unsigned long, unsigned long,
+			      struct arm_smccc_res *);
+
+static void gzvm_hvc(unsigned long a0, unsigned long a1, unsigned long a2,
+		      unsigned long a3, unsigned long a4, unsigned long a5,
+		      unsigned long a6, unsigned long a7,
+		      struct arm_smccc_res *res)
+{
+	arm_smccc_hvc(a0, a1, a2, a3, a4, a5, a6, a7, res);
+}
+
+static void gzvm_smc(unsigned long a0, unsigned long a1, unsigned long a2,
+		      unsigned long a3, unsigned long a4, unsigned long a5,
+		      unsigned long a6, unsigned long a7,
+		      struct arm_smccc_res *res)
+{
+	arm_smccc_smc(a0, a1, a2, a3, a4, a5, a6, a7, res);
+}
+
+static int gzvm_probe_conduit(void)
+{
+	struct arm_smccc_res res;
+
+	arm_smccc_hvc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res);
+	if (res.a0 == 0) {
+		invoke_gzvm_fn = gzvm_hvc;
+		return 0;
+	}
+
+	arm_smccc_smc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res);
+	if (res.a0 == 0) {
+		invoke_gzvm_fn = gzvm_smc;
+		return 0;
+	}
+
+	return -ENXIO;
+}
+
+/**
+ * @brief geniezone hypercall wrapper
+ * @return int geniezone's return value will be converted to Linux errno
+ */
+int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
+			 unsigned long a3, unsigned long a4, unsigned long a5,
+			 unsigned long a6, unsigned long a7,
+			 struct arm_smccc_res *res)
+{
+	invoke_gzvm_fn(a0, a1, a2, a3, a4, a5, a6, a7, res);
+	return gz_err_to_errno(res->a0);
+}
+
+/**
+ * @brief Convert geniezone return value to standard errno
+ *
+ * @param err return value from geniezone hypercall (a0)
+ * @return int errno
+ */
+int gz_err_to_errno(unsigned long err)
+{
+	int gz_err = (int) err;
+
+	switch (gz_err) {
+	case 0:
+		return 0;
+	case ERR_NO_MEMORY:
+		return -ENOMEM;
+	case ERR_NOT_SUPPORTED:
+		return -EOPNOTSUPP;
+	case ERR_NOT_IMPLEMENTED:
+		return -EOPNOTSUPP;
+	case ERR_FAULT:
+		return -EFAULT;
+	default:
+		return -EINVAL;
+	}
+
+	return -EINVAL;
+}
+
+static int gzvm_cap_arm_vm_ipa_size(void __user *argp)
+{
+	u64 value = CONFIG_ARM64_PA_BITS;
+
+	if (copy_to_user(argp, &value, sizeof(u64)))
+		return -EFAULT;
+
+	return 0;
+}
+
+/**
+ * @brief Check if given capability is support or not
+ *
+ * @param args in/out u64 pointer from userspace
+ * @retval 0: support, no error
+ * @retval -EOPNOTSUPP: not support
+ * @retval -EFAULT: failed to get data from userspace
+ */
+long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args)
+{
+	int ret = -EOPNOTSUPP;
+	__u64 cap, success = 1;
+	void __user *argp = (void __user *) args;
+
+	if (copy_from_user(&cap, argp, sizeof(uint64_t)))
+		return -EFAULT;
+
+	switch (cap) {
+	case GZVM_CAP_ARM_PROTECTED_VM:
+		if (copy_to_user(argp, &success, sizeof(uint64_t)))
+			return -EFAULT;
+		ret = 0;
+		break;
+	case GZVM_CAP_ARM_VM_IPA_SIZE:
+		ret = gzvm_cap_arm_vm_ipa_size(argp);
+		break;
+	default:
+		ret = -EOPNOTSUPP;
+	}
+
+	return ret;
+}
+
+static long gzvm_dev_ioctl(struct file *filp, unsigned int cmd,
+			    unsigned long user_args)
+{
+	long ret = -ENOTTY;
+
+	switch (cmd) {
+	case GZVM_CREATE_VM:
+		ret = gzvm_dev_ioctl_create_vm(user_args);
+		break;
+	case GZVM_CHECK_EXTENSION:
+		if (!user_args)
+			return -EINVAL;
+		ret = gzvm_dev_ioctl_check_extension(NULL, user_args);
+		break;
+	default:
+		ret = -ENOTTY;
+	}
+
+	return ret;
+}
+
+static const struct file_operations gzvm_chardev_ops = {
+	.unlocked_ioctl = gzvm_dev_ioctl,
+	.llseek		= noop_llseek,
+};
+
+static struct miscdevice gzvm_dev = {
+	.minor = MISC_DYNAMIC_MINOR,
+	.name = MODULE_NAME,
+	.fops = &gzvm_chardev_ops,
+};
+
+static int gzvm_drv_probe(struct platform_device *pdev)
+{
+	if (!of_device_is_available(dev_of_node(&pdev->dev))) {
+		dev_info(&pdev->dev, "GenieZone hypervisor is not available\n");
+		return -ENODEV;
+	}
+
+	if (gzvm_probe_conduit() != 0) {
+		dev_err(&pdev->dev, "Not found available conduit\n");
+		return -ENODEV;
+	}
+
+	return 0;
+}
+
+static int gzvm_drv_remove(struct platform_device *pdev)
+{
+	misc_deregister(&gzvm_dev);
+
+	return 0;
+}
+
+static const struct of_device_id gzvm_of_match[] = {
+	{ .compatible = "mediatek,gzvm", },
+	{},
+};
+MODULE_DEVICE_TABLE(of, gzvm_of_match);
+
+static struct platform_driver gzvm_driver = {
+	.probe = gzvm_drv_probe,
+	.remove = gzvm_drv_remove,
+	.driver = {
+			.name = MODULE_NAME,
+			.owner = THIS_MODULE,
+			.of_match_table = gzvm_of_match,
+	},
+};
+
+static int __init gzvm_init(void)
+{
+	int ret = 0;
+
+	ret = platform_driver_register(&gzvm_driver);
+	if (ret)
+		pr_err("Failed to register gzvm driver.\n");
+
+	return ret;
+}
+
+static void __exit gzvm_exit(void)
+{
+	platform_driver_unregister(&gzvm_driver);
+}
+
+module_init(gzvm_init);
+module_exit(gzvm_exit);
+
+MODULE_AUTHOR("MediaTek");
+MODULE_DESCRIPTION("GenieZone interface for VMM");
+MODULE_LICENSE("GPL");
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
new file mode 100644
index 000000000000..726db866dfcf
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
@@ -0,0 +1,266 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#include <asm/sysreg.h>
+#include <linux/anon_inodes.h>
+#include <linux/file.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include "gzvm.h"
+
+static int gzvm_vcpu_update_one_reg_hyp(struct gzvm_vcpu *vcpu, __u64 reg_id,
+					bool is_write, __u64 *data)
+{
+	struct arm_smccc_res res;
+	unsigned long a1;
+	int ret;
+
+	a1 = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid);
+	if (!is_write) {
+		ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_GET_ONE_REG,
+					   a1, reg_id, 0, 0, 0, 0, 0, &res);
+		if (ret == 0)
+			*data = res.a1;
+	} else {
+		ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_ONE_REG,
+					   a1, reg_id, *data, 0, 0, 0, 0, &res);
+	}
+
+	return ret;
+}
+
+static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *vcpu, void * __user argp,
+				     bool is_write)
+{
+	long ret;
+	__u64 reg_size, data = 0;
+	struct gzvm_one_reg reg;
+	void __user *reg_addr;
+
+	if (copy_from_user(&reg, argp, sizeof(reg)))
+		return -EFAULT;
+	reg_addr = (void __user *)reg.addr;
+
+	/* reg id follows KVM's encoding */
+	switch (reg.id & GZVM_REG_ARM_COPROC_MASK) {
+	case GZVM_REG_ARM_CORE:
+		break;
+	default:
+		return -EOPNOTSUPP;
+	}
+
+	reg_size = 1 << ((reg.id & GZVM_REG_SIZE_MASK) >> GZVM_REG_SIZE_SHIFT);
+	if (is_write) {
+		if (copy_from_user(&data, reg_addr, reg_size))
+			return -EFAULT;
+	}
+
+	ret = gzvm_vcpu_update_one_reg_hyp(vcpu, reg.id, is_write, &data);
+
+	if (!is_write && ret == 0) {
+		if (copy_to_user(reg_addr, &data, reg_size))
+			return -EFAULT;
+	}
+
+	return ret;
+}
+
+/**
+ * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest
+ *
+ * @param filp
+ * @param argp pointer to struct gzvm_vcpu_run in userspace
+ * @return long
+ */
+static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp)
+{
+	unsigned long id_tuple;
+	struct arm_smccc_res res;
+	bool need_userspace = false;
+
+	if (copy_from_user(vcpu->run, argp, sizeof(struct gzvm_vcpu_run)))
+		return -EFAULT;
+
+	if (vcpu->run->immediate_exit == 1)
+		return -EINTR;
+
+	id_tuple = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid);
+	do {
+		gzvm_hypcall_wrapper(MT_HVC_GZVM_RUN, id_tuple, 0, 0, 0, 0, 0,
+				     0, &res);
+		switch (res.a1) {
+		case GZVM_EXIT_MMIO:
+			need_userspace = true;
+			break;
+		/*
+		 * geniezone's responsibility to fill corresponding data
+		 * structure
+		 */
+		case GZVM_EXIT_HVC:
+		case GZVM_EXIT_EXCEPTION:
+		case GZVM_EXIT_DEBUG:
+		case GZVM_EXIT_FAIL_ENTRY:
+		case GZVM_EXIT_INTERNAL_ERROR:
+		case GZVM_EXIT_SYSTEM_EVENT:
+		case GZVM_EXIT_SHUTDOWN:
+			need_userspace = true;
+			break;
+		case GZVM_EXIT_IRQ:
+			break;
+		case GZVM_EXIT_UNKNOWN:
+		default:
+			pr_err("vcpu unknown exit\n");
+			need_userspace = true;
+			goto out;
+		}
+	} while (!need_userspace);
+
+out:
+	if (copy_to_user(argp, vcpu->run, sizeof(struct gzvm_vcpu_run)))
+		return -EFAULT;
+	return 0;
+}
+
+static long gzvm_vcpu_ioctl(struct file *filp, unsigned int ioctl,
+			    unsigned long arg)
+{
+	int ret = -ENOTTY;
+	void __user *argp = (void __user *)arg;
+	struct gzvm_vcpu *vcpu = filp->private_data;
+
+	switch (ioctl) {
+	case GZVM_RUN:
+		ret = gzvm_vcpu_run(vcpu, argp);
+		break;
+	case GZVM_GET_ONE_REG:
+		ret = gzvm_vcpu_update_one_reg(vcpu, argp, false /*is_write*/);
+		break;
+	case GZVM_SET_ONE_REG:
+		ret = gzvm_vcpu_update_one_reg(vcpu, argp, true  /*is_write*/);
+		break;
+	default:
+		ret = -ENOTTY;
+		break;
+	}
+
+	return ret;
+}
+
+static const struct file_operations gzvm_vcpu_fops = {
+	.unlocked_ioctl = gzvm_vcpu_ioctl,
+	.llseek		= noop_llseek,
+};
+
+static int gzvm_destroy_vcpu_hyp(gzvm_id_t vm_id, int vcpuid)
+{
+	struct arm_smccc_res res;
+	unsigned long a1;
+
+	a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid);
+	gzvm_hypcall_wrapper(MT_HVC_GZVM_DESTROY_VCPU, a1, 0, 0, 0, 0, 0, 0,
+			     &res);
+
+	return 0;
+}
+
+/**
+ * @brief call smc to gz hypervisor to create vcpu
+ *
+ * @param run virtual address of vcpu->run
+ * @return int
+ */
+static int gzvm_create_vcpu_hyp(gzvm_id_t vm_id, int vcpuid, void *run)
+{
+	struct arm_smccc_res res;
+	unsigned long a1, a2;
+	int ret;
+
+	a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid);
+	a2 = (__u64)virt_to_phys(run);
+	ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VCPU, a1, a2, 0, 0, 0, 0,
+				   0, &res);
+
+	return ret;
+}
+
+/* Caller must hold the vm lock */
+void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu)
+{
+	if (!vcpu)
+		return;
+
+	gzvm_destroy_vcpu_hyp(vcpu->gzvm->vm_id, vcpu->vcpuid);
+	/* clean guest's data */
+	memset(vcpu->run, 0, GZVM_VCPU_RUN_MAP_SIZE);
+	free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE);
+	kfree(vcpu);
+}
+
+#define ITOA_MAX_LEN 12 /* Maximum size needed for holding an integer. */
+/**
+ * @brief Allocates an inode for the vcpu.
+ */
+static int create_vcpu_fd(struct gzvm_vcpu *vcpu)
+{
+	/* sizeof("gzvm-vcpu:") + max(strlen(itoa(vcpuid))) + null */
+	char name[10 + ITOA_MAX_LEN + 1];
+
+	snprintf(name, sizeof(name), "gzvm-vcpu:%d", vcpu->vcpuid);
+	return anon_inode_getfd(name, &gzvm_vcpu_fops, vcpu, O_RDWR | O_CLOEXEC);
+}
+
+/**
+ * @brief GZVM_CREATE_VCPU
+ *
+ * @param cpuid = arg
+ * @return fd of vcpu, negative errno if error occurs
+ */
+int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid)
+{
+	struct gzvm_vcpu *vcpu;
+	int ret;
+
+	if (cpuid >= GZVM_MAX_VCPUS)
+		return -EINVAL;
+
+	vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL);
+	if (!vcpu)
+		return -ENOMEM;
+
+	BUILD_BUG_ON((sizeof(*vcpu->run)) > PAGE_SIZE);
+	BUILD_BUG_ON(sizeof(struct gzvm_vcpu_hwstate) > PAGE_SIZE);
+	/**
+	 * allocate 2 pages for data sharing between driver and gz hypervisor
+	 * |- page 0           -|- page 1      -|
+	 * |gzvm_vcpu_run|......|hwstate|.......|
+	 */
+	vcpu->run = alloc_pages_exact(GZVM_VCPU_RUN_MAP_SIZE,
+				      GFP_KERNEL_ACCOUNT | __GFP_ZERO);
+	if (!vcpu->run) {
+		ret = -ENOMEM;
+		goto free_vcpu;
+	}
+	vcpu->hwstate = (void *)vcpu->run + PAGE_SIZE;
+	vcpu->vcpuid = cpuid;
+	vcpu->gzvm = gzvm;
+	mutex_init(&vcpu->lock);
+
+	ret = gzvm_create_vcpu_hyp(gzvm->vm_id, vcpu->vcpuid, vcpu->run);
+	if (ret < 0)
+		goto free_vcpu_run;
+
+	ret = create_vcpu_fd(vcpu);
+	if (ret < 0)
+		goto free_vcpu_run;
+	gzvm->vcpus[cpuid] = vcpu;
+
+	return ret;
+
+free_vcpu_run:
+	free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE);
+free_vcpu:
+	kfree(vcpu);
+	return ret;
+}
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
new file mode 100644
index 000000000000..df4ccdc3b7f0
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
@@ -0,0 +1,444 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#include <linux/anon_inodes.h>
+#include <linux/arm-smccc.h>
+#include <linux/file.h>
+#include <linux/kdev_t.h>
+#include <linux/kvm_host.h>
+#include <linux/miscdevice.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/version.h>
+#include "gzvm.h"
+
+static DEFINE_MUTEX(gzvm_list_lock);
+static LIST_HEAD(gzvm_list);
+
+
+/**
+ * @brief Translate gfn (guest ipa) to pfn (host pa), result is in @pfn
+ *
+ * Leverage KVM's `gfn_to_pfn_memslot`. Because `gfn_to_pfn_memslot` needs
+ * kvm_memory_slot as parameter, this function populates necessary fileds
+ * for calling `gfn_to_pfn_memslot`.
+ *
+ * @retval 0 succeed
+ * @retval -EFAULT failed to convert
+ */
+int gzvm_gfn_to_pfn_memslot(struct gzvm_memslot *memslot, u64 gfn, u64 *pfn)
+{
+	hfn_t __pfn;
+	struct kvm_memory_slot kvm_slot = {0};
+
+	kvm_slot.base_gfn = memslot->base_gfn;
+	kvm_slot.npages = memslot->npages;
+	kvm_slot.dirty_bitmap = NULL;
+	kvm_slot.userspace_addr = memslot->userspace_addr;
+	kvm_slot.flags = memslot->flags;
+	kvm_slot.id = memslot->slot_id;
+	kvm_slot.as_id = 0;
+
+	__pfn = gfn_to_pfn_memslot(&kvm_slot, gfn);
+	if (is_error_noslot_pfn(__pfn)) {
+		*pfn = 0;
+		return -EFAULT;
+	}
+
+	*pfn = __pfn;
+	return 0;
+}
+
+/**
+ * @brief Populate pa to buffer until full
+ *
+ * @return int how much pages we've fill in, negative if error
+ */
+static int fill_constituents(struct mem_region_addr_range *consti,
+			     int *consti_cnt, int max_nr_consti, gfn_t gfn,
+			     u32 total_pages, struct gzvm_memslot *slot)
+{
+	int i, nr_pages;
+	hfn_t pfn, prev_pfn;
+	gfn_t gfn_end;
+
+	if (unlikely(total_pages == 0))
+		return -EINVAL;
+	gfn_end = gfn + total_pages;
+
+	/* entry 0 */
+	if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0)
+		return -EFAULT;
+	consti[0].address = PFN_PHYS(pfn);
+	consti[0].pg_cnt = 1;
+	gfn++;
+	prev_pfn = pfn;
+	i = 0;
+	nr_pages = 1;
+	while (i < max_nr_consti && gfn < gfn_end) {
+		if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0)
+			return -EFAULT;
+		if (pfn == (prev_pfn + 1)) {
+			consti[i].pg_cnt++;
+		} else {
+			i++;
+			if (i >= max_nr_consti)
+				break;
+			consti[i].address = PFN_PHYS(pfn);
+			consti[i].pg_cnt = 1;
+		}
+		prev_pfn = pfn;
+		gfn++;
+		nr_pages++;
+	}
+	if (i == max_nr_consti)
+		*consti_cnt = i;
+	else
+		*consti_cnt = (i + 1);
+
+	return nr_pages;
+}
+
+/**
+ * @brief Register memory region to GZ
+ *
+ * @param gzvm
+ * @param memslot
+ * @return int
+ */
+static int
+register_memslot_addr_range(struct gzvm *gzvm, struct gzvm_memslot *memslot)
+{
+	struct gzvm_memory_region_ranges *region;
+	u32 buf_size;
+	int max_nr_consti, remain_pages;
+	gfn_t gfn, gfn_end;
+
+	buf_size = PAGE_SIZE * 2;
+	region = alloc_pages_exact(buf_size, GFP_KERNEL);
+	if (!region)
+		return -ENOMEM;
+	max_nr_consti = (buf_size - sizeof(*region)) /
+			sizeof(struct mem_region_addr_range);
+
+	region->slot = memslot->slot_id;
+	remain_pages = memslot->npages;
+	gfn = memslot->base_gfn;
+	gfn_end = gfn + remain_pages;
+	while (gfn < gfn_end) {
+		struct arm_smccc_res res;
+		int nr_pages;
+
+		nr_pages = fill_constituents(region->constituents,
+					     &region->constituent_cnt,
+					     max_nr_consti, gfn,
+					     remain_pages, memslot);
+		region->gpa = PFN_PHYS(gfn);
+		region->total_pages = nr_pages;
+
+		remain_pages -= nr_pages;
+		gfn += nr_pages;
+
+		gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_MEMREGION, gzvm->vm_id,
+			     buf_size, virt_to_phys(region), 0, 0, 0, 0, &res);
+
+		if (res.a0 != 0) {
+			pr_err("Failed to register memregion to hypervisor\n");
+			free_pages_exact(region, buf_size);
+			return -EFAULT;
+		}
+	}
+	free_pages_exact(region, buf_size);
+	return 0;
+}
+
+/**
+ * @brief Set memory region of guest
+ *
+ * @param gzvm struct gzvm
+ * @param mem struct gzvm_userspace_memory_region: input from user
+ * @retval -EXIO memslot is out-of-range
+ * @retval -EFAULT  cannot find corresponding vma
+ * @retval -EINVAL  region size and vma size does not match
+ */
+static int gzvm_vm_ioctl_set_memory_region(struct gzvm *gzvm,
+				struct gzvm_userspace_memory_region *mem)
+{
+	struct vm_area_struct *vma;
+	struct gzvm_memslot *memslot;
+	unsigned long size;
+	__u32 slot;
+
+	slot = mem->slot;
+	if (slot >= GZVM_MAX_MEM_REGION)
+		return -ENXIO;
+	memslot = &gzvm->memslot[slot];
+
+	vma = vma_lookup(gzvm->mm, mem->userspace_addr);
+	if (!vma)
+		return -EFAULT;
+
+	size = vma->vm_end - vma->vm_start;
+	if (size != mem->memory_size)
+		return -EINVAL;
+
+	memslot->base_gfn = __phys_to_pfn(mem->guest_phys_addr);
+	memslot->npages = size >> PAGE_SHIFT;
+	memslot->userspace_addr = mem->userspace_addr;
+	memslot->vma = vma;
+	memslot->flags = mem->flags;
+	memslot->slot_id = mem->slot;
+	return register_memslot_addr_range(gzvm, memslot);
+}
+
+static int gzvm_vm_enable_cap_hyp(struct gzvm *gzvm,
+				  struct gzvm_enable_cap *cap,
+				  struct arm_smccc_res *res)
+{
+	int ret;
+
+	ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_ENABLE_CAP, gzvm->vm_id,
+				   cap->cap, cap->args[0], cap->args[1],
+				   cap->args[2], cap->args[3], cap->args[4],
+				   res);
+	return ret;
+}
+
+/**
+ * @brief Get pvmfw size from hypervisor, return in x1, and return to userspace
+ *        in args[1].
+ * @retval 0 succeed
+ * @retval -EINVAL hypervisor return invalid results
+ * @retval -EFAULT fail to copy back to userspace buffer
+ */
+static int gzvm_vm_ioctl_get_pvmfw_size(struct gzvm *gzvm,
+					struct gzvm_enable_cap *cap,
+					void __user *argp)
+{
+	struct arm_smccc_res res = {0};
+
+	if (gzvm_vm_enable_cap_hyp(gzvm, cap, &res) != 0)
+		return -EINVAL;
+
+	cap->args[1] = res.a1;
+	if (copy_to_user(argp, cap, sizeof(*cap)))
+		return -EFAULT;
+
+	return 0;
+}
+
+/**
+ * @brief Proceed GZVM_CAP_ARM_PROTECTED_VM's subcommands
+ * @retval 0 succeed
+ * @retval -EINVAL invalid subcommand or arguments
+ */
+static int gzvm_vm_ioctl_cap_pvm(struct gzvm *gzvm, struct gzvm_enable_cap *cap,
+				 void __user *argp)
+{
+	int ret = -EINVAL;
+	struct arm_smccc_res res = {0};
+
+	switch (cap->args[0]) {
+	case GZVM_CAP_ARM_PVM_SET_PVMFW_IPA:
+		ret = gzvm_vm_enable_cap_hyp(gzvm, cap, &res);
+		break;
+	case GZVM_CAP_ARM_PVM_GET_PVMFW_SIZE:
+		ret = gzvm_vm_ioctl_get_pvmfw_size(gzvm, cap, argp);
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+static int gzvm_vm_ioctl_enable_cap(struct gzvm *gzvm,
+				    struct gzvm_enable_cap *cap,
+				    void __user *argp)
+{
+	int ret = -EINVAL;
+
+	switch (cap->cap) {
+	case GZVM_CAP_ARM_PROTECTED_VM:
+		ret = gzvm_vm_ioctl_cap_pvm(gzvm, cap, argp);
+		break;
+	default:
+		ret = -EINVAL;
+		break;
+	}
+
+	return ret;
+}
+
+/**
+ * @brief ioctl handler of VM FD
+ */
+static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl,
+			  unsigned long arg)
+{
+	long ret = -ENOTTY;
+	void __user *argp = (void __user *)arg;
+	struct gzvm *gzvm = filp->private_data;
+
+	switch (ioctl) {
+	case GZVM_CHECK_EXTENSION:
+		ret = gzvm_dev_ioctl_check_extension(gzvm, arg);
+		break;
+	case GZVM_CREATE_VCPU:
+		ret = gzvm_vm_ioctl_create_vcpu(gzvm, arg);
+		break;
+	case GZVM_SET_USER_MEMORY_REGION: {
+		struct gzvm_userspace_memory_region userspace_mem;
+
+		ret = -EFAULT;
+		if (copy_from_user(&userspace_mem, argp,
+						sizeof(userspace_mem)))
+			goto out;
+		ret = gzvm_vm_ioctl_set_memory_region(gzvm, &userspace_mem);
+		break;
+	}
+	case GZVM_ENABLE_CAP: {
+		struct gzvm_enable_cap cap;
+
+		ret = -EFAULT;
+		if (copy_from_user(&cap, argp, sizeof(cap)))
+			goto out;
+
+		ret = gzvm_vm_ioctl_enable_cap(gzvm, &cap, argp);
+		break;
+	}
+	default:
+		ret = -ENOTTY;
+	}
+out:
+	return ret;
+}
+
+/**
+ * @brief Destroy all vcpus
+ *
+ * @param gzvm vm struct that owns the vcpus
+ * Caller has to hold the vm lock
+ */
+static void gzvm_destroy_vcpus(struct gzvm *gzvm)
+{
+	int i;
+
+	for (i = 0; i < GZVM_MAX_VCPUS; i++) {
+		gzvm_destroy_vcpu(gzvm->vcpus[i]);
+		gzvm->vcpus[i] = NULL;
+	}
+}
+
+static int gzvm_destroy_vm_hyp(gzvm_id_t vm_id)
+{
+	struct arm_smccc_res res;
+
+	gzvm_hypcall_wrapper(MT_HVC_GZVM_DESTROY_VM, vm_id, 0, 0, 0, 0, 0, 0,
+			     &res);
+
+	return 0;
+}
+
+static void gzvm_destroy_vm(struct gzvm *gzvm)
+{
+	pr_info("VM-%u is going to be destroyed\n", gzvm->vm_id);
+
+	mutex_lock(&gzvm->lock);
+
+	gzvm_destroy_vcpus(gzvm);
+	gzvm_destroy_vm_hyp(gzvm->vm_id);
+
+	mutex_lock(&gzvm_list_lock);
+	list_del(&gzvm->vm_list);
+	mutex_unlock(&gzvm_list_lock);
+
+	mutex_unlock(&gzvm->lock);
+
+	kfree(gzvm);
+}
+
+static int gzvm_vm_release(struct inode *inode, struct file *filp)
+{
+	struct gzvm *gzvm = filp->private_data;
+
+	gzvm_destroy_vm(gzvm);
+	return 0;
+}
+
+static const struct file_operations gzvm_vm_fops = {
+	.release        = gzvm_vm_release,
+	.unlocked_ioctl = gzvm_vm_ioctl,
+	.llseek		= noop_llseek,
+};
+
+static int gzvm_create_vm_hyp(void)
+{
+	struct arm_smccc_res res;
+
+	gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VM, 0, 0, 0, 0, 0, 0, 0, &res);
+
+	if (res.a0 != 0)
+		return -EFAULT;
+	return res.a1;
+}
+
+static struct gzvm *gzvm_create_vm(unsigned long vm_type)
+{
+	int ret;
+	struct gzvm *gzvm;
+
+	gzvm = kzalloc(sizeof(struct gzvm), GFP_KERNEL);
+	if (!gzvm)
+		return ERR_PTR(-ENOMEM);
+
+	ret = gzvm_create_vm_hyp();
+	if (ret < 0)
+		goto err;
+
+	gzvm->vm_id = ret;
+	gzvm->mm = current->mm;
+	mutex_init(&gzvm->lock);
+	INIT_LIST_HEAD(&gzvm->devices);
+	mutex_init(&gzvm->irq_lock);
+	pr_info("VM-%u is created\n", gzvm->vm_id);
+
+	mutex_lock(&gzvm_list_lock);
+	list_add(&gzvm->vm_list, &gzvm_list);
+	mutex_unlock(&gzvm_list_lock);
+
+	return gzvm;
+
+err:
+	kfree(gzvm);
+	return ERR_PTR(ret);
+}
+
+/**
+ * @brief create vm fd
+ *
+ * @param vm_type
+ * @return int fd of vm, negative if error
+ */
+int gzvm_dev_ioctl_create_vm(unsigned long vm_type)
+{
+	struct gzvm *gzvm;
+	int ret;
+
+	gzvm = gzvm_create_vm(vm_type);
+	if (IS_ERR(gzvm)) {
+		ret = PTR_ERR(gzvm);
+		goto error;
+	}
+
+	ret = anon_inode_getfd("gzvm-vm", &gzvm_vm_fops, gzvm,
+			       O_RDWR | O_CLOEXEC);
+	if (ret < 0)
+		goto error;
+
+error:
+	return ret;
+}
diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h
new file mode 100644
index 000000000000..aa97438bab8c
--- /dev/null
+++ b/include/uapi/linux/gzvm_common.h
@@ -0,0 +1,217 @@
+/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#ifndef __GZVM_COMMON_H__
+#define __GZVM_COMMON_H__
+
+#include <linux/const.h>
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+/* gzvm only supports aarch64 platform */
+#if defined(__aarch64__)
+#include <asm/gzvm_arch.h>
+#endif
+
+/**
+ * @brief This file declares common data structure shared between userspace,
+ *        kernel space, and GZ.
+ */
+enum {
+	GZVM_FUNC_CREATE_VM = 0,
+	GZVM_FUNC_DESTROY_VM,
+	GZVM_FUNC_CREATE_VCPU,
+	GZVM_FUNC_DESTROY_VCPU,
+	GZVM_FUNC_SET_MEMREGION,
+	GZVM_FUNC_RUN,
+	GZVM_FUNC_GET_REGS,
+	GZVM_FUNC_SET_REGS,
+	GZVM_FUNC_GET_ONE_REG,
+	GZVM_FUNC_SET_ONE_REG,
+	GZVM_FUNC_IRQ_LINE,
+	GZVM_FUNC_CREATE_DEVICE,
+	GZVM_FUNC_PROBE,
+	GZVM_FUNC_ENABLE_CAP,
+	NR_GZVM_FUNC
+};
+
+typedef __u16 gzvm_id_t;
+typedef __u16 gzvm_vcpu_id_t;
+
+/* VM exit reason */
+enum {
+	GZVM_EXIT_UNKNOWN = 0x92920000,
+	GZVM_EXIT_MMIO,
+	GZVM_EXIT_HVC,
+	GZVM_EXIT_IRQ,
+	GZVM_EXIT_EXCEPTION,
+	GZVM_EXIT_DEBUG,
+	GZVM_EXIT_FAIL_ENTRY,
+	GZVM_EXIT_INTERNAL_ERROR,
+	GZVM_EXIT_SYSTEM_EVENT,
+	GZVM_EXIT_SHUTDOWN,
+};
+
+/**
+ * @brief same purpose as kvm_run, this struct is shared between userspace,
+ *	kernel and GZ
+ * Note: keep identical layout between the 3 modules
+ */
+struct gzvm_vcpu_run {
+	/* to userspace */
+	__u32 exit_reason;
+	__u8 immediate_exit;
+	__u8 padding1[3];
+	/* union structure of collection of guest exit reason */
+	union {
+		/* GZVM_EXIT_MMIO */
+		struct {
+			__u64 phys_addr;		/* from FAR_EL2 */
+			__u8 data[8];
+			__u64 size;			/* from ESR_EL2 as */
+			__u32 reg_nr;			/* from ESR_EL2 */
+			__u8 is_write;			/* from ESR_EL2 */
+		} mmio;
+		/* GZVM_EXIT_FAIL_ENTRY */
+		struct {
+			__u64 hardware_entry_failure_reason;
+			__u32 cpu;
+		} fail_entry;
+		/* GZVM_EXIT_EXCEPTION */
+		struct {
+			__u32 exception;
+			__u32 error_code;
+		} exception;
+		/* GZVM_EXIT_HVC */
+		struct {
+			__u64 args[8];	/* in-out */
+		} hvc;
+		/* GZVM_EXIT_INTERNAL_ERROR */
+		struct {
+			__u32 suberror;
+			__u32 ndata;
+			__u64 data[16];
+		} internal;
+		/* GZVM_EXIT_SYSTEM_EVENT */
+		struct {
+#define GZVM_SYSTEM_EVENT_SHUTDOWN       1
+#define GZVM_SYSTEM_EVENT_RESET          2
+#define GZVM_SYSTEM_EVENT_CRASH          3
+#define GZVM_SYSTEM_EVENT_WAKEUP         4
+#define GZVM_SYSTEM_EVENT_SUSPEND        5
+#define GZVM_SYSTEM_EVENT_SEV_TERM       6
+#define GZVM_SYSTEM_EVENT_S2IDLE         7
+			__u32 type;
+			__u32 ndata;
+			__u64 data[16];
+		} system_event;
+		/* Fix the size of the union. */
+		char padding[256];
+	};
+};
+
+#define GIC_V3_NR_LRS		16
+
+struct gzvm_vcpu_hwstate {
+	__u32 nr_lrs;
+	__u64 lr[GIC_V3_NR_LRS];
+};
+
+/* GZVM ioctls */
+#define GZVM_IOC_MAGIC			0x92	/* gz */
+
+/*
+ * ioctls for /dev/gzvm fds:
+ */
+#define GZVM_GET_API_VERSION       _IO(GZVM_IOC_MAGIC,   0x00)
+#define GZVM_CREATE_VM             _IO(GZVM_IOC_MAGIC,   0x01)
+
+#define GZVM_CAP_ARM_VM_IPA_SIZE	165
+#define GZVM_CAP_ARM_PROTECTED_VM	0xffbadab1
+
+#define GZVM_CHECK_EXTENSION       _IO(GZVM_IOC_MAGIC,   0x03)
+
+/*
+ * ioctls for VM fds
+ */
+
+/* for GZVM_SET_MEMORY_REGION */
+struct gzvm_memory_region {
+	__u32 slot;
+	__u32 flags;
+	__u64 guest_phys_addr;
+	__u64 memory_size; /* bytes */
+};
+#define GZVM_SET_MEMORY_REGION     _IOW(GZVM_IOC_MAGIC,  0x40, \
+					struct gzvm_memory_region)
+/*
+ * GZVM_CREATE_VCPU receives as a parameter the vcpu slot, and returns
+ * a vcpu fd.
+ */
+#define GZVM_CREATE_VCPU           _IO(GZVM_IOC_MAGIC,   0x41)
+
+/* for GZVM_SET_USER_MEMORY_REGION */
+struct gzvm_userspace_memory_region {
+	__u32 slot;
+	__u32 flags;
+	__u64 guest_phys_addr;
+	__u64 memory_size; /* bytes */
+	__u64 userspace_addr; /* start of the userspace allocated memory */
+};
+#define GZVM_SET_USER_MEMORY_REGION _IOW(GZVM_IOC_MAGIC, 0x46, \
+					struct gzvm_userspace_memory_region)
+
+/* for GZVM_IRQ_LINE */
+/* GZVM_IRQ_LINE irq field index values */
+#define GZVM_IRQ_VCPU2_SHIFT		28
+#define GZVM_IRQ_VCPU2_MASK		0xf
+#define GZVM_IRQ_TYPE_SHIFT		24
+#define GZVM_IRQ_TYPE_MASK		0xf
+#define GZVM_IRQ_VCPU_SHIFT		16
+#define GZVM_IRQ_VCPU_MASK		0xff
+#define GZVM_IRQ_NUM_SHIFT		0
+#define GZVM_IRQ_NUM_MASK		0xffff
+
+/* irq_type field */
+#define GZVM_IRQ_TYPE_CPU		0
+#define GZVM_IRQ_TYPE_SPI		1
+#define GZVM_IRQ_TYPE_PPI		2
+
+/* out-of-kernel GIC cpu interrupt injection irq_number field */
+#define GZVM_IRQ_CPU_IRQ		0
+#define GZVM_IRQ_CPU_FIQ		1
+
+/*
+ * ioctls for vcpu fds
+ */
+#define GZVM_RUN                   _IO(GZVM_IOC_MAGIC,   0x80)
+
+/* sub-commands put in args[0] for GZVM_CAP_ARM_PROTECTED_VM */
+#define GZVM_CAP_ARM_PVM_SET_PVMFW_IPA		0
+#define GZVM_CAP_ARM_PVM_GET_PVMFW_SIZE		1
+
+/* for GZVM_ENABLE_CAP */
+struct gzvm_enable_cap {
+	/* in */
+	__u64 cap;
+	/* we have total 5 (8 - 3) registers can be used for additional args */
+	__u64 args[5];
+};
+#define GZVM_ENABLE_CAP            _IOW(GZVM_IOC_MAGIC,  0xa3, \
+					struct gzvm_enable_cap)
+
+struct gzvm_one_reg {
+	__u64 id;
+	__u64 addr;
+};
+#define GZVM_GET_ONE_REG	   _IOW(GZVM_IOC_MAGIC,  0xab, \
+					struct gzvm_one_reg)
+#define GZVM_SET_ONE_REG	   _IOW(GZVM_IOC_MAGIC,  0xac, \
+					struct gzvm_one_reg)
+
+#define GZVM_REG_ARCH_MASK	0xff00000000000000ULL
+#define GZVM_REG_GENERIC	0x0000000000000000ULL
+
+#endif /* __GZVM_COMMON_H__ */
-- 
2.18.0

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[kernel test robot]

Hi Yi-De,

kernel test robot noticed the following build warnings:

[auto build test WARNING on robh/for-next]
[also build test WARNING on krzk-dt/for-next arm64/for-next/core lwn/docs-next linus/master v6.3-rc6 next-20230412]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
base:   https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link:    https://lore.kernel.org/r/20230413090735.4182-4-yi-de.wu%40mediatek.com
patch subject: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support
reproduce:
        make versioncheck

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202304131836.ijgy8JuB-lkp@intel.com/

versioncheck warnings: (new ones prefixed by >>)
   INFO PATH=/opt/cross/clang/bin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
   /usr/bin/timeout -k 100 3h /usr/bin/make W=1 --keep-going HOSTCC=gcc-11 CC=gcc-11 -j32 ARCH=x86_64 versioncheck
   find ./* \( -name SCCS -o -name BitKeeper -o -name .svn -o -name CVS -o -name .pc -o -name .hg -o -name .git \) -prune -o \
   	-name '*.[hcS]' -type f -print | sort \
   	| xargs perl -w ./scripts/checkversion.pl
   ./drivers/accessibility/speakup/genmap.c: 13 linux/version.h not needed.
   ./drivers/accessibility/speakup/makemapdata.c: 13 linux/version.h not needed.
   ./drivers/net/ethernet/qlogic/qede/qede.h: 10 linux/version.h not needed.
   ./drivers/net/ethernet/qlogic/qede/qede_ethtool.c: 7 linux/version.h not needed.
>> ./drivers/soc/mediatek/virt/geniezone/gzvm_vm.c: 14 linux/version.h not needed.
   ./drivers/soc/tegra/cbb/tegra-cbb.c: 19 linux/version.h not needed.
   ./drivers/soc/tegra/cbb/tegra194-cbb.c: 26 linux/version.h not needed.
   ./drivers/soc/tegra/cbb/tegra234-cbb.c: 27 linux/version.h not needed.
   ./drivers/staging/media/atomisp/include/linux/atomisp.h: 23 linux/version.h not needed.
   ./samples/trace_events/trace_custom_sched.c: 11 linux/version.h not needed.
   ./sound/soc/codecs/cs42l42.c: 14 linux/version.h not needed.
   ./tools/lib/bpf/bpf_helpers.h: 289: need linux/version.h
   ./tools/perf/tests/bpf-script-example.c: 60: need linux/version.h
   ./tools/perf/tests/bpf-script-test-kbuild.c: 21: need linux/version.h
   ./tools/perf/tests/bpf-script-test-prologue.c: 49: need linux/version.h
   ./tools/perf/tests/bpf-script-test-relocation.c: 51: need linux/version.h
   ./tools/testing/selftests/bpf/progs/dev_cgroup.c: 9 linux/version.h not needed.
   ./tools/testing/selftests/bpf/progs/netcnt_prog.c: 3 linux/version.h not needed.
   ./tools/testing/selftests/bpf/progs/test_map_lock.c: 4 linux/version.h not needed.
   ./tools/testing/selftests/bpf/progs/test_send_signal_kern.c: 4 linux/version.h not needed.
   ./tools/testing/selftests/bpf/progs/test_spin_lock.c: 4 linux/version.h not needed.
   ./tools/testing/selftests/bpf/progs/test_tcp_estats.c: 37 linux/version.h not needed.
   ./tools/testing/selftests/wireguard/qemu/init.c: 27 linux/version.h not needed.

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[kernel test robot]

Hi Yi-De,

kernel test robot noticed the following build warnings:

[auto build test WARNING on robh/for-next]
[also build test WARNING on krzk-dt/for-next arm64/for-next/core lwn/docs-next linus/master v6.3-rc6 next-20230412]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
base:   https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link:    https://lore.kernel.org/r/20230413090735.4182-4-yi-de.wu%40mediatek.com
patch subject: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support
config: arm64-allyesconfig (https://download.01.org/0day-ci/archive/20230413/202304132051.wGFK08qz-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/2d01949ddd48b1bc2cc9849154afe60781068f39
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
        git checkout 2d01949ddd48b1bc2cc9849154afe60781068f39
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 olddefconfig
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash drivers/soc/mediatek/virt/geniezone/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202304132051.wGFK08qz-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> drivers/soc/mediatek/virt/geniezone/gzvm_vm.c:31:5: warning: no previous prototype for 'gzvm_gfn_to_pfn_memslot' [-Wmissing-prototypes]
      31 | int gzvm_gfn_to_pfn_memslot(struct gzvm_memslot *memslot, u64 gfn, u64 *pfn)
         |     ^~~~~~~~~~~~~~~~~~~~~~~


vim +/gzvm_gfn_to_pfn_memslot +31 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c

    19	
    20	
    21	/**
    22	 * @brief Translate gfn (guest ipa) to pfn (host pa), result is in @pfn
    23	 *
    24	 * Leverage KVM's `gfn_to_pfn_memslot`. Because `gfn_to_pfn_memslot` needs
    25	 * kvm_memory_slot as parameter, this function populates necessary fileds
    26	 * for calling `gfn_to_pfn_memslot`.
    27	 *
    28	 * @retval 0 succeed
    29	 * @retval -EFAULT failed to convert
    30	 */
  > 31	int gzvm_gfn_to_pfn_memslot(struct gzvm_memslot *memslot, u64 gfn, u64 *pfn)
    32	{
    33		hfn_t __pfn;
    34		struct kvm_memory_slot kvm_slot = {0};
    35	
    36		kvm_slot.base_gfn = memslot->base_gfn;
    37		kvm_slot.npages = memslot->npages;
    38		kvm_slot.dirty_bitmap = NULL;
    39		kvm_slot.userspace_addr = memslot->userspace_addr;
    40		kvm_slot.flags = memslot->flags;
    41		kvm_slot.id = memslot->slot_id;
    42		kvm_slot.as_id = 0;
    43	
    44		__pfn = gfn_to_pfn_memslot(&kvm_slot, gfn);
    45		if (is_error_noslot_pfn(__pfn)) {
    46			*pfn = 0;
    47			return -EFAULT;
    48		}
    49	
    50		*pfn = __pfn;
    51		return 0;
    52	}
    53	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[kernel test robot]

Hi Yi-De,

kernel test robot noticed the following build errors:

[auto build test ERROR on robh/for-next]
[also build test ERROR on krzk-dt/for-next arm64/for-next/core lwn/docs-next linus/master v6.3-rc6 next-20230412]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
base:   https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link:    https://lore.kernel.org/r/20230413090735.4182-4-yi-de.wu%40mediatek.com
patch subject: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support
config: x86_64-randconfig-a014-20230410 (https://download.01.org/0day-ci/archive/20230413/202304132053.o6E89lC6-lkp@intel.com/config)
compiler: gcc-11 (Debian 11.3.0-8) 11.3.0
reproduce (this is a W=1 build):
        # https://github.com/intel-lab-lkp/linux/commit/2d01949ddd48b1bc2cc9849154afe60781068f39
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
        git checkout 2d01949ddd48b1bc2cc9849154afe60781068f39
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        make W=1 O=build_dir ARCH=x86_64 olddefconfig
        make W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202304132053.o6E89lC6-lkp@intel.com/

All errors (new ones prefixed by >>):

>> usr/include/linux/gzvm_common.h:15: included file 'asm-x86/gzvm_arch.h' is not exported

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Krzysztof Kozlowski]

On 13/04/2023 11:07, Yi-De Wu wrote:
> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> 
> GenieZone is MediaTek proprietary hypervisor solution, and it is running
> in EL2 stand alone as a type-I hypervisor. This patch exports a set of
> ioctl interfaces for userspace VMM (e.g., crosvm) to operate guest VMs
> lifecycle (creation, running, and destroy) on GenieZone.
> 
> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> ---
>  arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>  drivers/soc/mediatek/Kconfig                  |   2 +
>  drivers/soc/mediatek/Makefile                 |   1 +
>  drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +

Hypervisor drivers do not go to soc. Stop shoving there everything from
your downstream. Find appropriate directory, e.g. maybe drivers/virt.
See:
https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@quicinc.com/

You should follow that discussion as well and be sure that all concerns
raised for Gunyah are solved also here.

Best regards,
Krzysztof

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Matthias Brugger]

On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
> On 13/04/2023 11:07, Yi-De Wu wrote:
>> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
>>
>> GenieZone is MediaTek proprietary hypervisor solution, and it is running
>> in EL2 stand alone as a type-I hypervisor. This patch exports a set of
>> ioctl interfaces for userspace VMM (e.g., crosvm) to operate guest VMs
>> lifecycle (creation, running, and destroy) on GenieZone.
>>
>> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
>> ---
>>   arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>>   drivers/soc/mediatek/Kconfig                  |   2 +
>>   drivers/soc/mediatek/Makefile                 |   1 +
>>   drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
> 
> Hypervisor drivers do not go to soc. Stop shoving there everything from
> your downstream. Find appropriate directory, e.g. maybe drivers/virt.

Acked, what is the reason you want to add this to drivers/soc instead of 
drivers/virt?

Regards,
Matthias

> See:
> https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@quicinc.com/
> 
> You should follow that discussion as well and be sure that all concerns
> raised for Gunyah are solved also here.
> 
> Best regards,
> Krzysztof
> 

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Yi-De Wu (吳一德)]

On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
> > On 13/04/2023 11:07, Yi-De Wu wrote:
> > > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > > 
> > > GenieZone is MediaTek proprietary hypervisor solution, and it is
> > > running
> > > in EL2 stand alone as a type-I hypervisor. This patch exports a
> > > set of
> > > ioctl interfaces for userspace VMM (e.g., crosvm) to operate
> > > guest VMs
> > > lifecycle (creation, running, and destroy) on GenieZone.
> > > 
> > > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > > ---
> > >   arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
> > >   drivers/soc/mediatek/Kconfig                  |   2 +
> > >   drivers/soc/mediatek/Makefile                 |   1 +
> > >   drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
> > 
> > Hypervisor drivers do not go to soc. Stop shoving there everything
> > from
> > your downstream. Find appropriate directory, e.g. maybe
> > drivers/virt.
> 
> Acked, what is the reason you want to add this to drivers/soc instead
> of
> drivers/virt?
> 
> Regards,
> Matthias
> 
Noted. We would take your advice and move it from
drivers/soc/mediatek/virt to /drivers/virt on next version.

The reason we put it under our soc/ is that the drver is highly
propietary for mediatek's product and for aarch64 only. Maybe it's not
general enough to put in under /drivers/virt.

Related discussions happened here for your information. 

https://android-review.googlesource.com/c/kernel/common/+/2447547/1..2/drivers/virt/geniezone/gzvm.h#b91

> > See:
> > 
https://lore.kernel.org/all/20230304010632.2127470-1-quic_eberman@quicinc.com/
> > 
> > You should follow that discussion as well and be sure that all
> > concerns
> > raised for Gunyah are solved also here.
> > 
Sure, we would look it up for the above-mentioned information.

> > Best regards,
> > Krzysztof
> > 

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Krzysztof Kozlowski]

On 14/04/2023 10:43, Yi-De Wu (吳一德) wrote:
> On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
>> External email : Please do not click links or open attachments until
>> you have verified the sender or the content.
>>
>>
>> On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
>>> On 13/04/2023 11:07, Yi-De Wu wrote:
>>>> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
>>>>
>>>> GenieZone is MediaTek proprietary hypervisor solution, and it is
>>>> running
>>>> in EL2 stand alone as a type-I hypervisor. This patch exports a
>>>> set of
>>>> ioctl interfaces for userspace VMM (e.g., crosvm) to operate
>>>> guest VMs
>>>> lifecycle (creation, running, and destroy) on GenieZone.
>>>>
>>>> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>>>> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
>>>> ---
>>>>   arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>>>>   drivers/soc/mediatek/Kconfig                  |   2 +
>>>>   drivers/soc/mediatek/Makefile                 |   1 +
>>>>   drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
>>>
>>> Hypervisor drivers do not go to soc. Stop shoving there everything
>>> from
>>> your downstream. Find appropriate directory, e.g. maybe
>>> drivers/virt.
>>
>> Acked, what is the reason you want to add this to drivers/soc instead
>> of
>> drivers/virt?
>>
>> Regards,
>> Matthias
>>
> Noted. We would take your advice and move it from
> drivers/soc/mediatek/virt to /drivers/virt on next version.
> 
> The reason we put it under our soc/ is that the drver is highly
> propietary for mediatek's product and for aarch64 only. Maybe it's not
> general enough to put in under /drivers/virt.

If virt folks reject the driver, because it is highly proprietary, then
it is not suitable for soc/mediatek either.

Your argument is actually not helping you. It's rather a proof that this
driver might not be suitable for Linux kernel at all.

>
https://android-review.googlesource.com/c/kernel/common/+/2447547/1..2/drivers/virt/geniezone/gzvm.h#b91

I don't see there anything suggesting moving to soc/mediatek. Comment
from Trilok (+Cc) suggests that your code is simply not portable. Write
code which is portable and properly organized.

Best regards,
Krzysztof

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Trilok Soni]

On 4/14/2023 1:51 AM, Krzysztof Kozlowski wrote:
> On 14/04/2023 10:43, Yi-De Wu (吳一德) wrote:
>> On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
>>> External email : Please do not click links or open attachments until
>>> you have verified the sender or the content.
>>>
>>>
>>> On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
>>>> On 13/04/2023 11:07, Yi-De Wu wrote:
>>>>> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
>>>>>
>>>>> GenieZone is MediaTek proprietary hypervisor solution, and it is
>>>>> running
>>>>> in EL2 stand alone as a type-I hypervisor. This patch exports a
>>>>> set of
>>>>> ioctl interfaces for userspace VMM (e.g., crosvm) to operate
>>>>> guest VMs
>>>>> lifecycle (creation, running, and destroy) on GenieZone.
>>>>>
>>>>> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>>>>> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
>>>>> ---
>>>>>    arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>>>>>    drivers/soc/mediatek/Kconfig                  |   2 +
>>>>>    drivers/soc/mediatek/Makefile                 |   1 +
>>>>>    drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
>>>>
>>>> Hypervisor drivers do not go to soc. Stop shoving there everything
>>>> from
>>>> your downstream. Find appropriate directory, e.g. maybe
>>>> drivers/virt.
>>>
>>> Acked, what is the reason you want to add this to drivers/soc instead
>>> of
>>> drivers/virt?
>>>
>>> Regards,
>>> Matthias
>>>
>> Noted. We would take your advice and move it from
>> drivers/soc/mediatek/virt to /drivers/virt on next version.
>>
>> The reason we put it under our soc/ is that the drver is highly
>> propietary for mediatek's product and for aarch64 only. Maybe it's not
>> general enough to put in under /drivers/virt.
> 
> If virt folks reject the driver, because it is highly proprietary, then
> it is not suitable for soc/mediatek either.
> 
> Your argument is actually not helping you. It's rather a proof that this
> driver might not be suitable for Linux kernel at all.
> 
>>
> https://android-review.googlesource.com/c/kernel/common/+/2447547/1..2/drivers/virt/geniezone/gzvm.h#b91
> 
> I don't see there anything suggesting moving to soc/mediatek. Comment
> from Trilok (+Cc) suggests that your code is simply not portable. Write
> code which is portable and properly organized.

Thanks for the CC. I don't know how different these patches are from the 
ACK post, but if they are similar then I am surprised that patches of 
that state are posted here since they will need lot of work to get it 
reviewed here.

Also, do you plan to open-source your hypervisor? I am not sure if that 
is the requirement but it will be good to know if some version of your 
Hypervisor is open-sourced or you have plan for that.

---Trilok Soni

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Yi-De Wu (吳一德)]

On Fri, 2023-04-14 at 10:17 -0700, Trilok Soni wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 4/14/2023 1:51 AM, Krzysztof Kozlowski wrote:
> > On 14/04/2023 10:43, Yi-De Wu (吳一德) wrote:
> > > On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
> > > > External email : Please do not click links or open attachments
> > > > until
> > > > you have verified the sender or the content.
> > > > 
> > > > 
> > > > On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
> > > > > On 13/04/2023 11:07, Yi-De Wu wrote:
> > > > > > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > > > > > 
> > > > > > GenieZone is MediaTek proprietary hypervisor solution, and
> > > > > > it is
> > > > > > running
> > > > > > in EL2 stand alone as a type-I hypervisor. This patch
> > > > > > exports a
> > > > > > set of
> > > > > > ioctl interfaces for userspace VMM (e.g., crosvm) to
> > > > > > operate
> > > > > > guest VMs
> > > > > > lifecycle (creation, running, and destroy) on GenieZone.
> > > > > > 
> > > > > > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > > > > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > > > > > ---
> > > > > >    arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
> > > > > >    drivers/soc/mediatek/Kconfig                  |   2 +
> > > > > >    drivers/soc/mediatek/Makefile                 |   1 +
> > > > > >    drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
> > > > > 
> > > > > Hypervisor drivers do not go to soc. Stop shoving there
> > > > > everything
> > > > > from
> > > > > your downstream. Find appropriate directory, e.g. maybe
> > > > > drivers/virt.
> > > > 
> > > > Acked, what is the reason you want to add this to drivers/soc
> > > > instead
> > > > of
> > > > drivers/virt?
> > > > 
> > > > Regards,
> > > > Matthias
> > > > 
> > > 
> > > Noted. We would take your advice and move it from
> > > drivers/soc/mediatek/virt to /drivers/virt on next version.
> > > 
> > > The reason we put it under our soc/ is that the drver is highly
> > > propietary for mediatek's product and for aarch64 only. Maybe
> > > it's not
> > > general enough to put in under /drivers/virt.
> > 
> > If virt folks reject the driver, because it is highly proprietary,
> > then
> > it is not suitable for soc/mediatek either.
> > 
> > Your argument is actually not helping you. It's rather a proof that
> > this
> > driver might not be suitable for Linux kernel at all.
> > 
> > > 
> > 
> > 
https://urldefense.com/v3/__https://android-review.googlesource.com/c/kernel/common/*/2447547/1..2/drivers/virt/geniezone/gzvm.h*b91__;KyM!!CTRNKA9wMg0ARbw!gV4z0n7DNq-QuX66Oln0w0grKWY3km14bMGVtv-keTfeOyCVsVQRYfgqKP4RWiA3BlgfkgS0OytOf12PquxZs9o$
> > 
> > I don't see there anything suggesting moving to soc/mediatek.
> > Comment
> > from Trilok (+Cc) suggests that your code is simply not portable.
> > Write
> > code which is portable and properly organized.
> 
> Thanks for the CC. I don't know how different these patches are from
> the
> ACK post, but if they are similar then I am surprised that patches of
> that state are posted here since they will need lot of work to get it
> reviewed here.
> 
> Also, do you plan to open-source your hypervisor? I am not sure if
> that
> is the requirement but it will be good to know if some version of
> your
> Hypervisor is open-sourced or you have plan for that.
> 
> ---Trilok Soni
> 

There would be some difficulties for us to open source our hypervisor
right now. But we will try our utmost to answer all the related
questions to make our design more comprehensive and transparent to the
public.

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Yi-De Wu (吳一德)]

On Fri, 2023-04-14 at 10:51 +0200, Krzysztof Kozlowski wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> On 14/04/2023 10:43, Yi-De Wu (吳一德) wrote:
> > On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
> > > External email : Please do not click links or open attachments
> > > until
> > > you have verified the sender or the content.
> > > 
> > > 
> > > On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
> > > > On 13/04/2023 11:07, Yi-De Wu wrote:
> > > > > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > > > > 
> > > > > GenieZone is MediaTek proprietary hypervisor solution, and it
> > > > > is
> > > > > running
> > > > > in EL2 stand alone as a type-I hypervisor. This patch exports
> > > > > a
> > > > > set of
> > > > > ioctl interfaces for userspace VMM (e.g., crosvm) to operate
> > > > > guest VMs
> > > > > lifecycle (creation, running, and destroy) on GenieZone.
> > > > > 
> > > > > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > > > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > > > > ---
> > > > >   arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
> > > > >   drivers/soc/mediatek/Kconfig                  |   2 +
> > > > >   drivers/soc/mediatek/Makefile                 |   1 +
> > > > >   drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
> > > > 
> > > > Hypervisor drivers do not go to soc. Stop shoving there
> > > > everything
> > > > from
> > > > your downstream. Find appropriate directory, e.g. maybe
> > > > drivers/virt.
> > > 
> > > Acked, what is the reason you want to add this to drivers/soc
> > > instead
> > > of
> > > drivers/virt?
> > > 
> > > Regards,
> > > Matthias
> > > 
> > 
> > Noted. We would take your advice and move it from
> > drivers/soc/mediatek/virt to /drivers/virt on next version.
> > 
> > The reason we put it under our soc/ is that the drver is highly
> > propietary for mediatek's product and for aarch64 only. Maybe it's
> > not
> > general enough to put in under /drivers/virt.
> 
> If virt folks reject the driver, because it is highly proprietary,
> then
> it is not suitable for soc/mediatek either.
> 
> Your argument is actually not helping you. It's rather a proof that
> this
> driver might not be suitable for Linux kernel at all.
> 
> > 
> 
> 
https://urldefense.com/v3/__https://android-review.googlesource.com/c/kernel/common/*/2447547/1..2/drivers/virt/geniezone/gzvm.h*b91__;KyM!!CTRNKA9wMg0ARbw!jJ0FRv_s6iLuc1rp4RPApkktGpIarf5qAVE0_Dq6X_KCq_283Kh5DKW6jlMyDfaHNFr1DglKyRKq1JcE4XTdQjw9XGz_4Q$
> 
> I don't see there anything suggesting moving to soc/mediatek. Comment
> from Trilok (+Cc) suggests that your code is simply not portable.
> Write
> code which is portable and properly organized.
> 
> Best regards,
> Krzysztof
> 

We've already moved most of all works from
drivers/soc/mediatek/virt/geniezone to drivers/virt/geniezone and
arch/arm64/geniezone for general and architecture dependent
implementations respectively. And hopefully the code could be reviewd
out there.

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[AngeloGioacchino Del Regno]

Il 14/04/23 10:43, Yi-De Wu (吳一德) ha scritto:
> On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
>> External email : Please do not click links or open attachments until
>> you have verified the sender or the content.
>>
>>
>> On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
>>> On 13/04/2023 11:07, Yi-De Wu wrote:
>>>> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
>>>>
>>>> GenieZone is MediaTek proprietary hypervisor solution, and it is
>>>> running
>>>> in EL2 stand alone as a type-I hypervisor. This patch exports a
>>>> set of
>>>> ioctl interfaces for userspace VMM (e.g., crosvm) to operate
>>>> guest VMs
>>>> lifecycle (creation, running, and destroy) on GenieZone.
>>>>
>>>> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
>>>> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
>>>> ---
>>>>    arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>>>>    drivers/soc/mediatek/Kconfig                  |   2 +
>>>>    drivers/soc/mediatek/Makefile                 |   1 +
>>>>    drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
>>>
>>> Hypervisor drivers do not go to soc. Stop shoving there everything
>>> from
>>> your downstream. Find appropriate directory, e.g. maybe
>>> drivers/virt.
>>
>> Acked, what is the reason you want to add this to drivers/soc instead
>> of
>> drivers/virt?
>>
>> Regards,
>> Matthias
>>
> Noted. We would take your advice and move it from
> drivers/soc/mediatek/virt to /drivers/virt on next version.
> 
> The reason we put it under our soc/ is that the drver is highly
> propietary for mediatek's product and for aarch64 only. Maybe it's not
> general enough to put in under /drivers/virt.

This is the same reason why mediatek-drm is in drivers/gpu/drm/ and the same why
mediatek-cpufreq is in drivers/cpufreq/.

I know that this is a MediaTek specific implementation, but it *is* a hypervisor
driver, hence it belongs to the hypervisor drivers folder.
It's not even granted that this will not support other MediaTek architectures in
the future, but that's not a discussion to do right here and right now, and it's
anyway irrelevant in this moment.

By the way, good job with upstreaming your drivers targeting MediaTek Android SW!
I'm enthusiast to see that.

Regards,
Angelo

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[Yi-De Wu (吳一德)]

On Fri, 2023-04-14 at 12:48 +0200, AngeloGioacchino Del Regno wrote:
> External email : Please do not click links or open attachments until
> you have verified the sender or the content.
> 
> 
> Il 14/04/23 10:43, Yi-De Wu (吳一德) ha scritto:
> > On Thu, 2023-04-13 at 19:08 +0200, Matthias Brugger wrote:
> > > External email : Please do not click links or open attachments
> > > until
> > > you have verified the sender or the content.
> > > 
> > > 
> > > On 13/04/2023 14:55, Krzysztof Kozlowski wrote:
> > > > On 13/04/2023 11:07, Yi-De Wu wrote:
> > > > > From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> > > > > 
> > > > > GenieZone is MediaTek proprietary hypervisor solution, and it
> > > > > is
> > > > > running
> > > > > in EL2 stand alone as a type-I hypervisor. This patch exports
> > > > > a
> > > > > set of
> > > > > ioctl interfaces for userspace VMM (e.g., crosvm) to operate
> > > > > guest VMs
> > > > > lifecycle (creation, running, and destroy) on GenieZone.
> > > > > 
> > > > > Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> > > > > Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> > > > > ---
> > > > >    arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
> > > > >    drivers/soc/mediatek/Kconfig                  |   2 +
> > > > >    drivers/soc/mediatek/Makefile                 |   1 +
> > > > >    drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
> > > > 
> > > > Hypervisor drivers do not go to soc. Stop shoving there
> > > > everything
> > > > from
> > > > your downstream. Find appropriate directory, e.g. maybe
> > > > drivers/virt.
> > > 
> > > Acked, what is the reason you want to add this to drivers/soc
> > > instead
> > > of
> > > drivers/virt?
> > > 
> > > Regards,
> > > Matthias
> > > 
> > 
> > Noted. We would take your advice and move it from
> > drivers/soc/mediatek/virt to /drivers/virt on next version.
> > 
> > The reason we put it under our soc/ is that the drver is highly
> > propietary for mediatek's product and for aarch64 only. Maybe it's
> > not
> > general enough to put in under /drivers/virt.
> 
> This is the same reason why mediatek-drm is in drivers/gpu/drm/ and
> the same why
> mediatek-cpufreq is in drivers/cpufreq/.
> 
> I know that this is a MediaTek specific implementation, but it *is* a
> hypervisor
> driver, hence it belongs to the hypervisor drivers folder.
> It's not even granted that this will not support other MediaTek
> architectures in
> the future, but that's not a discussion to do right here and right
> now, and it's
> anyway irrelevant in this moment.
> 
> By the way, good job with upstreaming your drivers targeting MediaTek
> Android SW!
> I'm enthusiast to see that.
> 
> Regards,
> Angelo
> 

Thank you Angelo, we've already separated the general and architecture
dependent implementations as you suggested and delivered in v2.

Re: [PATCH v1 3/6] soc: mediatek: virt: geniezone: Introduce GenieZone hypervisor support

[AngeloGioacchino Del Regno]

Il 13/04/23 11:07, Yi-De Wu ha scritto:
> From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>
> 
> GenieZone is MediaTek proprietary hypervisor solution, and it is running
> in EL2 stand alone as a type-I hypervisor. This patch exports a set of
> ioctl interfaces for userspace VMM (e.g., crosvm) to operate guest VMs
> lifecycle (creation, running, and destroy) on GenieZone.
> 
> Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
> Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
> ---
>   arch/arm64/include/uapi/asm/gzvm_arch.h       |  79 ++++
>   drivers/soc/mediatek/Kconfig                  |   2 +
>   drivers/soc/mediatek/Makefile                 |   1 +
>   drivers/soc/mediatek/virt/geniezone/Kconfig   |  17 +
>   drivers/soc/mediatek/virt/geniezone/Makefile  |   5 +
>   drivers/soc/mediatek/virt/geniezone/gzvm.h    | 103 ++++
>   .../soc/mediatek/virt/geniezone/gzvm_hyp.h    |  72 +++
>   .../soc/mediatek/virt/geniezone/gzvm_main.c   | 233 +++++++++
>   .../soc/mediatek/virt/geniezone/gzvm_vcpu.c   | 266 +++++++++++
>   drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 444 ++++++++++++++++++
>   include/uapi/linux/gzvm_common.h              | 217 +++++++++
>   11 files changed, 1439 insertions(+)
>   create mode 100644 arch/arm64/include/uapi/asm/gzvm_arch.h
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/Kconfig
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/Makefile
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm.h
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_hyp.h
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_main.c
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
>   create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
>   create mode 100644 include/uapi/linux/gzvm_common.h
> 
> diff --git a/arch/arm64/include/uapi/asm/gzvm_arch.h b/arch/arm64/include/uapi/asm/gzvm_arch.h
> new file mode 100644
> index 000000000000..3714f96c832b
> --- /dev/null
> +++ b/arch/arm64/include/uapi/asm/gzvm_arch.h
> @@ -0,0 +1,79 @@
> +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> +/*
> + * Copyright (c) 2023 MediaTek Inc.
> + */
> +
> +#ifndef __GZVM_ARCH_H__
> +#define __GZVM_ARCH_H__
> +
> +#include <asm/ptrace.h>
> +
> +/*
> + * Architecture specific registers are to be defined in arch headers and
> + * ORed with the arch identifier.
> + */
> +#define GZVM_REG_ARM		0x4000000000000000ULL
> +#define GZVM_REG_ARM64		0x6000000000000000ULL
> +
> +#define GZVM_REG_SIZE_SHIFT	52
> +#define GZVM_REG_SIZE_MASK	0x00f0000000000000ULL
> +#define GZVM_REG_SIZE_U8	0x0000000000000000ULL
> +#define GZVM_REG_SIZE_U16	0x0010000000000000ULL
> +#define GZVM_REG_SIZE_U32	0x0020000000000000ULL
> +#define GZVM_REG_SIZE_U64	0x0030000000000000ULL
> +#define GZVM_REG_SIZE_U128	0x0040000000000000ULL
> +#define GZVM_REG_SIZE_U256	0x0050000000000000ULL
> +#define GZVM_REG_SIZE_U512	0x0060000000000000ULL
> +#define GZVM_REG_SIZE_U1024	0x0070000000000000ULL
> +#define GZVM_REG_SIZE_U2048	0x0080000000000000ULL
> +
> +#define GZVM_NR_SPSR	5
> +struct gzvm_regs {
> +	struct user_pt_regs regs;	/* sp = sp_el0 */
> +
> +	__u64	sp_el1;
> +	__u64	elr_el1;
> +
> +	__u64	spsr[GZVM_NR_SPSR];
> +
> +	struct user_fpsimd_state fp_regs;
> +};
> +
> +/* If you need to interpret the index values, here is the key: */
> +#define GZVM_REG_ARM_COPROC_MASK	0x000000000FFF0000
> +#define GZVM_REG_ARM_COPROC_SHIFT	16
> +
> +/* Normal registers are mapped as coprocessor 16. */
> +#define GZVM_REG_ARM_CORE		(0x0010 << GZVM_REG_ARM_COPROC_SHIFT)
> +#define GZVM_REG_ARM_CORE_REG(name)	(offsetof(struct gzvm_regs, name) / sizeof(__u32))
> +
> +/* Some registers need more space to represent values. */
> +#define GZVM_REG_ARM_DEMUX		(0x0011 << GZVM_REG_ARM_COPROC_SHIFT)
> +#define GZVM_REG_ARM_DEMUX_ID_MASK	0x000000000000FF00
> +#define GZVM_REG_ARM_DEMUX_ID_SHIFT	8
> +#define GZVM_REG_ARM_DEMUX_ID_CCSIDR	(0x00 << GZVM_REG_ARM_DEMUX_ID_SHIFT)
> +#define GZVM_REG_ARM_DEMUX_VAL_MASK	0x00000000000000FF
> +#define GZVM_REG_ARM_DEMUX_VAL_SHIFT	0
> +
> +/* AArch64 system registers */
> +#define GZVM_REG_ARM64_SYSREG		(0x0013 << GZVM_REG_ARM_COPROC_SHIFT)
> +#define GZVM_REG_ARM64_SYSREG_OP0_MASK	0x000000000000c000
> +#define GZVM_REG_ARM64_SYSREG_OP0_SHIFT	14
> +#define GZVM_REG_ARM64_SYSREG_OP1_MASK	0x0000000000003800
> +#define GZVM_REG_ARM64_SYSREG_OP1_SHIFT	11
> +#define GZVM_REG_ARM64_SYSREG_CRN_MASK	0x0000000000000780
> +#define GZVM_REG_ARM64_SYSREG_CRN_SHIFT	7
> +#define GZVM_REG_ARM64_SYSREG_CRM_MASK	0x0000000000000078
> +#define GZVM_REG_ARM64_SYSREG_CRM_SHIFT	3
> +#define GZVM_REG_ARM64_SYSREG_OP2_MASK	0x0000000000000007
> +#define GZVM_REG_ARM64_SYSREG_OP2_SHIFT	0
> +
> +/* Physical Timer EL0 Registers */
> +#define GZVM_REG_ARM_PTIMER_CTL		ARM64_SYS_REG(3, 3, 14, 2, 1)
> +#define GZVM_REG_ARM_PTIMER_CVAL	ARM64_SYS_REG(3, 3, 14, 2, 2)
> +#define GZVM_REG_ARM_PTIMER_CNT		ARM64_SYS_REG(3, 3, 14, 0, 1)
> +
> +/* SVE registers */
> +#define GZVM_REG_ARM64_SVE		(0x0015 << KVM_REG_ARM_COPROC_SHIFT)
> +
> +#endif /* __GZVM_ARCH_H__ */
> diff --git a/drivers/soc/mediatek/Kconfig b/drivers/soc/mediatek/Kconfig
> index a88cf04fc803..01fad024a1c1 100644
> --- a/drivers/soc/mediatek/Kconfig
> +++ b/drivers/soc/mediatek/Kconfig
> @@ -91,4 +91,6 @@ config MTK_SVS
>   	  chip process corner, temperatures and other factors. Then DVFS
>   	  driver could apply SVS bank voltage to PMIC/Buck.
>   
> +source "drivers/soc/mediatek/virt/geniezone/Kconfig"
> +
>   endmenu
> diff --git a/drivers/soc/mediatek/Makefile b/drivers/soc/mediatek/Makefile
> index 8c0ddacbcde8..e5d7225c1d08 100644
> --- a/drivers/soc/mediatek/Makefile
> +++ b/drivers/soc/mediatek/Makefile
> @@ -9,3 +9,4 @@ obj-$(CONFIG_MTK_SCPSYS_PM_DOMAINS) += mtk-pm-domains.o
>   obj-$(CONFIG_MTK_MMSYS) += mtk-mmsys.o
>   obj-$(CONFIG_MTK_MMSYS) += mtk-mutex.o
>   obj-$(CONFIG_MTK_SVS) += mtk-svs.o
> +obj-$(CONFIG_MTK_GZVM) += virt/geniezone/
> diff --git a/drivers/soc/mediatek/virt/geniezone/Kconfig b/drivers/soc/mediatek/virt/geniezone/Kconfig
> new file mode 100644
> index 000000000000..6fad3c30f8d9
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/Kconfig
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +config MTK_GZVM
> +	tristate "GenieZone Hypervisor driver for guest VM operation"
> +	depends on ARM64
> +	depends on KVM
> +	help
> +	  This driver, gzvm, enables to run guest VMs on MTK GenieZone
> +	  hypervisor. It exports kvm-like interfaces for VMM (e.g., crosvm) in
> +	  order to operate guest VMs on GenieZone hypervisor.
> +
> +	  GenieZone hypervisor now only supports MediaTek SoC and arm64
> +	  architecture.
> +
> +	  Select M if you want it be built as a module (gzvm.ko).
> +
> +	  If unsure, say N.
> diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile
> new file mode 100644
> index 000000000000..e1dfbb9c568d
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/Makefile
> @@ -0,0 +1,5 @@
> +# SPDX-License-Identifier: GPL-2.0-only
> +
> +gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o
> +
> +obj-$(CONFIG_MTK_GZVM) += gzvm.o
> diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h
> new file mode 100644
> index 000000000000..43f215d4b0da
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h
> @@ -0,0 +1,103 @@
> +/* SPDX-License-Identifier: GPL-2.0 */
> +/*
> + * Copyright (c) 2023 MediaTek Inc.
> + */
> +
> +#ifndef __GZVM_H__
> +#define __GZVM_H__
> +
> +#include <linux/srcu.h>
> +#include <linux/arm-smccc.h>
> +#include <linux/gzvm_common.h>
> +#include "gzvm_hyp.h"
> +
> +#define MODULE_NAME	"gzvm"
> +#define GZVM_VCPU_MMAP_SIZE  PAGE_SIZE
> +#define INVALID_VM_ID   0xffff
> +
> +/* VM's memory slot descriptor */

You've documented almost all of the members of this struct ... can you please
change that to kerneldoc?

/**
  * struct gzvm_memslot - VM memory slot descriptor
  * @base_qfn:       Base of guest page frame
  * @npages:         Number of pages this slot covers
  * @userspace_addr: Corresponding user-space VA
  * .....other members
  */

> +struct gzvm_memslot {
> +	u64 base_gfn;			/* begin of guest page frame */
> +	unsigned long npages;		/* number of pages this slot covers */
> +	unsigned long userspace_addr;	/* corresponding userspace va */
> +	struct vm_area_struct *vma;	/* vma related to this userspace addr */
> +	u32 flags;
> +	u32 slot_id;
> +};
> +
> +/* pre-declaration for circular reference in struct gzvm */
> +struct gzvm_vcpu;
> +

Same here, if you could add kerneldoc description to this struct, that would
be highly appreciated, as it would greatly increase human readability.

> +struct gzvm {
> +	struct gzvm_vcpu *vcpus[GZVM_MAX_VCPUS];
> +	struct mm_struct *mm; /* userspace tied to this vm */
> +	struct gzvm_memslot memslot[GZVM_MAX_MEM_REGION];
> +	struct mutex lock;
> +	struct list_head vm_list;
> +	struct list_head devices;
> +	gzvm_id_t vm_id;
> +
> +	struct {
> +		spinlock_t        lock;
> +		struct list_head  items;
> +		struct list_head  resampler_list;
> +		struct mutex      resampler_lock;
> +	} irqfds;
> +	struct hlist_head irq_ack_notifier_list;
> +	struct srcu_struct irq_srcu;
> +	struct mutex irq_lock;
> +};
> +
> +struct gzvm_vcpu {
> +	struct gzvm *gzvm;
> +	int vcpuid;
> +	struct mutex lock;
> +	struct gzvm_vcpu_run *run;
> +	struct gzvm_vcpu_hwstate *hwstate;
> +};
> +
> +/**
> + * allocate 2 pages for data sharing between driver and gz hypervisor
> + * |- page 0           -|- page 1      -|
> + * |gzvm_vcpu_run|......|hwstate|.......|
> + */
> +#define GZVM_VCPU_RUN_MAP_SIZE		(PAGE_SIZE * 2)
> +
> +long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args);
> +
> +void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu);
> +int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid);
> +int gzvm_dev_ioctl_create_vm(unsigned long vm_type);
> +
> +int gzvm_arm_get_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg);
> +int gzvm_arm_set_reg(struct gzvm_vcpu *vcpu, const struct gzvm_one_reg *reg);
> +
> +int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
> +			 unsigned long a3, unsigned long a4, unsigned long a5,
> +			 unsigned long a6, unsigned long a7,
> +			 struct arm_smccc_res *res);

Could you please keep function signatures after *all* definitions?

> +
> +#define SMC_ENTITY_MTK			59
> +#define GZVM_FUNCID_START		(0x1000)
> +#define GZVM_HCALL_ID(func)				\
> +	ARM_SMCCC_CALL_VAL(ARM_SMCCC_FAST_CALL, ARM_SMCCC_SMC_32,	\
> +			   SMC_ENTITY_MTK, (GZVM_FUNCID_START + (func)))
> +
> +#define MT_HVC_GZVM_CREATE_VM		GZVM_HCALL_ID(GZVM_FUNC_CREATE_VM)
> +#define MT_HVC_GZVM_DESTROY_VM		GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VM)
> +#define MT_HVC_GZVM_CREATE_VCPU		GZVM_HCALL_ID(GZVM_FUNC_CREATE_VCPU)
> +#define MT_HVC_GZVM_DESTROY_VCPU	GZVM_HCALL_ID(GZVM_FUNC_DESTROY_VCPU)
> +#define MT_HVC_GZVM_SET_MEMREGION	GZVM_HCALL_ID(GZVM_FUNC_SET_MEMREGION)
> +#define MT_HVC_GZVM_RUN			GZVM_HCALL_ID(GZVM_FUNC_RUN)
> +#define MT_HVC_GZVM_GET_REGS		GZVM_HCALL_ID(GZVM_FUNC_GET_REGS)
> +#define MT_HVC_GZVM_SET_REGS		GZVM_HCALL_ID(GZVM_FUNC_SET_REGS)
> +#define MT_HVC_GZVM_GET_ONE_REG		GZVM_HCALL_ID(GZVM_FUNC_GET_ONE_REG)
> +#define MT_HVC_GZVM_SET_ONE_REG		GZVM_HCALL_ID(GZVM_FUNC_SET_ONE_REG)
> +#define MT_HVC_GZVM_IRQ_LINE		GZVM_HCALL_ID(GZVM_FUNC_IRQ_LINE)
> +#define MT_HVC_GZVM_CREATE_DEVICE	GZVM_HCALL_ID(GZVM_FUNC_CREATE_DEVICE)
> +#define MT_HVC_GZVM_PROBE		GZVM_HCALL_ID(GZVM_FUNC_PROBE)
> +#define MT_HVC_GZVM_ENABLE_CAP		GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP)
> +

(move them all here)

> +int gz_err_to_errno(unsigned long err);
> +
> +#endif /* __GZVM_H__ */

..snip..

> diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_main.c b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c
> new file mode 100644
> index 000000000000..1fabe4a579da
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_main.c
> @@ -0,0 +1,233 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2023 MediaTek Inc.
> + */
> +
> +#include <linux/anon_inodes.h>
> +#include <linux/arm-smccc.h>
> +#include <linux/device.h>
> +#include <linux/file.h>
> +#include <linux/kdev_t.h>
> +#include <linux/miscdevice.h>
> +#include <linux/module.h>
> +#include <linux/of.h>
> +#include <linux/platform_device.h>
> +#include <linux/slab.h>
> +#include "gzvm.h"
> +
> +static void (*invoke_gzvm_fn)(unsigned long, unsigned long, unsigned long,
> +			      unsigned long, unsigned long, unsigned long,
> +			      unsigned long, unsigned long,
> +			      struct arm_smccc_res *);
> +
> +static void gzvm_hvc(unsigned long a0, unsigned long a1, unsigned long a2,
> +		      unsigned long a3, unsigned long a4, unsigned long a5,
> +		      unsigned long a6, unsigned long a7,
> +		      struct arm_smccc_res *res)
> +{
> +	arm_smccc_hvc(a0, a1, a2, a3, a4, a5, a6, a7, res);
> +}
> +
> +static void gzvm_smc(unsigned long a0, unsigned long a1, unsigned long a2,
> +		      unsigned long a3, unsigned long a4, unsigned long a5,
> +		      unsigned long a6, unsigned long a7,
> +		      struct arm_smccc_res *res)
> +{
> +	arm_smccc_smc(a0, a1, a2, a3, a4, a5, a6, a7, res);
> +}

Why are you wrapping HVC and SMC functions? You're not doing anything special,
so you can simply call arm_amccc_{hvc,smc}() instead of specifying new wrappers.

Or are you worried about that changing signature all of a sudden?
This is not downstream, you don't have to worry about that. :-)

> +
> +static int gzvm_probe_conduit(void)
> +{
> +	struct arm_smccc_res res;
> +
> +	arm_smccc_hvc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res);
> +	if (res.a0 == 0) {
> +		invoke_gzvm_fn = gzvm_hvc;

invoke_gzvm_fn = arm_smccc_hvc;

> +		return 0;
> +	}
> +
> +	arm_smccc_smc(MT_HVC_GZVM_PROBE, 0, 0, 0, 0, 0, 0, 0, &res);
> +	if (res.a0 == 0) {
> +		invoke_gzvm_fn = gzvm_smc;

invoke_gzvm_fn = arm_smccc_smc;

> +		return 0;
> +	}
> +
> +	return -ENXIO;
> +}
> +
> +/**
> + * @brief geniezone hypercall wrapper
> + * @return int geniezone's return value will be converted to Linux errno
> + */

Kerneldoc please:

/**
  * gzvm_hypcall_wrapper() - GenieZone HyperCall wrapper
  * @a0: ...
  * @a1: ....
  * ......
  * Return: Zero for success, or a negative error number.
  *
  * The GenieZone return values are different from Linux error codes, hence
  * in case of error value, it is converted to a Linux negative error number.
  */

> +int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
> +			 unsigned long a3, unsigned long a4, unsigned long a5,
> +			 unsigned long a6, unsigned long a7,
> +			 struct arm_smccc_res *res)
> +{
> +	invoke_gzvm_fn(a0, a1, a2, a3, a4, a5, a6, a7, res);
> +	return gz_err_to_errno(res->a0);
> +}
> +
> +/**
> + * @brief Convert geniezone return value to standard errno
> + *
> + * @param err return value from geniezone hypercall (a0)
> + * @return int errno
> + */
> +int gz_err_to_errno(unsigned long err)
> +{
> +	int gz_err = (int) err;
> +
> +	switch (gz_err) {
> +	case 0:
> +		return 0;
> +	case ERR_NO_MEMORY:
> +		return -ENOMEM;
> +	case ERR_NOT_SUPPORTED:
> +		return -EOPNOTSUPP;
> +	case ERR_NOT_IMPLEMENTED:
> +		return -EOPNOTSUPP;
> +	case ERR_FAULT:
> +		return -EFAULT;
> +	default:

	default:
		break;
	}

	return -EINVAL;
};



> +
> +static int gzvm_cap_arm_vm_ipa_size(void __user *argp)
> +{
> +	u64 value = CONFIG_ARM64_PA_BITS;
> +
> +	if (copy_to_user(argp, &value, sizeof(u64)))
> +		return -EFAULT;
> +
> +	return 0;
> +}
> +
> +/**
> + * @brief Check if given capability is support or not
> + *
> + * @param args in/out u64 pointer from userspace
> + * @retval 0: support, no error
> + * @retval -EOPNOTSUPP: not support
> + * @retval -EFAULT: failed to get data from userspace
> + */

kerneldoc again, please.

> +long gzvm_dev_ioctl_check_extension(struct gzvm *gzvm, unsigned long args)
> +{
> +	int ret = -EOPNOTSUPP;
> +	__u64 cap, success = 1;
> +	void __user *argp = (void __user *) args;
> +
> +	if (copy_from_user(&cap, argp, sizeof(uint64_t)))
> +		return -EFAULT;
> +
> +	switch (cap) {
> +	case GZVM_CAP_ARM_PROTECTED_VM:
> +		if (copy_to_user(argp, &success, sizeof(uint64_t)))
> +			return -EFAULT;
> +		ret = 0;
> +		break;
> +	case GZVM_CAP_ARM_VM_IPA_SIZE:
> +		ret = gzvm_cap_arm_vm_ipa_size(argp);
> +		break;
> +	default:
> +		ret = -EOPNOTSUPP;
> +	}
> +
> +	return ret;
> +}
> + > +static long gzvm_dev_ioctl(struct file *filp, unsigned int cmd,
> +			    unsigned long user_args)
> +{
> +	long ret = -ENOTTY;
> +
> +	switch (cmd) {
> +	case GZVM_CREATE_VM:
> +		ret = gzvm_dev_ioctl_create_vm(user_args);
> +		break;
> +	case GZVM_CHECK_EXTENSION:
> +		if (!user_args)
> +			return -EINVAL;
> +		ret = gzvm_dev_ioctl_check_extension(NULL, user_args);
> +		break;
> +	default:
> +		ret = -ENOTTY;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct file_operations gzvm_chardev_ops = {
> +	.unlocked_ioctl = gzvm_dev_ioctl,
> +	.llseek		= noop_llseek,
> +};
> +
> +static struct miscdevice gzvm_dev = {
> +	.minor = MISC_DYNAMIC_MINOR,
> +	.name = MODULE_NAME,
> +	.fops = &gzvm_chardev_ops,
> +};
> +
> +static int gzvm_drv_probe(struct platform_device *pdev)
> +{
> +	if (!of_device_is_available(dev_of_node(&pdev->dev))) {

Uhm, is there something I don't get here?
This call looks odd; you're probing GZVM in this function....

> +		dev_info(&pdev->dev, "GenieZone hypervisor is not available\n");
> +		return -ENODEV;
> +	}
> +
> +	if (gzvm_probe_conduit() != 0) {
> +		dev_err(&pdev->dev, "Not found available conduit\n");
> +		return -ENODEV;
> +	}
> +
> +	return 0;
> +}
> +
> +static int gzvm_drv_remove(struct platform_device *pdev)
> +{
> +	misc_deregister(&gzvm_dev);
> +
> +	return 0;
> +}
> +
> +static const struct of_device_id gzvm_of_match[] = {
> +	{ .compatible = "mediatek,gzvm", },

	{ .compatible = "mediatek,geniezone-hyp" },
or
	{ .compatible = "mediatek,geniezone" },

..makes it a bit more descriptive and human readable.

> +	{},

^^^ always end with { /* sentinel */ },

> +};
> +MODULE_DEVICE_TABLE(of, gzvm_of_match);
> +
> +static struct platform_driver gzvm_driver = {
> +	.probe = gzvm_drv_probe,
> +	.remove = gzvm_drv_remove,
> +	.driver = {
> +			.name = MODULE_NAME,
> +			.owner = THIS_MODULE,
> +			.of_match_table = gzvm_of_match,

Fix indentation please.

> +	},
> +};
> +
> +static int __init gzvm_init(void)

You don't need this at all, as all you're doing here is registering your platform
driver; and that's even a module_init(), so you can simply do

module_platform_driver(gzvm_driver);

...without open coding init/exit functions.

> +{
> +	int ret = 0;
> +
> +	ret = platform_driver_register(&gzvm_driver);
> +	if (ret)
> +		pr_err("Failed to register gzvm driver.\n");
> +
> +	return ret;
> +}
> +
> +static void __exit gzvm_exit(void)
> +{
> +	platform_driver_unregister(&gzvm_driver);
> +}
> +
> +module_init(gzvm_init);
> +module_exit(gzvm_exit);
> +
> +MODULE_AUTHOR("MediaTek");
> +MODULE_DESCRIPTION("GenieZone interface for VMM");
> +MODULE_LICENSE("GPL");
> diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
> new file mode 100644
> index 000000000000..726db866dfcf
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
> @@ -0,0 +1,266 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2023 MediaTek Inc.
> + */
> +
> +#include <asm/sysreg.h>
> +#include <linux/anon_inodes.h>
> +#include <linux/file.h>
> +#include <linux/mm.h>
> +#include <linux/slab.h>
> +#include "gzvm.h"
> +
> +static int gzvm_vcpu_update_one_reg_hyp(struct gzvm_vcpu *vcpu, __u64 reg_id,
> +					bool is_write, __u64 *data)
> +{
> +	struct arm_smccc_res res;
> +	unsigned long a1;
> +	int ret;
> +
> +	a1 = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid);
> +	if (!is_write) {
> +		ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_GET_ONE_REG,
> +					   a1, reg_id, 0, 0, 0, 0, 0, &res);
> +		if (ret == 0)
> +			*data = res.a1;
> +	} else {
> +		ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_ONE_REG,
> +					   a1, reg_id, *data, 0, 0, 0, 0, &res);
> +	}
> +
> +	return ret;
> +}
> +
> +static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *vcpu, void * __user argp,
> +				     bool is_write)
> +{
> +	long ret;
> +	__u64 reg_size, data = 0;
> +	struct gzvm_one_reg reg;
> +	void __user *reg_addr;

Please reorder those variables.

	struct gzvm_one_reg reg;
	void __user *reg_addr;
	u64 data = 0;
	u64 reg_sz;
	long ret;

> +
> +	if (copy_from_user(&reg, argp, sizeof(reg)))
> +		return -EFAULT;
> +	reg_addr = (void __user *)reg.addr;
> +
> +	/* reg id follows KVM's encoding */
> +	switch (reg.id & GZVM_REG_ARM_COPROC_MASK) {
> +	case GZVM_REG_ARM_CORE:
> +		break;
> +	default:
> +		return -EOPNOTSUPP;
> +	}
> +

Make it more readable please...

	reg_size = (reg.id & GZVM_REG_SIZE_MASK) >> GZVM_REG_SIZE_SHIFT;
	reg_size = BIT(reg_size);

...or, even better, you can use bitfield macros, such as FIELD_PREP(), which
would simplify that even more.

> +	reg_size = 1 << ((reg.id & GZVM_REG_SIZE_MASK) >> GZVM_REG_SIZE_SHIFT);
> +	if (is_write) {
> +		if (copy_from_user(&data, reg_addr, reg_size))
> +			return -EFAULT;
> +	}
> +
> +	ret = gzvm_vcpu_update_one_reg_hyp(vcpu, reg.id, is_write, &data);
> +

	if (ret)
		return ret;

	if (!is_write) {
		if (copy_to_user.....)
			return -EFAULT;
	}

	return 0;

> +	if (!is_write && ret == 0) {
> +		if (copy_to_user(reg_addr, &data, reg_size))
> +			return -EFAULT;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest
> + *
> + * @param filp
> + * @param argp pointer to struct gzvm_vcpu_run in userspace
> + * @return long
> + */
> +static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp)
> +{
> +	unsigned long id_tuple;
> +	struct arm_smccc_res res;
> +	bool need_userspace = false;
> +
> +	if (copy_from_user(vcpu->run, argp, sizeof(struct gzvm_vcpu_run)))
> +		return -EFAULT;
> +
> +	if (vcpu->run->immediate_exit == 1)
> +		return -EINTR;
> +
> +	id_tuple = assemble_vm_vcpu_tuple(vcpu->gzvm->vm_id, vcpu->vcpuid);
> +	do {
> +		gzvm_hypcall_wrapper(MT_HVC_GZVM_RUN, id_tuple, 0, 0, 0, 0, 0,
> +				     0, &res);
> +		switch (res.a1) {
> +		case GZVM_EXIT_MMIO:
> +			need_userspace = true;
> +			break;
> +		/*
> +		 * geniezone's responsibility to fill corresponding data
> +		 * structure
> +		 */
> +		case GZVM_EXIT_HVC:

			/* fallthrough */

> +		case GZVM_EXIT_EXCEPTION:

			/* fallthrough */

> +		case GZVM_EXIT_DEBUG:

			/* fallthrough */

> +		case GZVM_EXIT_FAIL_ENTRY:

			/* fallthrough */

> +		case GZVM_EXIT_INTERNAL_ERROR:

			/* fallthrough */

> +		case GZVM_EXIT_SYSTEM_EVENT:

			/* fallthrough */

> +		case GZVM_EXIT_SHUTDOWN:
> +			need_userspace = true;
> +			break;
> +		case GZVM_EXIT_IRQ:
> +			break;
> +		case GZVM_EXIT_UNKNOWN:

			/* fallthrough */

> +		default:
> +			pr_err("vcpu unknown exit\n");

Also, please use dev_err() when possible.

> +			need_userspace = true;
> +			goto out;
> +		}
> +	} while (!need_userspace);
> +
> +out:
> +	if (copy_to_user(argp, vcpu->run, sizeof(struct gzvm_vcpu_run)))
> +		return -EFAULT;
> +	return 0;
> +}
> +
> +static long gzvm_vcpu_ioctl(struct file *filp, unsigned int ioctl,
> +			    unsigned long arg)
> +{
> +	int ret = -ENOTTY;
> +	void __user *argp = (void __user *)arg;
> +	struct gzvm_vcpu *vcpu = filp->private_data;
> +
> +	switch (ioctl) {
> +	case GZVM_RUN:
> +		ret = gzvm_vcpu_run(vcpu, argp);
> +		break;
> +	case GZVM_GET_ONE_REG:
> +		ret = gzvm_vcpu_update_one_reg(vcpu, argp, false /*is_write*/);
> +		break;
> +	case GZVM_SET_ONE_REG:
> +		ret = gzvm_vcpu_update_one_reg(vcpu, argp, true  /*is_write*/);
> +		break;
> +	default:
> +		ret = -ENOTTY;

instead of initializing `ret`, you can just....

		return -ENOTTY;

> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static const struct file_operations gzvm_vcpu_fops = {
> +	.unlocked_ioctl = gzvm_vcpu_ioctl,
> +	.llseek		= noop_llseek,
> +};
> +
> +static int gzvm_destroy_vcpu_hyp(gzvm_id_t vm_id, int vcpuid)
> +{
> +	struct arm_smccc_res res;
> +	unsigned long a1;
> +
> +	a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid);
> +	gzvm_hypcall_wrapper(MT_HVC_GZVM_DESTROY_VCPU, a1, 0, 0, 0, 0, 0, 0,
> +			     &res);
> +
> +	return 0;
> +}
> +
> +/**
> + * @brief call smc to gz hypervisor to create vcpu
> + *
> + * @param run virtual address of vcpu->run
> + * @return int
> + */

kerneldoc please

> +static int gzvm_create_vcpu_hyp(gzvm_id_t vm_id, int vcpuid, void *run)
> +{
> +	struct arm_smccc_res res;
> +	unsigned long a1, a2;
> +	int ret;
> +
> +	a1 = assemble_vm_vcpu_tuple(vm_id, vcpuid);
> +	a2 = (__u64)virt_to_phys(run);
> +	ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VCPU, a1, a2, 0, 0, 0, 0,
> +				   0, &res);
> +
> +	return ret;
> +}
> +
> +/* Caller must hold the vm lock */
> +void gzvm_destroy_vcpu(struct gzvm_vcpu *vcpu)
> +{
> +	if (!vcpu)
> +		return;
> +
> +	gzvm_destroy_vcpu_hyp(vcpu->gzvm->vm_id, vcpu->vcpuid);
> +	/* clean guest's data */
> +	memset(vcpu->run, 0, GZVM_VCPU_RUN_MAP_SIZE);
> +	free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE);
> +	kfree(vcpu);
> +}
> +
> +#define ITOA_MAX_LEN 12 /* Maximum size needed for holding an integer. */

Move definitions at the beginning of the file, please.

> +/**
> + * @brief Allocates an inode for the vcpu.
> + */

Kerneldoc!

> +static int create_vcpu_fd(struct gzvm_vcpu *vcpu)
> +{
> +	/* sizeof("gzvm-vcpu:") + max(strlen(itoa(vcpuid))) + null */
> +	char name[10 + ITOA_MAX_LEN + 1];
> +
> +	snprintf(name, sizeof(name), "gzvm-vcpu:%d", vcpu->vcpuid);
> +	return anon_inode_getfd(name, &gzvm_vcpu_fops, vcpu, O_RDWR | O_CLOEXEC);
> +}
> +
> +/**
> + * @brief GZVM_CREATE_VCPU
> + *
> + * @param cpuid = arg
> + * @return fd of vcpu, negative errno if error occurs
> + */

kerneldoc!!!! :-)

> +int gzvm_vm_ioctl_create_vcpu(struct gzvm *gzvm, u32 cpuid)
> +{
> +	struct gzvm_vcpu *vcpu;
> +	int ret;
> +
> +	if (cpuid >= GZVM_MAX_VCPUS)
> +		return -EINVAL;
> +
> +	vcpu = kzalloc(sizeof(*vcpu), GFP_KERNEL);
> +	if (!vcpu)
> +		return -ENOMEM;
> +
> +	BUILD_BUG_ON((sizeof(*vcpu->run)) > PAGE_SIZE);
> +	BUILD_BUG_ON(sizeof(struct gzvm_vcpu_hwstate) > PAGE_SIZE);

Do you really need to crash the kernel?!

You must have a very, very good reason to do that, please justify that carefully
and also write that as a comment in this function.

> +	/**
> +	 * allocate 2 pages for data sharing between driver and gz hypervisor
> +	 * |- page 0           -|- page 1      -|
> +	 * |gzvm_vcpu_run|......|hwstate|.......|
> +	 */
> +	vcpu->run = alloc_pages_exact(GZVM_VCPU_RUN_MAP_SIZE,
> +				      GFP_KERNEL_ACCOUNT | __GFP_ZERO);
> +	if (!vcpu->run) {
> +		ret = -ENOMEM;
> +		goto free_vcpu;
> +	}
> +	vcpu->hwstate = (void *)vcpu->run + PAGE_SIZE;
> +	vcpu->vcpuid = cpuid;
> +	vcpu->gzvm = gzvm;
> +	mutex_init(&vcpu->lock);
> +
> +	ret = gzvm_create_vcpu_hyp(gzvm->vm_id, vcpu->vcpuid, vcpu->run);
> +	if (ret < 0)
> +		goto free_vcpu_run;
> +
> +	ret = create_vcpu_fd(vcpu);
> +	if (ret < 0)
> +		goto free_vcpu_run;
> +	gzvm->vcpus[cpuid] = vcpu;
> +
> +	return ret;
> +
> +free_vcpu_run:
> +	free_pages_exact(vcpu->run, GZVM_VCPU_RUN_MAP_SIZE);
> +free_vcpu:
> +	kfree(vcpu);
> +	return ret;
> +}
> diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
> new file mode 100644
> index 000000000000..df4ccdc3b7f0
> --- /dev/null
> +++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
> @@ -0,0 +1,444 @@
> +// SPDX-License-Identifier: GPL-2.0
> +/*
> + * Copyright (c) 2023 MediaTek Inc.
> + */
> +
> +#include <linux/anon_inodes.h>
> +#include <linux/arm-smccc.h>
> +#include <linux/file.h>
> +#include <linux/kdev_t.h>
> +#include <linux/kvm_host.h>
> +#include <linux/miscdevice.h>
> +#include <linux/module.h>
> +#include <linux/slab.h>
> +#include <linux/version.h>
> +#include "gzvm.h"
> +
> +static DEFINE_MUTEX(gzvm_list_lock);
> +static LIST_HEAD(gzvm_list);
> +
> +
> +/**
> + * @brief Translate gfn (guest ipa) to pfn (host pa), result is in @pfn
> + *
> + * Leverage KVM's `gfn_to_pfn_memslot`. Because `gfn_to_pfn_memslot` needs
> + * kvm_memory_slot as parameter, this function populates necessary fileds
> + * for calling `gfn_to_pfn_memslot`.
> + *
> + * @retval 0 succeed
> + * @retval -EFAULT failed to convert
> + */

kerneldoc please

> +int gzvm_gfn_to_pfn_memslot(struct gzvm_memslot *memslot, u64 gfn, u64 *pfn)
> +{
> +	hfn_t __pfn;
> +	struct kvm_memory_slot kvm_slot = {0};
> +
> +	kvm_slot.base_gfn = memslot->base_gfn;
> +	kvm_slot.npages = memslot->npages;
> +	kvm_slot.dirty_bitmap = NULL;
> +	kvm_slot.userspace_addr = memslot->userspace_addr;
> +	kvm_slot.flags = memslot->flags;
> +	kvm_slot.id = memslot->slot_id;
> +	kvm_slot.as_id = 0;
> +
> +	__pfn = gfn_to_pfn_memslot(&kvm_slot, gfn);
> +	if (is_error_noslot_pfn(__pfn)) {
> +		*pfn = 0;
> +		return -EFAULT;
> +	}
> +
> +	*pfn = __pfn;
> +	return 0;
> +}
> +
> +/**
> + * @brief Populate pa to buffer until full
> + *
> + * @return int how much pages we've fill in, negative if error
> + */

kerneldoc

> +static int fill_constituents(struct mem_region_addr_range *consti,
> +			     int *consti_cnt, int max_nr_consti, gfn_t gfn,
> +			     u32 total_pages, struct gzvm_memslot *slot)
> +{
> +	int i, nr_pages;
> +	hfn_t pfn, prev_pfn;
> +	gfn_t gfn_end;

	hfn_t pfn, prev_pfn;
	gfn_t gfn_end;
	int nr_pages = 1;
	int i = 0;

> +
> +	if (unlikely(total_pages == 0))
> +		return -EINVAL;
> +	gfn_end = gfn + total_pages;
> +
> +	/* entry 0 */
> +	if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0)
> +		return -EFAULT;
> +	consti[0].address = PFN_PHYS(pfn);
> +	consti[0].pg_cnt = 1;
> +	gfn++;
> +	prev_pfn = pfn;


> +	i = 0;
> +	nr_pages = 1;
stack initialized, remove...

> +	while (i < max_nr_consti && gfn < gfn_end) {
> +		if (gzvm_gfn_to_pfn_memslot(slot, gfn, &pfn) != 0)
> +			return -EFAULT;
> +		if (pfn == (prev_pfn + 1)) {
> +			consti[i].pg_cnt++;
> +		} else {
> +			i++;
> +			if (i >= max_nr_consti)
> +				break;
> +			consti[i].address = PFN_PHYS(pfn);
> +			consti[i].pg_cnt = 1;
> +		}
> +		prev_pfn = pfn;
> +		gfn++;
> +		nr_pages++;
> +	}

	if (i == max_nr_consti)
		i++;

	*consti_cnt = i;

> +	if (i == max_nr_consti)
> +		*consti_cnt = i;
> +	else
> +		*consti_cnt = (i + 1);
> +
> +	return nr_pages;
> +}
> +
> +/**
> + * @brief Register memory region to GZ
> + *
> + * @param gzvm
> + * @param memslot
> + * @return int
> + */

kerneldoc

> +static int
> +register_memslot_addr_range(struct gzvm *gzvm, struct gzvm_memslot *memslot)
> +{
> +	struct gzvm_memory_region_ranges *region;
> +	u32 buf_size;
> +	int max_nr_consti, remain_pages;
> +	gfn_t gfn, gfn_end;

	struct gzvm_memory_region_ranges *region;
	gfn_t gfn, gfn_end;
	int max_nr_consti, remain_pages;
	u32 buf_size;

> +
> +	buf_size = PAGE_SIZE * 2;
> +	region = alloc_pages_exact(buf_size, GFP_KERNEL);
> +	if (!region)
> +		return -ENOMEM;
> +	max_nr_consti = (buf_size - sizeof(*region)) /
> +			sizeof(struct mem_region_addr_range);
> +
> +	region->slot = memslot->slot_id;
> +	remain_pages = memslot->npages;
> +	gfn = memslot->base_gfn;
> +	gfn_end = gfn + remain_pages;
> +	while (gfn < gfn_end) {
> +		struct arm_smccc_res res;
> +		int nr_pages;
> +
> +		nr_pages = fill_constituents(region->constituents,
> +					     &region->constituent_cnt,
> +					     max_nr_consti, gfn,
> +					     remain_pages, memslot);
> +		region->gpa = PFN_PHYS(gfn);
> +		region->total_pages = nr_pages;
> +
> +		remain_pages -= nr_pages;
> +		gfn += nr_pages;
> +
> +		gzvm_hypcall_wrapper(MT_HVC_GZVM_SET_MEMREGION, gzvm->vm_id,
> +			     buf_size, virt_to_phys(region), 0, 0, 0, 0, &res);
> +
> +		if (res.a0 != 0) {
> +			pr_err("Failed to register memregion to hypervisor\n");

dev_err() please

> +			free_pages_exact(region, buf_size);
> +			return -EFAULT;
> +		}
> +	}
> +	free_pages_exact(region, buf_size);
> +	return 0;
> +}
> +
> +/**
> + * @brief Set memory region of guest
> + *
> + * @param gzvm struct gzvm
> + * @param mem struct gzvm_userspace_memory_region: input from user
> + * @retval -EXIO memslot is out-of-range
> + * @retval -EFAULT  cannot find corresponding vma
> + * @retval -EINVAL  region size and vma size does not match
> + */

kerneldoc

> +static int gzvm_vm_ioctl_set_memory_region(struct gzvm *gzvm,
> +				struct gzvm_userspace_memory_region *mem)
> +{
> +	struct vm_area_struct *vma;
> +	struct gzvm_memslot *memslot;
> +	unsigned long size;
> +	__u32 slot;
> +
> +	slot = mem->slot;
> +	if (slot >= GZVM_MAX_MEM_REGION)
> +		return -ENXIO;
> +	memslot = &gzvm->memslot[slot];
> +
> +	vma = vma_lookup(gzvm->mm, mem->userspace_addr);
> +	if (!vma)
> +		return -EFAULT;
> +
> +	size = vma->vm_end - vma->vm_start;
> +	if (size != mem->memory_size)
> +		return -EINVAL;
> +
> +	memslot->base_gfn = __phys_to_pfn(mem->guest_phys_addr);
> +	memslot->npages = size >> PAGE_SHIFT;
> +	memslot->userspace_addr = mem->userspace_addr;
> +	memslot->vma = vma;
> +	memslot->flags = mem->flags;
> +	memslot->slot_id = mem->slot;
> +	return register_memslot_addr_range(gzvm, memslot);
> +}
> +
> +static int gzvm_vm_enable_cap_hyp(struct gzvm *gzvm,
> +				  struct gzvm_enable_cap *cap,
> +				  struct arm_smccc_res *res)

Please don't introduce new functions doing just one call.

> +{
> +	int ret;
> +
> +	ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_ENABLE_CAP, gzvm->vm_id,
> +				   cap->cap, cap->args[0], cap->args[1],
> +				   cap->args[2], cap->args[3], cap->args[4],
> +				   res); > +	return ret;
> +}
> +
> +/**
> + * @brief Get pvmfw size from hypervisor, return in x1, and return to userspace
> + *        in args[1].
> + * @retval 0 succeed
> + * @retval -EINVAL hypervisor return invalid results
> + * @retval -EFAULT fail to copy back to userspace buffer
> + */

kerneldoc ..... here and everywhere else, I will stop saying that every time.

> +static int gzvm_vm_ioctl_get_pvmfw_size(struct gzvm *gzvm,
> +					struct gzvm_enable_cap *cap,
> +					void __user *argp)
> +{
> +	struct arm_smccc_res res = {0};
> +
> +	if (gzvm_vm_enable_cap_hyp(gzvm, cap, &res) != 0)
> +		return -EINVAL;
> +
> +	cap->args[1] = res.a1;
> +	if (copy_to_user(argp, cap, sizeof(*cap)))
> +		return -EFAULT;
> +
> +	return 0;
> +}
> +
> +/**
> + * @brief Proceed GZVM_CAP_ARM_PROTECTED_VM's subcommands
> + * @retval 0 succeed
> + * @retval -EINVAL invalid subcommand or arguments
> + */
> +static int gzvm_vm_ioctl_cap_pvm(struct gzvm *gzvm, struct gzvm_enable_cap *cap,
> +				 void __user *argp)
> +{
> +	int ret = -EINVAL;
> +	struct arm_smccc_res res = {0};
> +
> +	switch (cap->args[0]) {
> +	case GZVM_CAP_ARM_PVM_SET_PVMFW_IPA:
> +		ret = gzvm_vm_enable_cap_hyp(gzvm, cap, &res);
> +		break;
> +	case GZVM_CAP_ARM_PVM_GET_PVMFW_SIZE:
> +		ret = gzvm_vm_ioctl_get_pvmfw_size(gzvm, cap, argp);
> +		break;
> +	default:
> +		ret = -EINVAL;

		return -EINVAL;

> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +static int gzvm_vm_ioctl_enable_cap(struct gzvm *gzvm,
> +				    struct gzvm_enable_cap *cap,
> +				    void __user *argp)
> +{
> +	int ret = -EINVAL;
> +
> +	switch (cap->cap) {
> +	case GZVM_CAP_ARM_PROTECTED_VM:
> +		ret = gzvm_vm_ioctl_cap_pvm(gzvm, cap, argp);
> +		break;
> +	default:
> +		ret = -EINVAL;

		return -EINVAL;

> +		break;
> +	}
> +
> +	return ret;
> +}
> +
> +/**
> + * @brief ioctl handler of VM FD
> + */
> +static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl,
> +			  unsigned long arg)
> +{
> +	long ret = -ENOTTY;
> +	void __user *argp = (void __user *)arg;
> +	struct gzvm *gzvm = filp->private_data;
> +
> +	switch (ioctl) {
> +	case GZVM_CHECK_EXTENSION:
> +		ret = gzvm_dev_ioctl_check_extension(gzvm, arg);
> +		break;
> +	case GZVM_CREATE_VCPU:
> +		ret = gzvm_vm_ioctl_create_vcpu(gzvm, arg);
> +		break;
> +	case GZVM_SET_USER_MEMORY_REGION: {
> +		struct gzvm_userspace_memory_region userspace_mem;
> +
> +		ret = -EFAULT;
> +		if (copy_from_user(&userspace_mem, argp,
> +						sizeof(userspace_mem)))
> +			goto out;
> +		ret = gzvm_vm_ioctl_set_memory_region(gzvm, &userspace_mem);
> +		break;
> +	}
> +	case GZVM_ENABLE_CAP: {
> +		struct gzvm_enable_cap cap;
> +
> +		ret = -EFAULT;
> +		if (copy_from_user(&cap, argp, sizeof(cap)))
> +			goto out;
> +
> +		ret = gzvm_vm_ioctl_enable_cap(gzvm, &cap, argp);
> +		break;
> +	}
> +	default:
> +		ret = -ENOTTY;

		return -ENOTTY;

> +	}
> +out:
> +	return ret;
> +}
> +

..snip..

> +
> +static void gzvm_destroy_vm(struct gzvm *gzvm)
> +{ > +	pr_info("VM-%u is going to be destroyed\n", gzvm->vm_id);

	dev_info() please.

> +
> +	mutex_lock(&gzvm->lock);
> +
> +	gzvm_destroy_vcpus(gzvm);
> +	gzvm_destroy_vm_hyp(gzvm->vm_id);
> +
> +	mutex_lock(&gzvm_list_lock);
> +	list_del(&gzvm->vm_list);
> +	mutex_unlock(&gzvm_list_lock);
> +
> +	mutex_unlock(&gzvm->lock);
> +
> +	kfree(gzvm);
> +}
> +
> +static int gzvm_vm_release(struct inode *inode, struct file *filp)
> +{
> +	struct gzvm *gzvm = filp->private_data;
> +
> +	gzvm_destroy_vm(gzvm);
> +	return 0;
> +}
> +
> +static const struct file_operations gzvm_vm_fops = {
> +	.release        = gzvm_vm_release,
> +	.unlocked_ioctl = gzvm_vm_ioctl,
> +	.llseek		= noop_llseek,
> +};
> +
> +static int gzvm_create_vm_hyp(void)
> +{
> +	struct arm_smccc_res res;
> +
> +	gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_VM, 0, 0, 0, 0, 0, 0, 0, &res);
> +
> +	if (res.a0 != 0)
> +		return -EFAULT;
> +	return res.a1;
> +}
> +
> +static struct gzvm *gzvm_create_vm(unsigned long vm_type)
> +{
> +	int ret;
> +	struct gzvm *gzvm;
> +
> +	gzvm = kzalloc(sizeof(struct gzvm), GFP_KERNEL);
> +	if (!gzvm)
> +		return ERR_PTR(-ENOMEM);
> +
> +	ret = gzvm_create_vm_hyp();
> +	if (ret < 0)
> +		goto err;

You're doing that only once in this function, so you don't need a label.

	if (ret) {
		kfree(gzvm);
		return ERR_PTR(ret);
	}

> +
> +	gzvm->vm_id = ret;
> +	gzvm->mm = current->mm;
> +	mutex_init(&gzvm->lock);
> +	INIT_LIST_HEAD(&gzvm->devices);
> +	mutex_init(&gzvm->irq_lock);
> +	pr_info("VM-%u is created\n", gzvm->vm_id);
> +
> +	mutex_lock(&gzvm_list_lock);
> +	list_add(&gzvm->vm_list, &gzvm_list);
> +	mutex_unlock(&gzvm_list_lock);
> +
> +	return gzvm;
> +
> +err:
> +	kfree(gzvm);
> +	return ERR_PTR(ret);
> +}
> +
> +/**
> + * @brief create vm fd
> + *
> + * @param vm_type
> + * @return int fd of vm, negative if error
> + */
> +int gzvm_dev_ioctl_create_vm(unsigned long vm_type)
> +{
> +	struct gzvm *gzvm;
> +	int ret;
> +
> +	gzvm = gzvm_create_vm(vm_type);
	if (IS_ERR(gzvm))
		return PTR_ERR(gzvm);

> +	if (IS_ERR(gzvm)) {
> +		ret = PTR_ERR(gzvm);
> +		goto error;
> +	}
> +
> +	ret = anon_inode_getfd("gzvm-vm", &gzvm_vm_fops, gzvm,
> +			       O_RDWR | O_CLOEXEC);
	if (ret)
		return ret;

	return 0;
}


> +	if (ret < 0)
> +		goto error;
> +
> +error:
> +	return ret;
> +}

...but anyway, all of this must go to drivers/virt/.

Regards,
Angelo

[PATCH v1 4/6] soc: mediatek: virt: geniezone: Introduce irqchip for virtual interrupt injection

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

Enable GenieZone to handle virtual interrupt injection request.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 drivers/soc/mediatek/virt/geniezone/Makefile  |  2 +-
 drivers/soc/mediatek/virt/geniezone/gzvm.h    |  5 +
 .../mediatek/virt/geniezone/gzvm_irqchip.c    | 94 +++++++++++++++++++
 .../soc/mediatek/virt/geniezone/gzvm_vcpu.c   |  6 ++
 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c | 77 +++++++++++++++
 include/uapi/linux/gzvm_common.h              | 32 +++++++
 6 files changed, 215 insertions(+), 1 deletion(-)
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c

diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile
index e1dfbb9c568d..d2302a3a93fc 100644
--- a/drivers/soc/mediatek/virt/geniezone/Makefile
+++ b/drivers/soc/mediatek/virt/geniezone/Makefile
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
-gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o
+gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o
 
 obj-$(CONFIG_MTK_GZVM) += gzvm.o
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h
index 43f215d4b0da..89cea5441a2d 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm.h
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h
@@ -98,6 +98,11 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
 #define MT_HVC_GZVM_PROBE		GZVM_HCALL_ID(GZVM_FUNC_PROBE)
 #define MT_HVC_GZVM_ENABLE_CAP		GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP)
 
+void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu);
+int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type,
+			 u32 irq, bool level);
+int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx,
+			 u32 spi_irq, bool level);
 int gz_err_to_errno(unsigned long err);
 
 #endif /* __GZVM_H__ */
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
new file mode 100644
index 000000000000..7aa5868a221c
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
@@ -0,0 +1,94 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2023 MediaTek Inc.
+ */
+
+#include <linux/irqchip/arm-gic-v3.h>
+#include <kvm/arm_vgic.h>
+
+#include "gzvm.h"
+
+/**
+ * @brief check all LRs synced from gz hypervisor
+ * Traverse all LRs, see if any EOIed vint, notify_acked_irq if any.
+ * GZ does not fold/unfold everytime KVM_RUN, so we have to traverse all saved
+ * LRs. It will not takes much more time comparing to fold/unfold everytime
+ * GZVM_RUN, because there are only few LRs.
+ */
+void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu)
+{
+	int i;
+
+	for (i = 0; i < vcpu->hwstate->nr_lrs; i++) {
+		uint64_t lr_val = vcpu->hwstate->lr[i];
+		/* 0 means unused */
+		if (!lr_val)
+			continue;
+	}
+}
+
+/**
+ * @brief Check the irq number and irq_type are matched
+ */
+static bool is_irq_valid(u32 irq, u32 irq_type)
+{
+	switch (irq_type) {
+	case GZVM_IRQ_TYPE_CPU:	/*  0 ~ 15: SGI */
+		if (likely(irq <= GZVM_IRQ_CPU_FIQ))
+			return true;
+		break;
+	case GZVM_IRQ_TYPE_PPI:	/* 16 ~ 31: PPI */
+		if (likely(irq >= VGIC_NR_SGIS && irq < VGIC_NR_PRIVATE_IRQS))
+			return true;
+		break;
+	case GZVM_IRQ_TYPE_SPI:	/* 32 ~ : SPT */
+		if (likely(irq >= VGIC_NR_PRIVATE_IRQS))
+			return true;
+		break;
+	default:
+		return false;
+	}
+	return false;
+}
+
+/**
+ * @brief Inject virtual interrupt to a VM
+ *
+ * @param gzvm
+ * @param vcpu_idx: vcpu index, only valid if PPI
+ * @param irq: irq number
+ * @param irq_type
+ * @param level, true: 1; false: 0
+ */
+int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type,
+			 u32 irq, bool level)
+{
+	unsigned long a1 = assemble_vm_vcpu_tuple(gzvm->vm_id, vcpu_idx);
+	struct arm_smccc_res res;
+
+	if (!unlikely(is_irq_valid(irq, irq_type)))
+		return -EINVAL;
+
+	gzvm_hypcall_wrapper(MT_HVC_GZVM_IRQ_LINE, a1, irq, level,
+			     0, 0, 0, 0, &res);
+	if (res.a0) {
+		pr_err("Failed to set IRQ level (%d) to irq#%u on vcpu %d with ret=%d\n",
+		       level, irq, vcpu_idx, (int)res.a0);
+		return -EFAULT;
+	}
+
+	return 0;
+}
+
+/**
+ * @brief Inject virtual spi interrupt
+ *
+ * @param spi_irq This is spi interrupt number (starts from 0 instead of 32)
+ * @return 0 succeed, other negative values are error
+ */
+int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx,
+			 u32 spi_irq, bool level)
+{
+	return gzvm_vgic_inject_irq(gzvm, 0, GZVM_IRQ_TYPE_SPI,
+				    spi_irq + VGIC_NR_PRIVATE_IRQS, level);
+}
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
index 726db866dfcf..5f2e24d13c41 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
@@ -67,6 +67,11 @@ static long gzvm_vcpu_update_one_reg(struct gzvm_vcpu *vcpu, void * __user argp,
 	return ret;
 }
 
+static void gzvm_sync_hwstate(struct gzvm_vcpu *vcpu)
+{
+	gzvm_sync_vgic_state(vcpu);
+}
+
 /**
  * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest
  *
@@ -115,6 +120,7 @@ static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp)
 			need_userspace = true;
 			goto out;
 		}
+		gzvm_sync_hwstate(vcpu);
 	} while (!need_userspace);
 
 out:
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
index df4ccdc3b7f0..7895f40b23eb 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
@@ -193,6 +193,69 @@ static int gzvm_vm_ioctl_set_memory_region(struct gzvm *gzvm,
 	return register_memslot_addr_range(gzvm, memslot);
 }
 
+static int gzvm_vm_ioctl_irq_line(struct gzvm *gzvm,
+				  struct gzvm_irq_level *irq_level)
+{
+	u32 irq = irq_level->irq;
+	unsigned int irq_type, vcpu_idx, irq_num;
+	bool level = irq_level->level;
+
+	irq_type = (irq >> GZVM_IRQ_TYPE_SHIFT) & GZVM_IRQ_TYPE_MASK;
+	vcpu_idx = (irq >> GZVM_IRQ_VCPU_SHIFT) & GZVM_IRQ_VCPU_MASK;
+	vcpu_idx += ((irq >> GZVM_IRQ_VCPU2_SHIFT) & GZVM_IRQ_VCPU2_MASK) *
+		(GZVM_IRQ_VCPU_MASK + 1);
+	irq_num = (irq >> GZVM_IRQ_NUM_SHIFT) & GZVM_IRQ_NUM_MASK;
+
+	return gzvm_vgic_inject_irq(gzvm, vcpu_idx, irq_num, irq_type, level);
+}
+
+static int gzvm_vm_ioctl_create_device(struct gzvm *gzvm, void __user *argp)
+{
+	struct gzvm_create_device *gzvm_dev;
+	void *dev_data = NULL;
+	struct arm_smccc_res res = {0};
+	int ret;
+
+	gzvm_dev = (struct gzvm_create_device *)alloc_pages_exact(PAGE_SIZE,
+								  GFP_KERNEL);
+	if (!gzvm_dev)
+		return -ENOMEM;
+	if (copy_from_user(gzvm_dev, argp, sizeof(*gzvm_dev))) {
+		ret = -EFAULT;
+		goto err_free_dev;
+	}
+
+	if (gzvm_dev->attr_addr != 0 && gzvm_dev->attr_size != 0) {
+		size_t attr_size = gzvm_dev->attr_size;
+		void __user *attr_addr = (void __user *)gzvm_dev->attr_addr;
+
+		/* Size of device specific data should not be over a page. */
+		if (attr_size > PAGE_SIZE)
+			return -EINVAL;
+
+		dev_data = alloc_pages_exact(attr_size, GFP_KERNEL);
+		if (!dev_data) {
+			ret = -ENOMEM;
+			goto err_free_dev;
+		}
+
+		if (copy_from_user(dev_data, attr_addr, attr_size)) {
+			ret = -EFAULT;
+			goto err_free_dev_data;
+		}
+		gzvm_dev->attr_addr = virt_to_phys(dev_data);
+	}
+
+	ret = gzvm_hypcall_wrapper(MT_HVC_GZVM_CREATE_DEVICE, gzvm->vm_id,
+				   virt_to_phys(gzvm_dev), 0, 0, 0, 0, 0, &res);
+err_free_dev_data:
+	if (dev_data)
+		free_pages_exact(dev_data, 0);
+err_free_dev:
+	free_pages_exact(gzvm_dev, 0);
+	return ret;
+}
+
 static int gzvm_vm_enable_cap_hyp(struct gzvm *gzvm,
 				  struct gzvm_enable_cap *cap,
 				  struct arm_smccc_res *res)
@@ -300,6 +363,20 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl,
 		ret = gzvm_vm_ioctl_set_memory_region(gzvm, &userspace_mem);
 		break;
 	}
+	case GZVM_IRQ_LINE: {
+		struct gzvm_irq_level irq_event;
+
+		ret = -EFAULT;
+		if (copy_from_user(&irq_event, argp, sizeof(irq_event)))
+			goto out;
+
+		ret = gzvm_vm_ioctl_irq_line(gzvm, &irq_event);
+		break;
+	}
+	case GZVM_CREATE_DEVICE: {
+		ret = gzvm_vm_ioctl_create_device(gzvm, argp);
+		break;
+	}
 	case GZVM_ENABLE_CAP: {
 		struct gzvm_enable_cap cap;
 
diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h
index aa97438bab8c..6808cfe59450 100644
--- a/include/uapi/linux/gzvm_common.h
+++ b/include/uapi/linux/gzvm_common.h
@@ -183,6 +183,38 @@ struct gzvm_userspace_memory_region {
 #define GZVM_IRQ_CPU_IRQ		0
 #define GZVM_IRQ_CPU_FIQ		1
 
+struct gzvm_irq_level {
+	union {
+		__u32 irq;
+		__s32 status;
+	};
+	__u32 level;
+};
+#define GZVM_IRQ_LINE              _IOW(GZVM_IOC_MAGIC,  0x61, \
+					struct gzvm_irq_level)
+
+enum gzvm_device_type {
+	GZVM_DEV_TYPE_ARM_VGIC_V3_DIST,
+	GZVM_DEV_TYPE_ARM_VGIC_V3_REDIST,
+	GZVM_DEV_TYPE_MAX,
+};
+
+struct gzvm_create_device {
+	__u32 dev_type;			/* device type */
+	__u32 id;			/* out: device id */
+	__u64 flags;			/* device specific flags */
+	__u64 dev_addr;			/* device ipa address in VM's view */
+	__u64 dev_reg_size;		/* device register range size */
+	/*
+	 * If user -> kernel, this is user virtual address of device specific
+	 * attributes (if needed). If kernel->hypervisor, this is ipa.
+	 */
+	__u64 attr_addr;
+	__u64 attr_size;		/* size of device specific attributes */
+};
+#define GZVM_CREATE_DEVICE	   _IOWR(GZVM_IOC_MAGIC,  0xe0, \
+					struct gzvm_create_device)
+
 /*
  * ioctls for vcpu fds
  */
-- 
2.18.0

[PATCH v1 5/6] soc: mediatek: virt: geniezone: Add ioeventfd support

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

ioeventfd leverages eventfd to provide asynchronous notification mechanism
for VMM. VMM can register a mmio address and bind with an eventfd. Once a
mmio trap occurs on this registered region, its corresponding eventfd will
be notified.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 drivers/soc/mediatek/virt/geniezone/Makefile  |   2 +-
 drivers/soc/mediatek/virt/geniezone/gzvm.h    |  10 +
 .../mediatek/virt/geniezone/gzvm_eventfd.c    | 252 ++++++++++++++++++
 .../soc/mediatek/virt/geniezone/gzvm_vcpu.c   |  26 +-
 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c |  14 +
 include/uapi/linux/gzvm_common.h              |  24 ++
 6 files changed, 326 insertions(+), 2 deletions(-)
 create mode 100644 drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c

diff --git a/drivers/soc/mediatek/virt/geniezone/Makefile b/drivers/soc/mediatek/virt/geniezone/Makefile
index d2302a3a93fc..dc6c760231e2 100644
--- a/drivers/soc/mediatek/virt/geniezone/Makefile
+++ b/drivers/soc/mediatek/virt/geniezone/Makefile
@@ -1,5 +1,5 @@
 # SPDX-License-Identifier: GPL-2.0-only
 
-gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o
+gzvm-y := gzvm_main.o gzvm_vm.o gzvm_vcpu.o gzvm_irqchip.o gzvm_eventfd.o
 
 obj-$(CONFIG_MTK_GZVM) += gzvm.o
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h
index 89cea5441a2d..b0edf56c2832 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm.h
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h
@@ -37,6 +37,7 @@ struct gzvm {
 	struct list_head devices;
 	gzvm_id_t vm_id;
 
+	struct list_head ioevents;
 	struct {
 		spinlock_t        lock;
 		struct list_head  items;
@@ -98,6 +99,10 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
 #define MT_HVC_GZVM_PROBE		GZVM_HCALL_ID(GZVM_FUNC_PROBE)
 #define MT_HVC_GZVM_ENABLE_CAP		GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP)
 
+int gzvm_init_ioeventfd(struct gzvm *gzvm);
+int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args);
+bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len,
+			const void *val);
 void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu);
 int gzvm_vgic_inject_irq(struct gzvm *gzvm, unsigned int vcpu_idx, u32 irq_type,
 			 u32 irq, bool level);
@@ -105,4 +110,9 @@ int gzvm_vgic_inject_spi(struct gzvm *gzvm, unsigned int vcpu_idx,
 			 u32 spi_irq, bool level);
 int gz_err_to_errno(unsigned long err);
 
+#include <linux/eventfd.h>
+void eventfd_ctx_do_read(struct eventfd_ctx *ctx, __u64 *cnt);
+struct vm_area_struct *vma_lookup(struct mm_struct *mm, unsigned long addr);
+void add_wait_queue_priority(struct wait_queue_head *wq_head, struct wait_queue_entry *wq_entry);
+
 #endif /* __GZVM_H__ */
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
new file mode 100644
index 000000000000..63f042fefbe5
--- /dev/null
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
@@ -0,0 +1,252 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2022 MediaTek Inc.
+ */
+
+#include <linux/eventfd.h>
+#include <linux/file.h>
+#include <linux/syscalls.h>
+#include <linux/wait.h>
+#include <linux/poll.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include "gzvm.h"
+
+struct gzvm_ioevent {
+	struct list_head list;
+	__u64 addr;
+	__u32 len;
+	struct eventfd_ctx  *evt_ctx;
+	__u64 datamatch;
+	bool wildcard;
+};
+
+/* assumes gzvm->slots_lock held */
+static bool
+ioeventfd_check_collision(struct gzvm *gzvm, struct gzvm_ioevent *p)
+{
+	struct gzvm_ioevent *_p;
+
+	list_for_each_entry(_p, &gzvm->ioevents, list)
+		if (_p->addr == p->addr &&
+		    (!_p->len || !p->len ||
+		     (_p->len == p->len &&
+		      (_p->wildcard || p->wildcard ||
+		       _p->datamatch == p->datamatch))))
+			return true;
+
+	return false;
+}
+
+static void gzvm_ioevent_release(struct gzvm_ioevent *p)
+{
+	eventfd_ctx_put(p->evt_ctx);
+	list_del(&p->list);
+	kfree(p);
+}
+
+static bool
+gzvm_ioevent_in_range(struct gzvm_ioevent *p, __u64 addr, int len,
+		      const void *val)
+{
+	u64 _val;
+
+	if (addr != p->addr)
+		/* address must be precise for a hit */
+		return false;
+
+	if (!p->len)
+		/* length = 0 means only look at the address, so always a hit */
+		return true;
+
+	if (len != p->len)
+		/* address-range must be precise for a hit */
+		return false;
+
+	if (p->wildcard)
+		/* all else equal, wildcard is always a hit */
+		return true;
+
+	/* otherwise, we have to actually compare the data */
+
+	WARN_ON_ONCE(!IS_ALIGNED((unsigned long)val, len));
+
+	switch (len) {
+	case 1:
+		_val = *(u8 *)val;
+		break;
+	case 2:
+		_val = *(u16 *)val;
+		break;
+	case 4:
+		_val = *(u32 *)val;
+		break;
+	case 8:
+		_val = *(u64 *)val;
+		break;
+	default:
+		return false;
+	}
+
+	return _val == p->datamatch;
+}
+
+static int gzvm_deassign_ioeventfd(struct gzvm *gzvm,
+				   struct gzvm_ioeventfd *args)
+{
+	struct gzvm_ioevent *p, *tmp;
+	struct eventfd_ctx *evt_ctx;
+	int ret = -ENOENT;
+	bool wildcard;
+
+	evt_ctx = eventfd_ctx_fdget(args->fd);
+	if (IS_ERR(evt_ctx))
+		return PTR_ERR(evt_ctx);
+
+	wildcard = !(args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH);
+
+	mutex_lock(&gzvm->lock);
+
+	list_for_each_entry_safe(p, tmp, &gzvm->ioevents, list) {
+		if (p->evt_ctx != evt_ctx  ||
+		    p->addr != args->addr  ||
+		    p->len != args->len ||
+		    p->wildcard != wildcard)
+			continue;
+
+		if (!p->wildcard && p->datamatch != args->datamatch)
+			continue;
+
+		gzvm_ioevent_release(p);
+		ret = 0;
+		break;
+	}
+
+	mutex_unlock(&gzvm->lock);
+
+	/* got in the front of this function */
+	eventfd_ctx_put(evt_ctx);
+
+	return ret;
+}
+
+static int gzvm_assign_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args)
+{
+	struct eventfd_ctx *evt_ctx;
+	struct gzvm_ioevent *evt;
+	int ret;
+
+	evt_ctx = eventfd_ctx_fdget(args->fd);
+	if (IS_ERR(evt_ctx))
+		return PTR_ERR(evt_ctx);
+
+	evt = kmalloc(sizeof(*evt), GFP_KERNEL);
+	if (!evt)
+		return -ENOMEM;
+	*evt = (struct gzvm_ioevent) {
+		.addr = args->addr,
+		.len = args->len,
+		.evt_ctx = evt_ctx,
+	};
+	if (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH) {
+		evt->datamatch = args->datamatch;
+		evt->wildcard = false;
+	} else {
+		evt->wildcard = true;
+	}
+
+	if (ioeventfd_check_collision(gzvm, evt)) {
+		ret = -EEXIST;
+		goto err_free;
+	}
+
+	mutex_lock(&gzvm->lock);
+	list_add_tail(&evt->list, &gzvm->ioevents);
+	mutex_unlock(&gzvm->lock);
+
+	return 0;
+
+err_free:
+	kfree(evt);
+	eventfd_ctx_put(evt_ctx);
+	return ret;
+}
+
+/**
+ * @brief Check user arguments is valid
+ *
+ * @param args
+ * @retval true valid arguments
+ * @retval false invalid arguments
+ */
+static bool gzvm_ioeventfd_check_valid(struct gzvm_ioeventfd *args)
+{
+	/* must be natural-word sized, or 0 to ignore length */
+	switch (args->len) {
+	case 0:
+	case 1:
+	case 2:
+	case 4:
+	case 8:
+		break;
+	default:
+		return false;
+	}
+
+	/* check for range overflow */
+	if (args->addr + args->len < args->addr)
+		return false;
+
+	/* check for extra flags that we don't understand */
+	if (args->flags & ~GZVM_IOEVENTFD_VALID_FLAG_MASK)
+		return false;
+
+	/* ioeventfd with no length can't be combined with DATAMATCH */
+	if (!args->len && (args->flags & GZVM_IOEVENTFD_FLAG_DATAMATCH))
+		return false;
+
+	/* gzvm does not support pio bus ioeventfd */
+	if (args->flags & GZVM_IOEVENTFD_FLAG_PIO)
+		return false;
+
+	return true;
+}
+
+/**
+ * @brief GZVM_IOEVENTFD, register ioevent to ioevent list
+ */
+int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args)
+{
+	if (gzvm_ioeventfd_check_valid(args) == false)
+		return -EINVAL;
+
+	if (args->flags & GZVM_IOEVENTFD_FLAG_DEASSIGN)
+		return gzvm_deassign_ioeventfd(gzvm, args);
+	return gzvm_assign_ioeventfd(gzvm, args);
+}
+
+/**
+ * @brief Travers this vm's registered ioeventfd to see if need notifying it
+ * @retval true if this io is already sent to ioeventfd's listner
+ * @retval false if we cannot find any ioeventfd registering this mmio write
+ */
+bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len,
+			const void *val)
+{
+	struct gzvm_ioevent *e;
+
+	list_for_each_entry(e, &vcpu->gzvm->ioevents, list) {
+		if (gzvm_ioevent_in_range(e, addr, len, val)) {
+			eventfd_signal(e->evt_ctx, 1);
+			return true;
+		}
+	}
+	return false;
+}
+
+int gzvm_init_ioeventfd(struct gzvm *gzvm)
+{
+	INIT_LIST_HEAD(&gzvm->ioevents);
+
+	return 0;
+}
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
index 5f2e24d13c41..14a9fe14e79d 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vcpu.c
@@ -72,6 +72,29 @@ static void gzvm_sync_hwstate(struct gzvm_vcpu *vcpu)
 	gzvm_sync_vgic_state(vcpu);
 }
 
+/**
+ * @brief try to handle mmio in kernel space
+ *
+ * @param vcpu
+ * @return true this mmio exit has been processed.
+ * @return false this mmio exit has not been processed, require userspace.
+ */
+static bool gzvm_vcpu_handle_mmio(struct gzvm_vcpu *vcpu)
+{
+	__u64 addr;
+	__u32 len;
+	const void *val_ptr;
+
+	/* So far, we don't have in-kernel mmio read handler */
+	if (!vcpu->run->mmio.is_write)
+		return false;
+	addr = vcpu->run->mmio.phys_addr;
+	len = vcpu->run->mmio.size;
+	val_ptr = &vcpu->run->mmio.data;
+
+	return gzvm_ioevent_write(vcpu, addr, len, val_ptr);
+}
+
 /**
  * @brief Handle vcpu run ioctl, entry point to guest and exit point from guest
  *
@@ -97,7 +120,8 @@ static long gzvm_vcpu_run(struct gzvm_vcpu *vcpu, void * __user argp)
 				     0, &res);
 		switch (res.a1) {
 		case GZVM_EXIT_MMIO:
-			need_userspace = true;
+			if (!gzvm_vcpu_handle_mmio(vcpu))
+				need_userspace = true;
 			break;
 		/*
 		 * geniezone's responsibility to fill corresponding data
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
index 7895f40b23eb..cceaa532c2ce 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
@@ -377,6 +377,15 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl,
 		ret = gzvm_vm_ioctl_create_device(gzvm, argp);
 		break;
 	}
+	case GZVM_IOEVENTFD: {
+		struct gzvm_ioeventfd data;
+
+		ret = -EFAULT;
+		if (copy_from_user(&data, argp, sizeof(data)))
+			goto out;
+		ret = gzvm_ioeventfd(gzvm, &data);
+		break;
+	}
 	case GZVM_ENABLE_CAP: {
 		struct gzvm_enable_cap cap;
 
@@ -481,6 +490,11 @@ static struct gzvm *gzvm_create_vm(unsigned long vm_type)
 	mutex_init(&gzvm->lock);
 	INIT_LIST_HEAD(&gzvm->devices);
 	mutex_init(&gzvm->irq_lock);
+	ret = gzvm_init_ioeventfd(gzvm);
+	if (ret) {
+		pr_err("Failed to initialize ioeventfd\n");
+		goto err;
+	}
 	pr_info("VM-%u is created\n", gzvm->vm_id);
 
 	mutex_lock(&gzvm_list_lock);
diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h
index 6808cfe59450..6dbaddd77ec7 100644
--- a/include/uapi/linux/gzvm_common.h
+++ b/include/uapi/linux/gzvm_common.h
@@ -193,6 +193,30 @@ struct gzvm_irq_level {
 #define GZVM_IRQ_LINE              _IOW(GZVM_IOC_MAGIC,  0x61, \
 					struct gzvm_irq_level)
 
+enum {
+	gzvm_ioeventfd_flag_nr_datamatch,
+	gzvm_ioeventfd_flag_nr_pio,
+	gzvm_ioeventfd_flag_nr_deassign,
+	gzvm_ioeventfd_flag_nr_max,
+};
+
+#define GZVM_IOEVENTFD_FLAG_DATAMATCH (1 << gzvm_ioeventfd_flag_nr_datamatch)
+#define GZVM_IOEVENTFD_FLAG_PIO       (1 << gzvm_ioeventfd_flag_nr_pio)
+#define GZVM_IOEVENTFD_FLAG_DEASSIGN  (1 << gzvm_ioeventfd_flag_nr_deassign)
+#define GZVM_IOEVENTFD_VALID_FLAG_MASK  ((1 << gzvm_ioeventfd_flag_nr_max) - 1)
+
+struct gzvm_ioeventfd {
+	__u64 datamatch;
+	__u64 addr;        /* legal pio/mmio address */
+	__u32 len;         /* 1, 2, 4, or 8 bytes; or 0 to ignore length */
+	__s32 fd;
+	__u32 flags;
+	__u8  pad[36];
+};
+
+#define GZVM_IOEVENTFD             _IOW(GZVM_IOC_MAGIC,  0x79, \
+					struct gzvm_ioeventfd)
+
 enum gzvm_device_type {
 	GZVM_DEV_TYPE_ARM_VGIC_V3_DIST,
 	GZVM_DEV_TYPE_ARM_VGIC_V3_REDIST,
-- 
2.18.0

[PATCH v1 6/6] soc: mediatek: virt: geniezone: Add irqfd support

[Yi-De Wu]

From: "Yingshiuan Pan" <yingshiuan.pan@mediatek.com>

irqfd enables other threads than vcpu threads to inject virtual interrupt
through irqfd asynchronously rather through ioctl interface. This interface
is necessary for VMM which creates separated thread for IO handling or uses
vhost devices.

Signed-off-by: Yingshiuan Pan <yingshiuan.pan@mediatek.com>
Signed-off-by: Yi-De Wu <yi-de.wu@mediatek.com>
---
 drivers/soc/mediatek/virt/geniezone/gzvm.h    |   8 +
 .../mediatek/virt/geniezone/gzvm_eventfd.c    | 497 ++++++++++++++++++
 .../mediatek/virt/geniezone/gzvm_irqchip.c    |  13 +
 drivers/soc/mediatek/virt/geniezone/gzvm_vm.c |  18 +-
 include/uapi/linux/gzvm_common.h              |  18 +
 5 files changed, 553 insertions(+), 1 deletion(-)

diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm.h b/drivers/soc/mediatek/virt/geniezone/gzvm.h
index b0edf56c2832..c79f84c17bb1 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm.h
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm.h
@@ -99,6 +99,14 @@ int gzvm_hypcall_wrapper(unsigned long a0, unsigned long a1, unsigned long a2,
 #define MT_HVC_GZVM_PROBE		GZVM_HCALL_ID(GZVM_FUNC_PROBE)
 #define MT_HVC_GZVM_ENABLE_CAP		GZVM_HCALL_ID(GZVM_FUNC_ENABLE_CAP)
 
+#define GZVM_USERSPACE_IRQ_SOURCE_ID		0
+#define GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID	1
+
+void gzvm_notify_acked_irq(struct gzvm *gzvm, unsigned int gsi);
+int gzvm_irqfd(struct gzvm *gzvm, struct gzvm_irqfd *args);
+int gzvm_irqfd_init(void);
+void gzvm_irqfd_exit(void);
+int gzvm_init_irqfd(struct gzvm *gzvm);
 int gzvm_init_ioeventfd(struct gzvm *gzvm);
 int gzvm_ioeventfd(struct gzvm *gzvm, struct gzvm_ioeventfd *args);
 bool gzvm_ioevent_write(struct gzvm_vcpu *vcpu, __u64 addr, int len,
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
index 63f042fefbe5..d8eea6c1a2d6 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c
@@ -12,6 +12,503 @@
 #include <linux/slab.h>
 #include "gzvm.h"
 
+struct gzvm_irq_ack_notifier {
+	struct hlist_node link;
+	unsigned int gsi;
+	void (*irq_acked)(struct gzvm_irq_ack_notifier *ian);
+};
+
+/*
+ * Resampling irqfds are a special variety of irqfds used to emulate
+ * level triggered interrupts.  The interrupt is asserted on eventfd
+ * trigger.  On acknowledgment through the irq ack notifier, the
+ * interrupt is de-asserted and userspace is notified through the
+ * resamplefd.  All resamplers on the same gsi are de-asserted
+ * together, so we don't need to track the state of each individual
+ * user.  We can also therefore share the same irq source ID.
+ */
+struct gzvm_kernel_irqfd_resampler {
+	struct gzvm *gzvm;
+	/*
+	 * List of resampling struct _irqfd objects sharing this gsi.
+	 * RCU list modified under gzvm->irqfds.resampler_lock
+	 */
+	struct list_head list;
+	struct gzvm_irq_ack_notifier notifier;
+	/*
+	 * Entry in list of gzvm->irqfd.resampler_list.  Use for sharing
+	 * resamplers among irqfds on the same gsi.
+	 * Accessed and modified under gzvm->irqfds.resampler_lock
+	 */
+	struct list_head link;
+};
+
+struct gzvm_kernel_irqfd {
+	struct gzvm *gzvm;
+	wait_queue_entry_t wait;
+	/* Used for level IRQ fast-path */
+	int gsi;
+	/* The resampler used by this irqfd (resampler-only) */
+	struct gzvm_kernel_irqfd_resampler *resampler;
+	/* Eventfd notified on resample (resampler-only) */
+	struct eventfd_ctx *resamplefd;
+	/* Entry in list of irqfds for a resampler (resampler-only) */
+	struct list_head resampler_link;
+	/* Used for setup/shutdown */
+	struct eventfd_ctx *eventfd;
+	struct list_head list;
+	poll_table pt;
+	struct work_struct shutdown;
+};
+
+static struct workqueue_struct *irqfd_cleanup_wq;
+
+/**
+ * @brief irqfd to inject virtual interrupt
+ * @param irq This is spi interrupt number (starts from 0 instead of 32)
+ */
+static void irqfd_set_spi(struct gzvm *gzvm, int irq_source_id, u32 irq,
+			  int level, bool line_status)
+{
+	if (level)
+		gzvm_vgic_inject_spi(gzvm, 0, irq, level);
+}
+
+/*
+ * Since resampler irqfds share an IRQ source ID, we de-assert once
+ * then notify all of the resampler irqfds using this GSI.  We can't
+ * do multiple de-asserts or we risk racing with incoming re-asserts.
+ */
+static void
+irqfd_resampler_ack(struct gzvm_irq_ack_notifier *ian)
+{
+	struct gzvm_kernel_irqfd_resampler *resampler;
+	struct gzvm *gzvm;
+	struct gzvm_kernel_irqfd *irqfd;
+	int idx;
+
+	resampler = container_of(ian,
+			struct gzvm_kernel_irqfd_resampler, notifier);
+	gzvm = resampler->gzvm;
+
+	irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID,
+		      resampler->notifier.gsi, 0, false);
+
+	idx = srcu_read_lock(&gzvm->irq_srcu);
+
+	list_for_each_entry_srcu(irqfd, &resampler->list, resampler_link,
+	    srcu_read_lock_held(&gzvm->irq_srcu)) {
+		eventfd_signal(irqfd->resamplefd, 1);
+	}
+
+	srcu_read_unlock(&gzvm->irq_srcu, idx);
+}
+
+static void gzvm_register_irq_ack_notifier(struct gzvm *gzvm,
+					   struct gzvm_irq_ack_notifier *ian)
+{
+	mutex_lock(&gzvm->irq_lock);
+	hlist_add_head_rcu(&ian->link, &gzvm->irq_ack_notifier_list);
+	mutex_unlock(&gzvm->irq_lock);
+}
+
+static void gzvm_unregister_irq_ack_notifier(struct gzvm *gzvm,
+					     struct gzvm_irq_ack_notifier *ian)
+{
+	mutex_lock(&gzvm->irq_lock);
+	hlist_del_init_rcu(&ian->link);
+	mutex_unlock(&gzvm->irq_lock);
+	synchronize_srcu(&gzvm->irq_srcu);
+}
+
+static void
+irqfd_resampler_shutdown(struct gzvm_kernel_irqfd *irqfd)
+{
+	struct gzvm_kernel_irqfd_resampler *resampler = irqfd->resampler;
+	struct gzvm *gzvm = resampler->gzvm;
+
+	mutex_lock(&gzvm->irqfds.resampler_lock);
+
+	list_del_rcu(&irqfd->resampler_link);
+	synchronize_srcu(&gzvm->irq_srcu);
+
+	if (list_empty(&resampler->list)) {
+		list_del(&resampler->link);
+		gzvm_unregister_irq_ack_notifier(gzvm, &resampler->notifier);
+		irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID,
+			      resampler->notifier.gsi, 0, false);
+		kfree(resampler);
+	}
+
+	mutex_unlock(&gzvm->irqfds.resampler_lock);
+}
+
+/*
+ * Race-free decouple logic (ordering is critical)
+ */
+static void
+irqfd_shutdown(struct work_struct *work)
+{
+	struct gzvm_kernel_irqfd *irqfd =
+		container_of(work, struct gzvm_kernel_irqfd, shutdown);
+	struct gzvm *gzvm = irqfd->gzvm;
+	u64 cnt;
+
+	/* Make sure irqfd has been initialized in assign path. */
+	synchronize_srcu(&gzvm->irq_srcu);
+
+	/*
+	 * Synchronize with the wait-queue and unhook ourselves to prevent
+	 * further events.
+	 */
+	eventfd_ctx_remove_wait_queue(irqfd->eventfd, &irqfd->wait, &cnt);
+
+	if (irqfd->resampler) {
+		irqfd_resampler_shutdown(irqfd);
+		eventfd_ctx_put(irqfd->resamplefd);
+	}
+
+	/*
+	 * It is now safe to release the object's resources
+	 */
+	eventfd_ctx_put(irqfd->eventfd);
+	kfree(irqfd);
+}
+
+
+/* assumes gzvm->irqfds.lock is held */
+static bool
+irqfd_is_active(struct gzvm_kernel_irqfd *irqfd)
+{
+	return list_empty(&irqfd->list) ? false : true;
+}
+
+/*
+ * Mark the irqfd as inactive and schedule it for removal
+ *
+ * assumes gzvm->irqfds.lock is held
+ */
+static void
+irqfd_deactivate(struct gzvm_kernel_irqfd *irqfd)
+{
+	if (!irqfd_is_active(irqfd))
+		return;
+
+	list_del_init(&irqfd->list);
+
+	queue_work(irqfd_cleanup_wq, &irqfd->shutdown);
+}
+
+/**
+ * @brief Wake up irqfd to do virtual interrupt injection
+ */
+static int
+irqfd_wakeup(wait_queue_entry_t *wait, unsigned int mode, int sync, void *key)
+{
+	struct gzvm_kernel_irqfd *irqfd =
+		container_of(wait, struct gzvm_kernel_irqfd, wait);
+	__poll_t flags = key_to_poll(key);
+	struct gzvm *gzvm = irqfd->gzvm;
+
+	if (flags & EPOLLIN) {
+		u64 cnt;
+
+		eventfd_ctx_do_read(irqfd->eventfd, &cnt);
+		/* gzvm's irq injection is not blocked, don't need workq */
+		irqfd_set_spi(gzvm, GZVM_USERSPACE_IRQ_SOURCE_ID, irqfd->gsi, 1,
+			      false);
+	}
+
+	if (flags & EPOLLHUP) {
+		/* The eventfd is closing, detach from GZVM */
+		unsigned long iflags;
+
+		spin_lock_irqsave(&gzvm->irqfds.lock, iflags);
+
+		/*
+		 * Do more check if someone deactivated the irqfd before
+		 * we could acquire the irqfds.lock.
+		 */
+		if (irqfd_is_active(irqfd))
+			irqfd_deactivate(irqfd);
+
+		spin_unlock_irqrestore(&gzvm->irqfds.lock, iflags);
+	}
+
+	return 0;
+}
+
+static void
+irqfd_ptable_queue_proc(struct file *file, wait_queue_head_t *wqh,
+			poll_table *pt)
+{
+	struct gzvm_kernel_irqfd *irqfd =
+		container_of(pt, struct gzvm_kernel_irqfd, pt);
+	add_wait_queue_priority(wqh, &irqfd->wait);
+}
+
+static int
+gzvm_irqfd_assign(struct gzvm *gzvm, struct gzvm_irqfd *args)
+{
+	struct gzvm_kernel_irqfd *irqfd, *tmp;
+	struct fd f;
+	struct eventfd_ctx *eventfd = NULL, *resamplefd = NULL;
+	int ret;
+	__poll_t events;
+	int idx;
+
+	irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL_ACCOUNT);
+	if (!irqfd)
+		return -ENOMEM;
+
+	irqfd->gzvm = gzvm;
+	irqfd->gsi = args->gsi;
+	irqfd->resampler = NULL;
+	INIT_LIST_HEAD(&irqfd->list);
+	INIT_WORK(&irqfd->shutdown, irqfd_shutdown);
+
+	f = fdget(args->fd);
+	if (!f.file) {
+		ret = -EBADF;
+		goto out;
+	}
+
+	eventfd = eventfd_ctx_fileget(f.file);
+	if (IS_ERR(eventfd)) {
+		ret = PTR_ERR(eventfd);
+		goto fail;
+	}
+
+	irqfd->eventfd = eventfd;
+
+	if (args->flags & GZVM_IRQFD_FLAG_RESAMPLE) {
+		struct gzvm_kernel_irqfd_resampler *resampler;
+
+		resamplefd = eventfd_ctx_fdget(args->resamplefd);
+		if (IS_ERR(resamplefd)) {
+			ret = PTR_ERR(resamplefd);
+			goto fail;
+		}
+
+		irqfd->resamplefd = resamplefd;
+		INIT_LIST_HEAD(&irqfd->resampler_link);
+
+		mutex_lock(&gzvm->irqfds.resampler_lock);
+
+		list_for_each_entry(resampler,
+				    &gzvm->irqfds.resampler_list, link) {
+			if (resampler->notifier.gsi == irqfd->gsi) {
+				irqfd->resampler = resampler;
+				break;
+			}
+		}
+
+		if (!irqfd->resampler) {
+			resampler = kzalloc(sizeof(*resampler),
+					    GFP_KERNEL_ACCOUNT);
+			if (!resampler) {
+				ret = -ENOMEM;
+				mutex_unlock(&gzvm->irqfds.resampler_lock);
+				goto fail;
+			}
+
+			resampler->gzvm = gzvm;
+			INIT_LIST_HEAD(&resampler->list);
+			resampler->notifier.gsi = irqfd->gsi;
+			resampler->notifier.irq_acked = irqfd_resampler_ack;
+			INIT_LIST_HEAD(&resampler->link);
+
+			list_add(&resampler->link, &gzvm->irqfds.resampler_list);
+			gzvm_register_irq_ack_notifier(gzvm,
+						       &resampler->notifier);
+			irqfd->resampler = resampler;
+		}
+
+		list_add_rcu(&irqfd->resampler_link, &irqfd->resampler->list);
+		synchronize_srcu(&gzvm->irq_srcu);
+
+		mutex_unlock(&gzvm->irqfds.resampler_lock);
+	}
+
+	/*
+	 * Install our own custom wake-up handling so we are notified via
+	 * a callback whenever someone signals the underlying eventfd
+	 */
+	init_waitqueue_func_entry(&irqfd->wait, irqfd_wakeup);
+	init_poll_funcptr(&irqfd->pt, irqfd_ptable_queue_proc);
+
+	spin_lock_irq(&gzvm->irqfds.lock);
+
+	ret = 0;
+	list_for_each_entry(tmp, &gzvm->irqfds.items, list) {
+		if (irqfd->eventfd != tmp->eventfd)
+			continue;
+		/* This fd is used for another irq already. */
+		pr_err("already used: gsi=%d fd=%d\n", args->gsi, args->fd);
+		ret = -EBUSY;
+		spin_unlock_irq(&gzvm->irqfds.lock);
+		goto fail;
+	}
+
+	idx = srcu_read_lock(&gzvm->irq_srcu);
+
+	list_add_tail(&irqfd->list, &gzvm->irqfds.items);
+
+	spin_unlock_irq(&gzvm->irqfds.lock);
+
+	/*
+	 * Check if there was an event already pending on the eventfd
+	 * before we registered, and trigger it as if we didn't miss it.
+	 */
+	events = vfs_poll(f.file, &irqfd->pt);
+
+	/* In case there is already a pending event */
+	if (events & EPOLLIN)
+		irqfd_set_spi(gzvm, GZVM_IRQFD_RESAMPLE_IRQ_SOURCE_ID,
+			      irqfd->gsi, 1, false);
+
+	srcu_read_unlock(&gzvm->irq_srcu, idx);
+
+	/*
+	 * do not drop the file until the irqfd is fully initialized, otherwise
+	 * we might race against the EPOLLHUP
+	 */
+	fdput(f);
+	return 0;
+
+fail:
+	if (irqfd->resampler)
+		irqfd_resampler_shutdown(irqfd);
+
+	if (resamplefd && !IS_ERR(resamplefd))
+		eventfd_ctx_put(resamplefd);
+
+	if (eventfd && !IS_ERR(eventfd))
+		eventfd_ctx_put(eventfd);
+
+	fdput(f);
+
+out:
+	kfree(irqfd);
+	return ret;
+}
+
+static void gzvm_notify_acked_gsi(struct gzvm *gzvm, int gsi)
+{
+	struct gzvm_irq_ack_notifier *gian;
+
+	hlist_for_each_entry_srcu(gian, &gzvm->irq_ack_notifier_list,
+				  link, srcu_read_lock_held(&gzvm->irq_srcu))
+		if (gian->gsi == gsi)
+			gian->irq_acked(gian);
+}
+
+void gzvm_notify_acked_irq(struct gzvm *gzvm, unsigned int gsi)
+{
+	int idx;
+
+	idx = srcu_read_lock(&gzvm->irq_srcu);
+	gzvm_notify_acked_gsi(gzvm, gsi);
+	srcu_read_unlock(&gzvm->irq_srcu, idx);
+}
+
+/*
+ * shutdown any irqfd's that match fd+gsi
+ */
+static int gzvm_irqfd_deassign(struct gzvm *gzvm, struct gzvm_irqfd *args)
+{
+	struct gzvm_kernel_irqfd *irqfd, *tmp;
+	struct eventfd_ctx *eventfd;
+
+	eventfd = eventfd_ctx_fdget(args->fd);
+	if (IS_ERR(eventfd))
+		return PTR_ERR(eventfd);
+
+	spin_lock_irq(&gzvm->irqfds.lock);
+
+	list_for_each_entry_safe(irqfd, tmp, &gzvm->irqfds.items, list) {
+		if (irqfd->eventfd == eventfd && irqfd->gsi == args->gsi)
+			irqfd_deactivate(irqfd);
+	}
+
+	spin_unlock_irq(&gzvm->irqfds.lock);
+	eventfd_ctx_put(eventfd);
+
+	/*
+	 * Block until we know all outstanding shutdown jobs have completed
+	 * so that we guarantee there will not be any more interrupts on this
+	 * gsi once this deassign function returns.
+	 */
+	flush_workqueue(irqfd_cleanup_wq);
+
+	return 0;
+}
+
+int gzvm_irqfd(struct gzvm *gzvm, struct gzvm_irqfd *args)
+{
+	if (args->flags &
+	    ~(GZVM_IRQFD_FLAG_DEASSIGN | GZVM_IRQFD_FLAG_RESAMPLE))
+		return -EINVAL;
+
+	if (args->flags & GZVM_IRQFD_FLAG_DEASSIGN)
+		return gzvm_irqfd_deassign(gzvm, args);
+
+	return gzvm_irqfd_assign(gzvm, args);
+}
+
+/*
+ * This function is called as the gzvm VM fd is being released. Shutdown all
+ * irqfds that still remain open
+ */
+void gzvm_irqfd_release(struct gzvm *gzvm)
+{
+	struct gzvm_kernel_irqfd *irqfd, *tmp;
+
+	spin_lock_irq(&gzvm->irqfds.lock);
+
+	list_for_each_entry_safe(irqfd, tmp, &gzvm->irqfds.items, list)
+		irqfd_deactivate(irqfd);
+
+	spin_unlock_irq(&gzvm->irqfds.lock);
+
+	/*
+	 * Block until we know all outstanding shutdown jobs have completed.
+	 */
+	flush_workqueue(irqfd_cleanup_wq);
+}
+
+/*
+ * create a host-wide workqueue for issuing deferred shutdown requests
+ * aggregated from all vm* instances. We need our own isolated
+ * queue to ease flushing work items when a VM exits.
+ */
+int gzvm_irqfd_init(void)
+{
+	irqfd_cleanup_wq = alloc_workqueue("gzvm-irqfd-cleanup", 0, 0);
+	if (!irqfd_cleanup_wq)
+		return -ENOMEM;
+
+	return 0;
+}
+
+void gzvm_irqfd_exit(void)
+{
+	destroy_workqueue(irqfd_cleanup_wq);
+}
+
+int gzvm_init_irqfd(struct gzvm *gzvm)
+{
+	spin_lock_init(&gzvm->irqfds.lock);
+	INIT_LIST_HEAD(&gzvm->irqfds.items);
+	INIT_LIST_HEAD(&gzvm->irqfds.resampler_list);
+	if (init_srcu_struct(&gzvm->irq_srcu))
+		return -EINVAL;
+	INIT_HLIST_HEAD(&gzvm->irq_ack_notifier_list);
+	mutex_init(&gzvm->irqfds.resampler_lock);
+
+	return 0;
+}
+
 struct gzvm_ioevent {
 	struct list_head list;
 	__u64 addr;
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
index 7aa5868a221c..134fa5e5715e 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_irqchip.c
@@ -8,6 +8,12 @@
 
 #include "gzvm.h"
 
+static bool lr_signals_eoi(uint64_t lr_val)
+{
+	return !(lr_val & ICH_LR_STATE) && (lr_val & ICH_LR_EOI) &&
+	       !(lr_val & ICH_LR_HW);
+}
+
 /**
  * @brief check all LRs synced from gz hypervisor
  * Traverse all LRs, see if any EOIed vint, notify_acked_irq if any.
@@ -20,10 +26,17 @@ void gzvm_sync_vgic_state(struct gzvm_vcpu *vcpu)
 	int i;
 
 	for (i = 0; i < vcpu->hwstate->nr_lrs; i++) {
+		uint32_t vintid;
 		uint64_t lr_val = vcpu->hwstate->lr[i];
 		/* 0 means unused */
 		if (!lr_val)
 			continue;
+
+		vintid = lr_val & ICH_LR_VIRTUAL_ID_MASK;
+		if (lr_signals_eoi(lr_val)) {
+			gzvm_notify_acked_irq(vcpu->gzvm,
+					      vintid - VGIC_NR_PRIVATE_IRQS);
+		}
 	}
 }
 
diff --git a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
index cceaa532c2ce..e4bea025e45e 100644
--- a/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
+++ b/drivers/soc/mediatek/virt/geniezone/gzvm_vm.c
@@ -386,6 +386,15 @@ static long gzvm_vm_ioctl(struct file *filp, unsigned int ioctl,
 		ret = gzvm_ioeventfd(gzvm, &data);
 		break;
 	}
+	case GZVM_IRQFD: {
+		struct gzvm_irqfd data;
+
+		ret = -EFAULT;
+		if (copy_from_user(&data, argp, sizeof(data)))
+			goto out;
+		ret = gzvm_irqfd(gzvm, &data);
+		break;
+	}
 	case GZVM_ENABLE_CAP: {
 		struct gzvm_enable_cap cap;
 
@@ -490,13 +499,20 @@ static struct gzvm *gzvm_create_vm(unsigned long vm_type)
 	mutex_init(&gzvm->lock);
 	INIT_LIST_HEAD(&gzvm->devices);
 	mutex_init(&gzvm->irq_lock);
+
 	ret = gzvm_init_ioeventfd(gzvm);
 	if (ret) {
 		pr_err("Failed to initialize ioeventfd\n");
 		goto err;
 	}
-	pr_info("VM-%u is created\n", gzvm->vm_id);
 
+	ret = gzvm_init_irqfd(gzvm);
+	if (ret) {
+		pr_err("Failed to initialize irqfd\n");
+		goto err;
+	}
+
+	pr_info("VM-%u is created\n", gzvm->vm_id);
 	mutex_lock(&gzvm_list_lock);
 	list_add(&gzvm->vm_list, &gzvm_list);
 	mutex_unlock(&gzvm_list_lock);
diff --git a/include/uapi/linux/gzvm_common.h b/include/uapi/linux/gzvm_common.h
index 6dbaddd77ec7..3af9d148042f 100644
--- a/include/uapi/linux/gzvm_common.h
+++ b/include/uapi/linux/gzvm_common.h
@@ -193,6 +193,24 @@ struct gzvm_irq_level {
 #define GZVM_IRQ_LINE              _IOW(GZVM_IOC_MAGIC,  0x61, \
 					struct gzvm_irq_level)
 
+#define GZVM_IRQFD_FLAG_DEASSIGN (1 << 0)
+/*
+ * GZVM_IRQFD_FLAG_RESAMPLE indicates resamplefd is valid and specifies
+ * the irqfd to operate in resampling mode for level triggered interrupt
+ * emulation.
+ */
+#define GZVM_IRQFD_FLAG_RESAMPLE (1 << 1)
+
+struct gzvm_irqfd {
+	__u32 fd;
+	__u32 gsi;
+	__u32 flags;
+	__u32 resamplefd;
+	__u8  pad[16];
+};
+#define GZVM_IRQFD                 _IOW(GZVM_IOC_MAGIC,  0x76, \
+					struct gzvm_irqfd)
+
 enum {
 	gzvm_ioeventfd_flag_nr_datamatch,
 	gzvm_ioeventfd_flag_nr_pio,
-- 
2.18.0

Re: [PATCH v1 6/6] soc: mediatek: virt: geniezone: Add irqfd support

[kernel test robot]

Hi Yi-De,

kernel test robot noticed the following build warnings:

[auto build test WARNING on robh/for-next]
[also build test WARNING on krzk-dt/for-next arm64/for-next/core lwn/docs-next linus/master v6.3-rc6 next-20230412]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]

url:    https://github.com/intel-lab-lkp/linux/commits/Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
base:   https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link:    https://lore.kernel.org/r/20230413090735.4182-7-yi-de.wu%40mediatek.com
patch subject: [PATCH v1 6/6] soc: mediatek: virt: geniezone: Add irqfd support
config: arm64-allyesconfig (https://download.01.org/0day-ci/archive/20230413/202304132123.rrVq3AEP-lkp@intel.com/config)
compiler: aarch64-linux-gcc (GCC) 12.1.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/intel-lab-lkp/linux/commit/ae996a1c7d12837f16f28975712a8bf63525cac4
        git remote add linux-review https://github.com/intel-lab-lkp/linux
        git fetch --no-tags linux-review Yi-De-Wu/docs-geniezone-Introduce-GenieZone-hypervisor/20230413-170932
        git checkout ae996a1c7d12837f16f28975712a8bf63525cac4
        # save the config file
        mkdir build_dir && cp config build_dir/.config
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 olddefconfig
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-12.1.0 make.cross W=1 O=build_dir ARCH=arm64 SHELL=/bin/bash drivers/soc/mediatek/virt/geniezone/

If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <lkp@intel.com>
| Link: https://lore.kernel.org/oe-kbuild-all/202304132123.rrVq3AEP-lkp@intel.com/

All warnings (new ones prefixed by >>):

>> drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c:463:6: warning: no previous prototype for 'gzvm_irqfd_release' [-Wmissing-prototypes]
     463 | void gzvm_irqfd_release(struct gzvm *gzvm)
         |      ^~~~~~~~~~~~~~~~~~


vim +/gzvm_irqfd_release +463 drivers/soc/mediatek/virt/geniezone/gzvm_eventfd.c

   458	
   459	/*
   460	 * This function is called as the gzvm VM fd is being released. Shutdown all
   461	 * irqfds that still remain open
   462	 */
 > 463	void gzvm_irqfd_release(struct gzvm *gzvm)
   464	{
   465		struct gzvm_kernel_irqfd *irqfd, *tmp;
   466	
   467		spin_lock_irq(&gzvm->irqfds.lock);
   468	
   469		list_for_each_entry_safe(irqfd, tmp, &gzvm->irqfds.items, list)
   470			irqfd_deactivate(irqfd);
   471	
   472		spin_unlock_irq(&gzvm->irqfds.lock);
   473	
   474		/*
   475		 * Block until we know all outstanding shutdown jobs have completed.
   476		 */
   477		flush_workqueue(irqfd_cleanup_wq);
   478	}
   479	

-- 
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests