[PATCH v2] virtio: Add polling virtio-mmio device reset completion

[PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Jianguo Sun]

We write 0 to device_status to initiate reset of a virtio-mmio device.
The reset operation itself may or may not be completed by the time
write instruction completes. Add polling device_status to return
0 to ensure reset completion before reinitializing the device.

Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
---
 drivers/virtio/virtio_mmio.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 90e784e7b721..cf1a4a3dd35d 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -70,6 +70,7 @@
 #include <linux/virtio_config.h>
 #include <uapi/linux/virtio_mmio.h>
 #include <linux/virtio_ring.h>
+#include <linux/delay.h>
 
 
 
@@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
 
 	/* 0 status means a reset. */
 	writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
+	/* After writing 0 to device_status, the driver MUST wait for a read of
+	 * device_status to return 0 before reinitializing the device.
+	 */
+	while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
+		msleep(1);
 }
 
 
-- 
2.17.1

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Jason Wang]

On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
>
> We write 0 to device_status to initiate reset of a virtio-mmio device.
> The reset operation itself may or may not be completed by the time
> write instruction completes. Add polling device_status to return
> 0 to ensure reset completion before reinitializing the device.
>
> Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> ---
>  drivers/virtio/virtio_mmio.c | 6 ++++++
>  1 file changed, 6 insertions(+)
>
> diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> index 90e784e7b721..cf1a4a3dd35d 100644
> --- a/drivers/virtio/virtio_mmio.c
> +++ b/drivers/virtio/virtio_mmio.c
> @@ -70,6 +70,7 @@
>  #include <linux/virtio_config.h>
>  #include <uapi/linux/virtio_mmio.h>
>  #include <linux/virtio_ring.h>
> +#include <linux/delay.h>
>
>
>
> @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
>
>         /* 0 status means a reset. */
>         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> +       /* After writing 0 to device_status, the driver MUST wait for a read of
> +        * device_status to return 0 before reinitializing the device.
> +        */

Is this copied from the spec? I can only see a similar description in
the PCI but not MMIO part.

> +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> +               msleep(1);

Again, we probably need a feature bit to unbreak existing MMIO devices?

THanks

>  }
>
>
> --
> 2.17.1
>

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Michael S. Tsirkin]

On Thu, Sep 26, 2024 at 11:21:25AM +0800, Jason Wang wrote:
> On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
> >
> > We write 0 to device_status to initiate reset of a virtio-mmio device.
> > The reset operation itself may or may not be completed by the time
> > write instruction completes. Add polling device_status to return
> > 0 to ensure reset completion before reinitializing the device.
> >
> > Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> > Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> > Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> > Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> > ---
> >  drivers/virtio/virtio_mmio.c | 6 ++++++
> >  1 file changed, 6 insertions(+)
> >
> > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > index 90e784e7b721..cf1a4a3dd35d 100644
> > --- a/drivers/virtio/virtio_mmio.c
> > +++ b/drivers/virtio/virtio_mmio.c
> > @@ -70,6 +70,7 @@
> >  #include <linux/virtio_config.h>
> >  #include <uapi/linux/virtio_mmio.h>
> >  #include <linux/virtio_ring.h>
> > +#include <linux/delay.h>
> >
> >
> >
> > @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
> >
> >         /* 0 status means a reset. */
> >         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> > +       /* After writing 0 to device_status, the driver MUST wait for a read of
> > +        * device_status to return 0 before reinitializing the device.
> > +        */
> 
> Is this copied from the spec? I can only see a similar description in
> the PCI but not MMIO part.

Not in spec. We can add something like this in spec, but it
can not be a MUST, has to be a SHOULD.

> > +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> > +               msleep(1);
> 
> Again, we probably need a feature bit to unbreak existing MMIO devices?
> 
> THanks

We can do this unconditionally since legacy devices return 0
immediately, feature negotiation happens after reset, won't be useful
here.


> >  }
> >
> >
> > --
> > 2.17.1
> >

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Jason Wang]

On Thu, Sep 26, 2024 at 2:09 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Sep 26, 2024 at 11:21:25AM +0800, Jason Wang wrote:
> > On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
> > >
> > > We write 0 to device_status to initiate reset of a virtio-mmio device.
> > > The reset operation itself may or may not be completed by the time
> > > write instruction completes. Add polling device_status to return
> > > 0 to ensure reset completion before reinitializing the device.
> > >
> > > Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> > > Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> > > Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> > > Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> > > ---
> > >  drivers/virtio/virtio_mmio.c | 6 ++++++
> > >  1 file changed, 6 insertions(+)
> > >
> > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > index 90e784e7b721..cf1a4a3dd35d 100644
> > > --- a/drivers/virtio/virtio_mmio.c
> > > +++ b/drivers/virtio/virtio_mmio.c
> > > @@ -70,6 +70,7 @@
> > >  #include <linux/virtio_config.h>
> > >  #include <uapi/linux/virtio_mmio.h>
> > >  #include <linux/virtio_ring.h>
> > > +#include <linux/delay.h>
> > >
> > >
> > >
> > > @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
> > >
> > >         /* 0 status means a reset. */
> > >         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> > > +       /* After writing 0 to device_status, the driver MUST wait for a read of
> > > +        * device_status to return 0 before reinitializing the device.
> > > +        */
> >
> > Is this copied from the spec? I can only see a similar description in
> > the PCI but not MMIO part.
>
> Not in spec. We can add something like this in spec, but it
> can not be a MUST, has to be a SHOULD.
>
> > > +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> > > +               msleep(1);
> >
> > Again, we probably need a feature bit to unbreak existing MMIO devices?
> >
> > THanks
>
> We can do this unconditionally since legacy devices return 0
> immediately,

The problem is that we can't audit all legacy device implementations.
Current driver still works if a device doesn't reset status to 0.

> feature negotiation happens after reset, won't be useful
> here.

Right, as said in another thread, it could be mmio specific flag or version.

Thanks

>
>
> > >  }
> > >
> > >
> > > --
> > > 2.17.1
> > >
>

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Michael S. Tsirkin]

On Fri, Sep 27, 2024 at 12:08:34PM +0800, Jason Wang wrote:
> On Thu, Sep 26, 2024 at 2:09 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Thu, Sep 26, 2024 at 11:21:25AM +0800, Jason Wang wrote:
> > > On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
> > > >
> > > > We write 0 to device_status to initiate reset of a virtio-mmio device.
> > > > The reset operation itself may or may not be completed by the time
> > > > write instruction completes. Add polling device_status to return
> > > > 0 to ensure reset completion before reinitializing the device.
> > > >
> > > > Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> > > > Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> > > > Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> > > > Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> > > > ---
> > > >  drivers/virtio/virtio_mmio.c | 6 ++++++
> > > >  1 file changed, 6 insertions(+)
> > > >
> > > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > > index 90e784e7b721..cf1a4a3dd35d 100644
> > > > --- a/drivers/virtio/virtio_mmio.c
> > > > +++ b/drivers/virtio/virtio_mmio.c
> > > > @@ -70,6 +70,7 @@
> > > >  #include <linux/virtio_config.h>
> > > >  #include <uapi/linux/virtio_mmio.h>
> > > >  #include <linux/virtio_ring.h>
> > > > +#include <linux/delay.h>
> > > >
> > > >
> > > >
> > > > @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
> > > >
> > > >         /* 0 status means a reset. */
> > > >         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> > > > +       /* After writing 0 to device_status, the driver MUST wait for a read of
> > > > +        * device_status to return 0 before reinitializing the device.
> > > > +        */
> > >
> > > Is this copied from the spec? I can only see a similar description in
> > > the PCI but not MMIO part.
> >
> > Not in spec. We can add something like this in spec, but it
> > can not be a MUST, has to be a SHOULD.
> >
> > > > +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> > > > +               msleep(1);
> > >
> > > Again, we probably need a feature bit to unbreak existing MMIO devices?
> > >
> > > THanks
> >
> > We can do this unconditionally since legacy devices return 0
> > immediately,
> 
> The problem is that we can't audit all legacy device implementations.
> Current driver still works if a device doesn't reset status to 0.

These would be out of spec though, would they not?
The reset value for the register is 0, and it is R/W.


> > feature negotiation happens after reset, won't be useful
> > here.
> 
> Right, as said in another thread, it could be mmio specific flag or version.
> 
> Thanks
> 
> >
> >
> > > >  }
> > > >
> > > >
> > > > --
> > > > 2.17.1
> > > >
> >

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Jason Wang]

On Mon, Sep 30, 2024 at 1:51 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Fri, Sep 27, 2024 at 12:08:34PM +0800, Jason Wang wrote:
> > On Thu, Sep 26, 2024 at 2:09 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > >
> > > On Thu, Sep 26, 2024 at 11:21:25AM +0800, Jason Wang wrote:
> > > > On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
> > > > >
> > > > > We write 0 to device_status to initiate reset of a virtio-mmio device.
> > > > > The reset operation itself may or may not be completed by the time
> > > > > write instruction completes. Add polling device_status to return
> > > > > 0 to ensure reset completion before reinitializing the device.
> > > > >
> > > > > Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> > > > > Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> > > > > Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> > > > > Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> > > > > ---
> > > > >  drivers/virtio/virtio_mmio.c | 6 ++++++
> > > > >  1 file changed, 6 insertions(+)
> > > > >
> > > > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > > > index 90e784e7b721..cf1a4a3dd35d 100644
> > > > > --- a/drivers/virtio/virtio_mmio.c
> > > > > +++ b/drivers/virtio/virtio_mmio.c
> > > > > @@ -70,6 +70,7 @@
> > > > >  #include <linux/virtio_config.h>
> > > > >  #include <uapi/linux/virtio_mmio.h>
> > > > >  #include <linux/virtio_ring.h>
> > > > > +#include <linux/delay.h>
> > > > >
> > > > >
> > > > >
> > > > > @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
> > > > >
> > > > >         /* 0 status means a reset. */
> > > > >         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> > > > > +       /* After writing 0 to device_status, the driver MUST wait for a read of
> > > > > +        * device_status to return 0 before reinitializing the device.
> > > > > +        */
> > > >
> > > > Is this copied from the spec? I can only see a similar description in
> > > > the PCI but not MMIO part.
> > >
> > > Not in spec. We can add something like this in spec, but it
> > > can not be a MUST, has to be a SHOULD.
> > >
> > > > > +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> > > > > +               msleep(1);
> > > >
> > > > Again, we probably need a feature bit to unbreak existing MMIO devices?
> > > >
> > > > THanks
> > >
> > > We can do this unconditionally since legacy devices return 0
> > > immediately,
> >
> > The problem is that we can't audit all legacy device implementations.
> > Current driver still works if a device doesn't reset status to 0.
>
> These would be out of spec though, would they not?

Yes, but it would work. Should we care about those?

> The reset value for the register is 0, and it is R/W.
>
>
> > > feature negotiation happens after reset, won't be useful
> > > here.
> >
> > Right, as said in another thread, it could be mmio specific flag or version.
> >
> > Thanks
> >
> > >
> > >
> > > > >  }
> > > > >
> > > > >
> > > > > --
> > > > > 2.17.1
> > > > >
> > >
>

Thanks

Re: [PATCH v2] virtio: Add polling virtio-mmio device reset completion

[Michael S. Tsirkin]

On Wed, Oct 09, 2024 at 04:24:38PM +0800, Jason Wang wrote:
> On Mon, Sep 30, 2024 at 1:51 AM Michael S. Tsirkin <mst@redhat.com> wrote:
> >
> > On Fri, Sep 27, 2024 at 12:08:34PM +0800, Jason Wang wrote:
> > > On Thu, Sep 26, 2024 at 2:09 PM Michael S. Tsirkin <mst@redhat.com> wrote:
> > > >
> > > > On Thu, Sep 26, 2024 at 11:21:25AM +0800, Jason Wang wrote:
> > > > > On Thu, Sep 26, 2024 at 10:27 AM Jianguo Sun <quic_jianguos@quicinc.com> wrote:
> > > > > >
> > > > > > We write 0 to device_status to initiate reset of a virtio-mmio device.
> > > > > > The reset operation itself may or may not be completed by the time
> > > > > > write instruction completes. Add polling device_status to return
> > > > > > 0 to ensure reset completion before reinitializing the device.
> > > > > >
> > > > > > Change-Id: If15d11d090dfd0d4972ad35f49af03e076872413
> > > > > > Signed-off-by: Jianguo Sun <quic_jianguos@quicinc.com>
> > > > > > Signed-off-by: Sreenad Menon <quic_sreemeno@quicinc.com>
> > > > > > Signed-off-by: Anant Goel <quic_anantg@quicinc.com>
> > > > > > ---
> > > > > >  drivers/virtio/virtio_mmio.c | 6 ++++++
> > > > > >  1 file changed, 6 insertions(+)
> > > > > >
> > > > > > diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
> > > > > > index 90e784e7b721..cf1a4a3dd35d 100644
> > > > > > --- a/drivers/virtio/virtio_mmio.c
> > > > > > +++ b/drivers/virtio/virtio_mmio.c
> > > > > > @@ -70,6 +70,7 @@
> > > > > >  #include <linux/virtio_config.h>
> > > > > >  #include <uapi/linux/virtio_mmio.h>
> > > > > >  #include <linux/virtio_ring.h>
> > > > > > +#include <linux/delay.h>
> > > > > >
> > > > > >
> > > > > >
> > > > > > @@ -269,6 +270,11 @@ static void vm_reset(struct virtio_device *vdev)
> > > > > >
> > > > > >         /* 0 status means a reset. */
> > > > > >         writel(0, vm_dev->base + VIRTIO_MMIO_STATUS);
> > > > > > +       /* After writing 0 to device_status, the driver MUST wait for a read of
> > > > > > +        * device_status to return 0 before reinitializing the device.
> > > > > > +        */
> > > > >
> > > > > Is this copied from the spec? I can only see a similar description in
> > > > > the PCI but not MMIO part.
> > > >
> > > > Not in spec. We can add something like this in spec, but it
> > > > can not be a MUST, has to be a SHOULD.
> > > >
> > > > > > +       while (readl(vm_dev->base + VIRTIO_MMIO_STATUS))
> > > > > > +               msleep(1);
> > > > >
> > > > > Again, we probably need a feature bit to unbreak existing MMIO devices?
> > > > >
> > > > > THanks
> > > >
> > > > We can do this unconditionally since legacy devices return 0
> > > > immediately,
> > >
> > > The problem is that we can't audit all legacy device implementations.
> > > Current driver still works if a device doesn't reset status to 0.
> >
> > These would be out of spec though, would they not?
> 
> Yes, but it would work. Should we care about those?

Not unless there's an actual, common device that does it.

> > The reset value for the register is 0, and it is R/W.
> >
> >
> > > > feature negotiation happens after reset, won't be useful
> > > > here.
> > >
> > > Right, as said in another thread, it could be mmio specific flag or version.
> > >
> > > Thanks
> > >
> > > >
> > > >
> > > > > >  }
> > > > > >
> > > > > >
> > > > > > --
> > > > > > 2.17.1
> > > > > >
> > > >
> >
> 
> Thanks