* [PATCH] 9p/trans_virtio: fix hot-unplug
@ 2015-03-09 14:48 ` Michael S. Tsirkin
0 siblings, 0 replies; 8+ messages in thread
From: Michael S. Tsirkin @ 2015-03-09 14:48 UTC (permalink / raw)
To: linux-kernel
Cc: virtualization, Rusty Russell, Eric Van Hensbergen, Ron Minnich,
Latchesar Ionkov, David S. Miller, v9fs-developer, netdev
On device hot-unplug, 9p/virtio currently will kfree channel while
it might still be in use.
Of course, it might stay used forever, so it's an extremely ugly hack,
but it seems better than use-after-free that we have now.
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
---
net/9p/trans_virtio.c | 25 +++++++++++++++++++++----
1 file changed, 21 insertions(+), 4 deletions(-)
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index d8e376a..d1b2f306 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -658,14 +658,31 @@ p9_virtio_create(struct p9_client *client, const char *devname, char *args)
static void p9_virtio_remove(struct virtio_device *vdev)
{
struct virtio_chan *chan = vdev->priv;
-
- if (chan->inuse)
- p9_virtio_close(chan->client);
- vdev->config->del_vqs(vdev);
+ unsigned long warning_time;
+ bool inuse;
mutex_lock(&virtio_9p_lock);
+
+ /* Remove self from list so we don't get new users. */
list_del(&chan->chan_list);
+ warning_time = jiffies;
+
+ /* Wait for existing users to close. */
+ while (chan->inuse) {
+ mutex_unlock(&virtio_9p_lock);
+ msleep(250);
+ if (time_after(jiffies, warning_time + 10 * HZ)) {
+ dev_emerg(&vdev->dev, "p9_virtio_remove: "
+ "waiting for device in use.\n");
+ warning_time = jiffies;
+ }
+ mutex_lock(&virtio_9p_lock);
+ }
+
mutex_unlock(&virtio_9p_lock);
+
+ vdev->config->del_vqs(vdev);
+
sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
kfree(chan->tag);
--
MST
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
2015-03-09 14:48 ` Michael S. Tsirkin
(?)
@ 2015-03-12 1:24 ` Rusty Russell
2015-03-12 6:36 ` Michael S. Tsirkin
-1 siblings, 1 reply; 8+ messages in thread
From: Rusty Russell @ 2015-03-12 1:24 UTC (permalink / raw)
To: Michael S. Tsirkin, linux-kernel
Cc: virtualization, Eric Van Hensbergen, Ron Minnich,
Latchesar Ionkov, David S. Miller, v9fs-developer, netdev
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On device hot-unplug, 9p/virtio currently will kfree channel while
> it might still be in use.
>
> Of course, it might stay used forever, so it's an extremely ugly hack,
> but it seems better than use-after-free that we have now.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
I'll apply it, but it looks like a bandaid.
The right answer would seem to be:
1) Use reference counting: 1 for client, 1 for transport.
2) When hot unplug, complete (ie. fail) all outstanding requests.
3) From then on, fail all incoming requests.
4) When refcount hits 0, free the structure.
Thanks,
Rusty.
> ---
> net/9p/trans_virtio.c | 25 +++++++++++++++++++++----
> 1 file changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
> index d8e376a..d1b2f306 100644
> --- a/net/9p/trans_virtio.c
> +++ b/net/9p/trans_virtio.c
> @@ -658,14 +658,31 @@ p9_virtio_create(struct p9_client *client, const char *devname, char *args)
> static void p9_virtio_remove(struct virtio_device *vdev)
> {
> struct virtio_chan *chan = vdev->priv;
> -
> - if (chan->inuse)
> - p9_virtio_close(chan->client);
> - vdev->config->del_vqs(vdev);
> + unsigned long warning_time;
> + bool inuse;
>
> mutex_lock(&virtio_9p_lock);
> +
> + /* Remove self from list so we don't get new users. */
> list_del(&chan->chan_list);
> + warning_time = jiffies;
> +
> + /* Wait for existing users to close. */
> + while (chan->inuse) {
> + mutex_unlock(&virtio_9p_lock);
> + msleep(250);
> + if (time_after(jiffies, warning_time + 10 * HZ)) {
> + dev_emerg(&vdev->dev, "p9_virtio_remove: "
> + "waiting for device in use.\n");
> + warning_time = jiffies;
> + }
> + mutex_lock(&virtio_9p_lock);
> + }
> +
> mutex_unlock(&virtio_9p_lock);
> +
> + vdev->config->del_vqs(vdev);
> +
> sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
> kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
> kfree(chan->tag);
> --
> MST
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
2015-03-12 1:24 ` Rusty Russell
@ 2015-03-12 6:36 ` Michael S. Tsirkin
0 siblings, 0 replies; 8+ messages in thread
From: Michael S. Tsirkin @ 2015-03-12 6:36 UTC (permalink / raw)
To: Rusty Russell
Cc: Eric Van Hensbergen, netdev, linux-kernel, virtualization,
v9fs-developer, Ron Minnich, David S. Miller
On Thu, Mar 12, 2015 at 11:54:10AM +1030, Rusty Russell wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
> > On device hot-unplug, 9p/virtio currently will kfree channel while
> > it might still be in use.
> >
> > Of course, it might stay used forever, so it's an extremely ugly hack,
> > but it seems better than use-after-free that we have now.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> I'll apply it, but it looks like a bandaid.
Absolutely.
> The right answer would seem to be:
> 1) Use reference counting: 1 for client, 1 for transport.
> 2) When hot unplug, complete (ie. fail) all outstanding requests.
> 3) From then on, fail all incoming requests.
> 4) When refcount hits 0, free the structure.
>
> Thanks,
> Rusty.
Right. Will try to do for 4.1.
>
>
> > ---
> > net/9p/trans_virtio.c | 25 +++++++++++++++++++++----
> > 1 file changed, 21 insertions(+), 4 deletions(-)
> >
> > diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
> > index d8e376a..d1b2f306 100644
> > --- a/net/9p/trans_virtio.c
> > +++ b/net/9p/trans_virtio.c
> > @@ -658,14 +658,31 @@ p9_virtio_create(struct p9_client *client, const char *devname, char *args)
> > static void p9_virtio_remove(struct virtio_device *vdev)
> > {
> > struct virtio_chan *chan = vdev->priv;
> > -
> > - if (chan->inuse)
> > - p9_virtio_close(chan->client);
> > - vdev->config->del_vqs(vdev);
> > + unsigned long warning_time;
> > + bool inuse;
> >
> > mutex_lock(&virtio_9p_lock);
> > +
> > + /* Remove self from list so we don't get new users. */
> > list_del(&chan->chan_list);
> > + warning_time = jiffies;
> > +
> > + /* Wait for existing users to close. */
> > + while (chan->inuse) {
> > + mutex_unlock(&virtio_9p_lock);
> > + msleep(250);
> > + if (time_after(jiffies, warning_time + 10 * HZ)) {
> > + dev_emerg(&vdev->dev, "p9_virtio_remove: "
> > + "waiting for device in use.\n");
> > + warning_time = jiffies;
> > + }
> > + mutex_lock(&virtio_9p_lock);
> > + }
> > +
> > mutex_unlock(&virtio_9p_lock);
> > +
> > + vdev->config->del_vqs(vdev);
> > +
> > sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
> > kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
> > kfree(chan->tag);
> > --
> > MST
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
@ 2015-03-12 6:36 ` Michael S. Tsirkin
0 siblings, 0 replies; 8+ messages in thread
From: Michael S. Tsirkin @ 2015-03-12 6:36 UTC (permalink / raw)
To: Rusty Russell
Cc: linux-kernel, virtualization, Eric Van Hensbergen, Ron Minnich,
Latchesar Ionkov, David S. Miller, v9fs-developer, netdev
On Thu, Mar 12, 2015 at 11:54:10AM +1030, Rusty Russell wrote:
> "Michael S. Tsirkin" <mst@redhat.com> writes:
> > On device hot-unplug, 9p/virtio currently will kfree channel while
> > it might still be in use.
> >
> > Of course, it might stay used forever, so it's an extremely ugly hack,
> > but it seems better than use-after-free that we have now.
> >
> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>
> I'll apply it, but it looks like a bandaid.
Absolutely.
> The right answer would seem to be:
> 1) Use reference counting: 1 for client, 1 for transport.
> 2) When hot unplug, complete (ie. fail) all outstanding requests.
> 3) From then on, fail all incoming requests.
> 4) When refcount hits 0, free the structure.
>
> Thanks,
> Rusty.
Right. Will try to do for 4.1.
>
>
> > ---
> > net/9p/trans_virtio.c | 25 +++++++++++++++++++++----
> > 1 file changed, 21 insertions(+), 4 deletions(-)
> >
> > diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
> > index d8e376a..d1b2f306 100644
> > --- a/net/9p/trans_virtio.c
> > +++ b/net/9p/trans_virtio.c
> > @@ -658,14 +658,31 @@ p9_virtio_create(struct p9_client *client, const char *devname, char *args)
> > static void p9_virtio_remove(struct virtio_device *vdev)
> > {
> > struct virtio_chan *chan = vdev->priv;
> > -
> > - if (chan->inuse)
> > - p9_virtio_close(chan->client);
> > - vdev->config->del_vqs(vdev);
> > + unsigned long warning_time;
> > + bool inuse;
> >
> > mutex_lock(&virtio_9p_lock);
> > +
> > + /* Remove self from list so we don't get new users. */
> > list_del(&chan->chan_list);
> > + warning_time = jiffies;
> > +
> > + /* Wait for existing users to close. */
> > + while (chan->inuse) {
> > + mutex_unlock(&virtio_9p_lock);
> > + msleep(250);
> > + if (time_after(jiffies, warning_time + 10 * HZ)) {
> > + dev_emerg(&vdev->dev, "p9_virtio_remove: "
> > + "waiting for device in use.\n");
> > + warning_time = jiffies;
> > + }
> > + mutex_lock(&virtio_9p_lock);
> > + }
> > +
> > mutex_unlock(&virtio_9p_lock);
> > +
> > + vdev->config->del_vqs(vdev);
> > +
> > sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
> > kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
> > kfree(chan->tag);
> > --
> > MST
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
2015-03-12 6:36 ` Michael S. Tsirkin
@ 2015-03-13 1:24 ` Rusty Russell
-1 siblings, 0 replies; 8+ messages in thread
From: Rusty Russell @ 2015-03-13 1:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Eric Van Hensbergen, netdev, linux-kernel, virtualization,
v9fs-developer, Ron Minnich, David S. Miller
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Thu, Mar 12, 2015 at 11:54:10AM +1030, Rusty Russell wrote:
>> "Michael S. Tsirkin" <mst@redhat.com> writes:
>> > On device hot-unplug, 9p/virtio currently will kfree channel while
>> > it might still be in use.
>> >
>> > Of course, it might stay used forever, so it's an extremely ugly hack,
>> > but it seems better than use-after-free that we have now.
>> >
>> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>
>> I'll apply it, but it looks like a bandaid.
>
>
> Absolutely.
Applied this, too:
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index 73fab71397ce..781315d97ed5 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -659,7 +659,6 @@ static void p9_virtio_remove(struct virtio_device *vdev)
{
struct virtio_chan *chan = vdev->priv;
unsigned long warning_time;
- bool inuse;
mutex_lock(&virtio_9p_lock);
^ permalink raw reply related [flat|nested] 8+ messages in thread* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
@ 2015-03-13 1:24 ` Rusty Russell
0 siblings, 0 replies; 8+ messages in thread
From: Rusty Russell @ 2015-03-13 1:24 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: linux-kernel, virtualization, Eric Van Hensbergen, Ron Minnich,
Latchesar Ionkov, David S. Miller, v9fs-developer, netdev
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On Thu, Mar 12, 2015 at 11:54:10AM +1030, Rusty Russell wrote:
>> "Michael S. Tsirkin" <mst@redhat.com> writes:
>> > On device hot-unplug, 9p/virtio currently will kfree channel while
>> > it might still be in use.
>> >
>> > Of course, it might stay used forever, so it's an extremely ugly hack,
>> > but it seems better than use-after-free that we have now.
>> >
>> > Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
>>
>> I'll apply it, but it looks like a bandaid.
>
>
> Absolutely.
Applied this, too:
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
index 73fab71397ce..781315d97ed5 100644
--- a/net/9p/trans_virtio.c
+++ b/net/9p/trans_virtio.c
@@ -659,7 +659,6 @@ static void p9_virtio_remove(struct virtio_device *vdev)
{
struct virtio_chan *chan = vdev->priv;
unsigned long warning_time;
- bool inuse;
mutex_lock(&virtio_9p_lock);
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH] 9p/trans_virtio: fix hot-unplug
2015-03-09 14:48 ` Michael S. Tsirkin
(?)
(?)
@ 2015-03-12 1:24 ` Rusty Russell
-1 siblings, 0 replies; 8+ messages in thread
From: Rusty Russell @ 2015-03-12 1:24 UTC (permalink / raw)
To: Michael S. Tsirkin, linux-kernel
Cc: Eric Van Hensbergen, netdev, virtualization, v9fs-developer,
Ron Minnich, David S. Miller
"Michael S. Tsirkin" <mst@redhat.com> writes:
> On device hot-unplug, 9p/virtio currently will kfree channel while
> it might still be in use.
>
> Of course, it might stay used forever, so it's an extremely ugly hack,
> but it seems better than use-after-free that we have now.
>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
I'll apply it, but it looks like a bandaid.
The right answer would seem to be:
1) Use reference counting: 1 for client, 1 for transport.
2) When hot unplug, complete (ie. fail) all outstanding requests.
3) From then on, fail all incoming requests.
4) When refcount hits 0, free the structure.
Thanks,
Rusty.
> ---
> net/9p/trans_virtio.c | 25 +++++++++++++++++++++----
> 1 file changed, 21 insertions(+), 4 deletions(-)
>
> diff --git a/net/9p/trans_virtio.c b/net/9p/trans_virtio.c
> index d8e376a..d1b2f306 100644
> --- a/net/9p/trans_virtio.c
> +++ b/net/9p/trans_virtio.c
> @@ -658,14 +658,31 @@ p9_virtio_create(struct p9_client *client, const char *devname, char *args)
> static void p9_virtio_remove(struct virtio_device *vdev)
> {
> struct virtio_chan *chan = vdev->priv;
> -
> - if (chan->inuse)
> - p9_virtio_close(chan->client);
> - vdev->config->del_vqs(vdev);
> + unsigned long warning_time;
> + bool inuse;
>
> mutex_lock(&virtio_9p_lock);
> +
> + /* Remove self from list so we don't get new users. */
> list_del(&chan->chan_list);
> + warning_time = jiffies;
> +
> + /* Wait for existing users to close. */
> + while (chan->inuse) {
> + mutex_unlock(&virtio_9p_lock);
> + msleep(250);
> + if (time_after(jiffies, warning_time + 10 * HZ)) {
> + dev_emerg(&vdev->dev, "p9_virtio_remove: "
> + "waiting for device in use.\n");
> + warning_time = jiffies;
> + }
> + mutex_lock(&virtio_9p_lock);
> + }
> +
> mutex_unlock(&virtio_9p_lock);
> +
> + vdev->config->del_vqs(vdev);
> +
> sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
> kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
> kfree(chan->tag);
> --
> MST
^ permalink raw reply [flat|nested] 8+ messages in thread