[PATCH] xen-blkback: prevent premature module unload

[PATCH] xen-blkback: prevent premature module unload

[Paul Durrant]

Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
cache. This cache is destoyed when xen-blkif is unloaded so it is
necessary to wait for the deferred free routine used for such objects to
complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by
taking/releasing extra module references in xen_blkif_alloc/free()
respectively.

Signed-off-by: Paul Durrant <pdurrant@amazon.com>
---
Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
Cc: "Roger Pau Monné" <roger.pau@citrix.com>
Cc: Jens Axboe <axboe@kernel.dk>
---
 drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index e8c5c54e1d26..59d576d27ca7 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
 	blkif->domid = domid;
 	atomic_set(&blkif->refcnt, 1);
 	init_completion(&blkif->drain_complete);
+
+	/*
+	 * Because freeing back to the cache may be deferred, it is not
+	 * safe to unload the module (and hence destroy the cache) until
+	 * this has completed. To prevent premature unloading, take an
+	 * extra module reference here and release only when the object
+	 * has been free back to the cache.
+	 */
+	__module_get(THIS_MODULE);
 	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
 
 	return blkif;
@@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
 
 	/* Make sure everything is drained before shutting down */
 	kmem_cache_free(xen_blkif_cachep, blkif);
+	module_put(THIS_MODULE);
 }
 
 int __init xen_blkif_interface_init(void)
-- 
2.20.1

Re: [PATCH] xen-blkback: prevent premature module unload

[Roger Pau Monné]

On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
> 
> Signed-off-by: Paul Durrant <pdurrant@amazon.com>

Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>

One nit below.

> ---
> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> Cc: Jens Axboe <axboe@kernel.dk>
> ---
>  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
>  1 file changed, 10 insertions(+)
> 
> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
> index e8c5c54e1d26..59d576d27ca7 100644
> --- a/drivers/block/xen-blkback/xenbus.c
> +++ b/drivers/block/xen-blkback/xenbus.c
> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
>  	blkif->domid = domid;
>  	atomic_set(&blkif->refcnt, 1);
>  	init_completion(&blkif->drain_complete);
> +
> +	/*
> +	 * Because freeing back to the cache may be deferred, it is not
> +	 * safe to unload the module (and hence destroy the cache) until
> +	 * this has completed. To prevent premature unloading, take an
> +	 * extra module reference here and release only when the object
> +	 * has been free back to the cache.
                    ^ freed
> +	 */
> +	__module_get(THIS_MODULE);
>  	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
>  
>  	return blkif;
> @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
>  
>  	/* Make sure everything is drained before shutting down */
>  	kmem_cache_free(xen_blkif_cachep, blkif);
> +	module_put(THIS_MODULE);
>  }
>  
>  int __init xen_blkif_interface_init(void)
> -- 
> 2.20.1
> 

RE: [PATCH] xen-blkback: prevent premature module unload

[Durrant, Paul]

> -----Original Message-----
> From: Roger Pau Monné <roger.pau@citrix.com>
> Sent: 11 December 2019 11:29
> To: Durrant, Paul <pdurrant@amazon.com>
> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> Jens Axboe <axboe@kernel.dk>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> 
> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > necessary to wait for the deferred free routine used for such objects to
> > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > allow module to be cleanly unloaded". This patch fixes the problem by
> > taking/releasing extra module references in xen_blkif_alloc/free()
> > respectively.
> >
> > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> 
> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> 
> One nit below.
> 
> > ---
> > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > Cc: Jens Axboe <axboe@kernel.dk>
> > ---
> >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> >  1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> blkback/xenbus.c
> > index e8c5c54e1d26..59d576d27ca7 100644
> > --- a/drivers/block/xen-blkback/xenbus.c
> > +++ b/drivers/block/xen-blkback/xenbus.c
> > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> domid)
> >  	blkif->domid = domid;
> >  	atomic_set(&blkif->refcnt, 1);
> >  	init_completion(&blkif->drain_complete);
> > +
> > +	/*
> > +	 * Because freeing back to the cache may be deferred, it is not
> > +	 * safe to unload the module (and hence destroy the cache) until
> > +	 * this has completed. To prevent premature unloading, take an
> > +	 * extra module reference here and release only when the object
> > +	 * has been free back to the cache.
>                     ^ freed

Oh yes. Can this be done on commit, or would you like me to send a v2?

  Paul

> > +	 */
> > +	__module_get(THIS_MODULE);
> >  	INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
> >
> >  	return blkif;
> > @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
> >
> >  	/* Make sure everything is drained before shutting down */
> >  	kmem_cache_free(xen_blkif_cachep, blkif);
> > +	module_put(THIS_MODULE);
> >  }
> >
> >  int __init xen_blkif_interface_init(void)
> > --
> > 2.20.1
> >

Re: [PATCH] xen-blkback: prevent premature module unload

[Roger Pau Monné]

On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monné <roger.pau@citrix.com>
> > Sent: 11 December 2019 11:29
> > To: Durrant, Paul <pdurrant@amazon.com>
> > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> > kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> > Jens Axboe <axboe@kernel.dk>
> > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> > 
> > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > necessary to wait for the deferred free routine used for such objects to
> > > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > > allow module to be cleanly unloaded". This patch fixes the problem by
> > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > respectively.
> > >
> > > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> > 
> > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> > 
> > One nit below.
> > 
> > > ---
> > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > > Cc: Jens Axboe <axboe@kernel.dk>
> > > ---
> > >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > >  1 file changed, 10 insertions(+)
> > >
> > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > blkback/xenbus.c
> > > index e8c5c54e1d26..59d576d27ca7 100644
> > > --- a/drivers/block/xen-blkback/xenbus.c
> > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> > domid)
> > >  	blkif->domid = domid;
> > >  	atomic_set(&blkif->refcnt, 1);
> > >  	init_completion(&blkif->drain_complete);
> > > +
> > > +	/*
> > > +	 * Because freeing back to the cache may be deferred, it is not
> > > +	 * safe to unload the module (and hence destroy the cache) until
> > > +	 * this has completed. To prevent premature unloading, take an
> > > +	 * extra module reference here and release only when the object
> > > +	 * has been free back to the cache.
> >                     ^ freed
> 
> Oh yes. Can this be done on commit, or would you like me to send a v2?

Adjusting on commit would be fine for me, but it's up to Juergen since
he is the one that will pick this up. IIRC the module unload patches
didn't go through the block subsystem.

Thanks, Roger.

Re: [PATCH] xen-blkback: prevent premature module unload

[Jürgen Groß]

On 11.12.19 14:55, Roger Pau Monné wrote:
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
>>> -----Original Message-----
>>> From: Roger Pau Monné <roger.pau@citrix.com>
>>> Sent: 11 December 2019 11:29
>>> To: Durrant, Paul <pdurrant@amazon.com>
>>> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
>>> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
>>> Jens Axboe <axboe@kernel.dk>
>>> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
>>>
>>> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
>>>> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
>>>> cache. This cache is destoyed when xen-blkif is unloaded so it is
>>>> necessary to wait for the deferred free routine used for such objects to
>>>> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
>>>> allow module to be cleanly unloaded". This patch fixes the problem by
>>>> taking/releasing extra module references in xen_blkif_alloc/free()
>>>> respectively.
>>>>
>>>> Signed-off-by: Paul Durrant <pdurrant@amazon.com>
>>>
>>> Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
>>>
>>> One nit below.
>>>
>>>> ---
>>>> Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
>>>> Cc: "Roger Pau Monné" <roger.pau@citrix.com>
>>>> Cc: Jens Axboe <axboe@kernel.dk>
>>>> ---
>>>>   drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
>>>>   1 file changed, 10 insertions(+)
>>>>
>>>> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
>>> blkback/xenbus.c
>>>> index e8c5c54e1d26..59d576d27ca7 100644
>>>> --- a/drivers/block/xen-blkback/xenbus.c
>>>> +++ b/drivers/block/xen-blkback/xenbus.c
>>>> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
>>> domid)
>>>>   	blkif->domid = domid;
>>>>   	atomic_set(&blkif->refcnt, 1);
>>>>   	init_completion(&blkif->drain_complete);
>>>> +
>>>> +	/*
>>>> +	 * Because freeing back to the cache may be deferred, it is not
>>>> +	 * safe to unload the module (and hence destroy the cache) until
>>>> +	 * this has completed. To prevent premature unloading, take an
>>>> +	 * extra module reference here and release only when the object
>>>> +	 * has been free back to the cache.
>>>                      ^ freed
>>
>> Oh yes. Can this be done on commit, or would you like me to send a v2?
> 
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.

Oh, right. Yes, will fix this when committing.


Juergen

RE: [PATCH] xen-blkback: prevent premature module unload

[Durrant, Paul]

> -----Original Message-----
> From: Roger Pau Monné <roger.pau@citrix.com>
> Sent: 11 December 2019 13:55
> To: Durrant, Paul <pdurrant@amazon.com>; Juergen Gross <jgross@suse.com>
> Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org; linux-
> kernel@vger.kernel.org; Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>;
> Jens Axboe <axboe@kernel.dk>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> 
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > > -----Original Message-----
> > > From: Roger Pau Monné <roger.pau@citrix.com>
> > > Sent: 11 December 2019 11:29
> > > To: Durrant, Paul <pdurrant@amazon.com>
> > > Cc: xen-devel@lists.xenproject.org; linux-block@vger.kernel.org;
> linux-
> > > kernel@vger.kernel.org; Konrad Rzeszutek Wilk
> <konrad.wilk@oracle.com>;
> > > Jens Axboe <axboe@kernel.dk>
> > > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> > >
> > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache'
> kmem
> > > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > > necessary to wait for the deferred free routine used for such
> objects to
> > > > complete. This necessity was missed in commit 14855954f636 "xen-
> blkback:
> > > > allow module to be cleanly unloaded". This patch fixes the problem
> by
> > > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > > respectively.
> > > >
> > > > Signed-off-by: Paul Durrant <pdurrant@amazon.com>
> > >
> > > Reviewed-by: Roger Pau Monné <roger.pau@citrix.com>
> > >
> > > One nit below.
> > >
> > > > ---
> > > > Cc: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
> > > > Cc: "Roger Pau Monné" <roger.pau@citrix.com>
> > > > Cc: Jens Axboe <axboe@kernel.dk>
> > > > ---
> > > >  drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > > blkback/xenbus.c
> > > > index e8c5c54e1d26..59d576d27ca7 100644
> > > > --- a/drivers/block/xen-blkback/xenbus.c
> > > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > > @@ -171,6 +171,15 @@ static struct xen_blkif
> *xen_blkif_alloc(domid_t
> > > domid)
> > > >  	blkif->domid = domid;
> > > >  	atomic_set(&blkif->refcnt, 1);
> > > >  	init_completion(&blkif->drain_complete);
> > > > +
> > > > +	/*
> > > > +	 * Because freeing back to the cache may be deferred, it is
> not
> > > > +	 * safe to unload the module (and hence destroy the cache)
> until
> > > > +	 * this has completed. To prevent premature unloading, take an
> > > > +	 * extra module reference here and release only when the
> object
> > > > +	 * has been free back to the cache.
> > >                     ^ freed
> >
> > Oh yes. Can this be done on commit, or would you like me to send a v2?
> 
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.

True. I forgot manually add Juergen cc list.

  Paul

> 
> Thanks, Roger.

Re: [PATCH] xen-blkback: prevent premature module unload

[Jürgen Groß]

On 10.12.19 15:53, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
> 
> Signed-off-by: Paul Durrant <pdurrant@amazon.com>

Pushed to xen/tip.git for-linus-5.5b


Juergen