[PATCH 0/1] virtiofsd: Do not use a thread pool by default

[PATCH 0/1] virtiofsd: Do not use a thread pool by default

[Vivek Goyal]

Hi,

This is a proposal to not use a thread pool by default. It hurts
performance in most of the cases. Hence lets switch default so
that most of the users get better performance out of the box.

I have run virtiofs-tests with and without thread pool and posted
results here.

https://github.com/rhvgoyal/virtiofs-tests/tree/master/performance-results/feb-10-2021

One can notice, that except few cases, results are much better with
no thread pool.

Vivek


Vivek Goyal (1):
  virtiofsd: Do not use a thread pool by default

 tools/virtiofsd/fuse_lowlevel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.25.4

[PATCH 1/1] virtiofsd: Do not use a thread pool by default

[Vivek Goyal]

Currently we created a thread pool (With 64 max threads per pool) for
each virtqueue. We hoped that this will provide us with better scalability
and performance.

But in practice, we are getting better numbers in most of the cases
when we don't create a thread pool at all and a single thread per
virtqueue receives the request and processes it.

Hence, I am proposing that we switch to no thread pool by default
(equivalent of --thread-pool-size=0). This will provide out of
box better performance to most of the users. In fact other users
have confirmed that not using a thread pool gives them better
numbers. So why not use this as default. It can be changed when
somebody can fix the issues with thread pool performance.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
---
 tools/virtiofsd/fuse_lowlevel.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/virtiofsd/fuse_lowlevel.c b/tools/virtiofsd/fuse_lowlevel.c
index e94b71110b..fbdf62ee9b 100644
--- a/tools/virtiofsd/fuse_lowlevel.c
+++ b/tools/virtiofsd/fuse_lowlevel.c
@@ -18,7 +18,7 @@
 
 #include <sys/file.h>
 
-#define THREAD_POOL_SIZE 64
+#define THREAD_POOL_SIZE 0
 
 #define OFFSET_MAX 0x7fffffffffffffffLL
 
-- 
2.25.4

Re: [Virtio-fs] [PATCH 1/1] virtiofsd: Do not use a thread pool by default

[Dr. David Alan Gilbert]

* Vivek Goyal (vgoyal@redhat.com) wrote:
> Currently we created a thread pool (With 64 max threads per pool) for
> each virtqueue. We hoped that this will provide us with better scalability
> and performance.
> 
> But in practice, we are getting better numbers in most of the cases
> when we don't create a thread pool at all and a single thread per
> virtqueue receives the request and processes it.
> 
> Hence, I am proposing that we switch to no thread pool by default
> (equivalent of --thread-pool-size=0). This will provide out of
> box better performance to most of the users. In fact other users
> have confirmed that not using a thread pool gives them better
> numbers. So why not use this as default. It can be changed when
> somebody can fix the issues with thread pool performance.
> 
> Signed-off-by: Vivek Goyal <vgoyal@redhat.com>

OK, lets try it - I still worry it really means we're missing something
silly about them, you'd really think eventually the threads should help.



Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>

> ---
>  tools/virtiofsd/fuse_lowlevel.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/virtiofsd/fuse_lowlevel.c b/tools/virtiofsd/fuse_lowlevel.c
> index e94b71110b..fbdf62ee9b 100644
> --- a/tools/virtiofsd/fuse_lowlevel.c
> +++ b/tools/virtiofsd/fuse_lowlevel.c
> @@ -18,7 +18,7 @@
>  
>  #include <sys/file.h>
>  
> -#define THREAD_POOL_SIZE 64
> +#define THREAD_POOL_SIZE 0
>  
>  #define OFFSET_MAX 0x7fffffffffffffffLL
>  
> -- 
> 2.25.4
-- 
Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK

Re: [Virtio-fs] [PATCH 1/1] virtiofsd: Do not use a thread pool by default

[Vivek Goyal]

On Tue, Feb 16, 2021 at 05:55:26PM +0000, Dr. David Alan Gilbert wrote:
> * Vivek Goyal (vgoyal@redhat.com) wrote:
> > Currently we created a thread pool (With 64 max threads per pool) for
> > each virtqueue. We hoped that this will provide us with better scalability
> > and performance.
> > 
> > But in practice, we are getting better numbers in most of the cases
> > when we don't create a thread pool at all and a single thread per
> > virtqueue receives the request and processes it.
> > 
> > Hence, I am proposing that we switch to no thread pool by default
> > (equivalent of --thread-pool-size=0). This will provide out of
> > box better performance to most of the users. In fact other users
> > have confirmed that not using a thread pool gives them better
> > numbers. So why not use this as default. It can be changed when
> > somebody can fix the issues with thread pool performance.
> > 
> > Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
> 
> OK, lets try it - I still worry it really means we're missing something
> silly about them, you'd really think eventually the threads should help.

Agreed. Once we figure out what are we missing with threading and
start getting better performance with multi-threading, we will need
to make thread pool default again.

Vivek
> 
> 
> 
> Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
> 
> > ---
> >  tools/virtiofsd/fuse_lowlevel.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/tools/virtiofsd/fuse_lowlevel.c b/tools/virtiofsd/fuse_lowlevel.c
> > index e94b71110b..fbdf62ee9b 100644
> > --- a/tools/virtiofsd/fuse_lowlevel.c
> > +++ b/tools/virtiofsd/fuse_lowlevel.c
> > @@ -18,7 +18,7 @@
> >  
> >  #include <sys/file.h>
> >  
> > -#define THREAD_POOL_SIZE 64
> > +#define THREAD_POOL_SIZE 0
> >  
> >  #define OFFSET_MAX 0x7fffffffffffffffLL
> >  
> > -- 
> > 2.25.4
> -- 
> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK
> 
> _______________________________________________
> Virtio-fs mailing list
> Virtio-fs@listman.redhat.com
> https://listman.redhat.com/mailman/listinfo/virtio-fs