Re: [PATCH] CIFS: make cifsd (more)

Re: [PATCH] CIFS: make cifsd (more)

[Christoph Hellwig]

On Mon, Jun 25, 2007 at 05:25:00PM -0500, Steve French wrote:
> Jeff,
> Not seeing any objections to your revised approach (to not allowing
> signals for cifsd kernel thread), I just merged something similar to
> your patch to the cifs-2.6.git tree (also fixed some nearby lines that
> went past 80 columns).

Ok, I'm back to this.

As I said mixing force_sig with the kthread infrastructure is a bad idea.
The proper short-term (aka 2.6.22) fix is to revert the kthread conversion
for this particular thread.  Just go back to what worked before.

Now the right fix is a lot more complicated and involved:

	Stop using blocking recvmsg (or read) in kernel threads!

If you look at what the other consumers of networking reads from kernel
threads do is they either use tcp_read_sock and hooks into the sk_ callbacks
which would be nice for high performance reads in cifs aswell, but probably
not the demultiplexer thread, or they use MSG_DONTWAIT to avoid this problems
and deal with the blocking behaviour on a higher level.

Re: [PATCH] CIFS: make cifsd (more)

[Jeff Layton]

On Sat, 30 Jun 2007 09:42:09 +0100
Christoph Hellwig <hch@infradead.org> wrote:

> On Mon, Jun 25, 2007 at 05:25:00PM -0500, Steve French wrote:
> > Jeff,
> > Not seeing any objections to your revised approach (to not allowing
> > signals for cifsd kernel thread), I just merged something similar to
> > your patch to the cifs-2.6.git tree (also fixed some nearby lines that
> > went past 80 columns).
> 
> Ok, I'm back to this.
> 
> As I said mixing force_sig with the kthread infrastructure is a bad idea.
> The proper short-term (aka 2.6.22) fix is to revert the kthread conversion
> for this particular thread.  Just go back to what worked before.

Could you clarify why this is? It looks like kthreads and signalling
should be more or less orthogonal. Or is it just an issue of the
complexity added when you mix signalling into kthreads?

Note that the problem of insulation from userspace signals predates the
conversion to using the kthreads interface for cifsd. So even if we
revert the switch of the demultiplexer thread to kthreads in the near
term, I'd like to keep the recent change to block all signals from
userspace and use force_sig in lieu of send_sig.

Does that sound reasonable?

>
> Now the right fix is a lot more complicated and involved:
> 
> 	Stop using blocking recvmsg (or read) in kernel threads!
> 
> If you look at what the other consumers of networking reads from kernel
> threads do is they either use tcp_read_sock and hooks into the sk_ callbacks
> which would be nice for high performance reads in cifs aswell, but probably
> not the demultiplexer thread, or they use MSG_DONTWAIT to avoid this problems
> and deal with the blocking behaviour on a higher level.

-- 
Jeff Layton <jlayton@redhat.com>

Re: [PATCH] CIFS: make cifsd (more)

[Steve French]

The reason that cifs switched from wait_for_completion to the kthread
call to cifs_demultiplex_thread in the first place is because without
use of kthread it won't work with a linux-vserver.   See the thread:

http://marc.info/?l=linux-cifs-client&m=117552761703381&w=2

If we take out the kthread call, we break those guys.

I agree that using sk_callbacks is worth looking into - I only found
ocfs2 and SunRPC (NFS) though that used it.   Is there a better
example though?   The NFS socket handling code is huge
(net/sunrpc/xprtsck.c) - something seems wrong when replacing a few
lines of code with a new 1675 line file.  There must be a better
example of doing what you suggest...

I am tempted to drop the socket timeout (which cifs sets to 7 seconds)
to a smaller number and not use signals at all rather than add that
much complexity

On 6/30/07, Jeff Layton <jlayton@redhat.com> wrote:
> On Sat, 30 Jun 2007 09:42:09 +0100
> Christoph Hellwig <hch@infradead.org> wrote:
>
> > On Mon, Jun 25, 2007 at 05:25:00PM -0500, Steve French wrote:
> > > Jeff,
> > > Not seeing any objections to your revised approach (to not allowing
> > > signals for cifsd kernel thread), I just merged something similar to
> > > your patch to the cifs-2.6.git tree (also fixed some nearby lines that
> > > went past 80 columns).
> >
> > Ok, I'm back to this.
> >
> > As I said mixing force_sig with the kthread infrastructure is a bad idea.
> > The proper short-term (aka 2.6.22) fix is to revert the kthread conversion
> > for this particular thread.  Just go back to what worked before.
>
> Could you clarify why this is? It looks like kthreads and signalling
> should be more or less orthogonal. Or is it just an issue of the
> complexity added when you mix signalling into kthreads?
>
> Note that the problem of insulation from userspace signals predates the
> conversion to using the kthreads interface for cifsd. So even if we
> revert the switch of the demultiplexer thread to kthreads in the near
> term, I'd like to keep the recent change to block all signals from
> userspace and use force_sig in lieu of send_sig.
>
> Does that sound reasonable?
>
> >
> > Now the right fix is a lot more complicated and involved:
> >
> >       Stop using blocking recvmsg (or read) in kernel threads!
> >
> > If you look at what the other consumers of networking reads from kernel
> > threads do is they either use tcp_read_sock and hooks into the sk_ callbacks
> > which would be nice for high performance reads in cifs aswell, but probably
> > not the demultiplexer thread, or they use MSG_DONTWAIT to avoid this problems
> > and deal with the blocking behaviour on a higher level.
>
> --
> Jeff Layton <jlayton@redhat.com>
>


-- 
Thanks,

Steve

Re: [PATCH] CIFS: make cifsd (more)

[Satyam Sharma]

[ Trimmed Cc: list ]

On 6/30/07, Steve French <smfrench@gmail.com> wrote:
> The reason that cifs switched from wait_for_completion to the kthread
> call to cifs_demultiplex_thread in the first place is because without
> use of kthread it won't work with a linux-vserver.   See the thread:
>
> http://marc.info/?l=linux-cifs-client&m=117552761703381&w=2
>
> If we take out the kthread call, we break those guys.
>
> I agree that using sk_callbacks is worth looking into - I only found
> ocfs2 and SunRPC (NFS) though that used it.   Is there a better
> example though?   The NFS socket handling code is huge
> (net/sunrpc/xprtsck.c) - something seems wrong when replacing a few
> lines of code with a new 1675 line file.  There must be a better
> example of doing what you suggest...

You're correct. "Right" / "elegant" solutions are rarely (if ever?) complex
and involved. Simplicity _is_ good. I see no point in converting 5 good
lines of maintainable, readable, solid code with 1000 lines of kludge :-)
just to work-around this kthreads limitation. But then, of course, the call
is yours.

> I am tempted to drop the socket timeout (which cifs sets to 7 seconds)
> to a smaller number and not use signals at all rather than add that
> much complexity

Timeout too low => CPU wastage => power wastage. [ Think laptop
batteries, with say 5 cifsd kthreads waking up once every second ... ]
Timeout too high => umount(2) hangs, annoys user, user takes
drastic actions ... so think of some good "magic number" :-)

I don't quite think of all these suggestions as solutions at all -- they
are workarounds at best, IMHO (for kthread's limitation in dealing with
kernel threads that want to block -- I still don't see any fundamental
reasoning / logic behind why kthreads should be banned from doing
blocking recv's -- if there is, please let me know too).

Don't have much else to say than what I already have on the two
threads discussing this.

Satyam