cpu_active vs pcrypt & padata

cpu_active vs pcrypt & padata

[Peter Zijlstra]

Hi Steffen,

I found cpu_active usage in crypto/pcrypt.c and was wondering what
that's doing there. I would really like to contain that thing to as
narrow a piece of kernel as I possible can (sched/cpuset/hotplug) but it
appears to be spreading.

Also, wth is all this kernel/padata.c stuff? There's next to no useful
comment in there and the only consumer seems to be pcrypt, does that
really need to be in kernel/ ?

Re: cpu_active vs pcrypt & padata

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 16 Mar 2012 11:13:08 +0100

> Also, wth is all this kernel/padata.c stuff? There's next to no useful
> comment in there and the only consumer seems to be pcrypt, does that
> really need to be in kernel/ ?

There's really nothing specific to crypto in that padata.c file,
anything trying to compute things in parallel with some kind
of converging synchronization points can use that code.

Re: cpu_active vs pcrypt & padata

[Peter Zijlstra]

On Fri, 2012-03-16 at 03:16 -0700, David Miller wrote:
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Fri, 16 Mar 2012 11:13:08 +0100
> 
> > Also, wth is all this kernel/padata.c stuff? There's next to no useful
> > comment in there and the only consumer seems to be pcrypt, does that
> > really need to be in kernel/ ?
> 
> There's really nothing specific to crypto in that padata.c file,
> anything trying to compute things in parallel with some kind
> of converging synchronization points can use that code.

So why isn't anybody else using that? Don't the block/fs people need
similar things?

Re: cpu_active vs pcrypt & padata

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Fri, 16 Mar 2012 11:20:15 +0100

> On Fri, 2012-03-16 at 03:16 -0700, David Miller wrote:
>> From: Peter Zijlstra <peterz@infradead.org>
>> Date: Fri, 16 Mar 2012 11:13:08 +0100
>> 
>> > Also, wth is all this kernel/padata.c stuff? There's next to no useful
>> > comment in there and the only consumer seems to be pcrypt, does that
>> > really need to be in kernel/ ?
>> 
>> There's really nothing specific to crypto in that padata.c file,
>> anything trying to compute things in parallel with some kind
>> of converging synchronization points can use that code.
> 
> So why isn't anybody else using that? Don't the block/fs people need
> similar things?

Beats me, but just because there's only one user doesn't mean it belongs
tucked away in that one user's subsystem.

Re: cpu_active vs pcrypt & padata

[Peter Zijlstra]

On Fri, 2012-03-16 at 03:32 -0700, David Miller wrote:

> Beats me, but just because there's only one user doesn't mean it belongs
> tucked away in that one user's subsystem.

Sure, but I was mostly wondering wth it was, the file doesn't really
explain itself. 

Also, most times when introducing generic functionality people try to
collect all interested parties and make something that works for
everybody.

I know for a fact that btrfs does the fan out and regroup thing for
block checksumming etc.. Does Chris even know it exists? If he does why
isn't brtfs using it? If its unsuitable, why wasn't it fixed.

I'm not saying its crap or anything -- I haven't looked at it at all,
beyond my grep for cpu_active -- I'm just wondering, and so far it looks
like there's something weird.

Re: cpu_active vs pcrypt & padata

[Steffen Klassert]

Hi.

On Fri, Mar 16, 2012 at 11:13:08AM +0100, Peter Zijlstra wrote:
> Hi Steffen,
> 
> I found cpu_active usage in crypto/pcrypt.c and was wondering what
> that's doing there. I would really like to contain that thing to as
> narrow a piece of kernel as I possible can (sched/cpuset/hotplug) but it
> appears to be spreading.

pcrypt uses cpu_active to tell padata which cpuset it whishes
to use for parallelization. I could try to push the cpumask
handling down to padata if you want to limit this to the core
kernel.

> 
> Also, wth is all this kernel/padata.c stuff? There's next to no useful
> comment in there and the only consumer seems to be pcrypt, does that
> really need to be in kernel/ ?

The padata code is generic and not limited to crypto, you can find
a documentation at Documentation/padata.txt.

Re: cpu_active vs pcrypt & padata

[Peter Zijlstra]

On Fri, 2012-03-16 at 11:44 +0100, Steffen Klassert wrote:
> Hi.
> 
> On Fri, Mar 16, 2012 at 11:13:08AM +0100, Peter Zijlstra wrote:
> > Hi Steffen,
> > 
> > I found cpu_active usage in crypto/pcrypt.c and was wondering what
> > that's doing there. I would really like to contain that thing to as
> > narrow a piece of kernel as I possible can (sched/cpuset/hotplug) but it
> > appears to be spreading.
> 
> pcrypt uses cpu_active to tell padata which cpuset it whishes
> to use for parallelization. I could try to push the cpumask
> handling down to padata if you want to limit this to the core
> kernel.

/me more confused now.. cpu_active isn't in any way shape or form
related to cpusets.

> > Also, wth is all this kernel/padata.c stuff? There's next to no useful
> > comment in there and the only consumer seems to be pcrypt, does that
> > really need to be in kernel/ ?
> 
> The padata code is generic and not limited to crypto, you can find
> a documentation at Documentation/padata.txt.

Would be nice to have a short blurb in kernel/padata.c with a reference
to that Documentation stuff for more in-depth bits.

Re: cpu_active vs pcrypt & padata

[Steffen Klassert]

On Sat, Mar 17, 2012 at 10:00:34PM +0100, Peter Zijlstra wrote:
> On Fri, 2012-03-16 at 11:44 +0100, Steffen Klassert wrote:
> > 
> > pcrypt uses cpu_active to tell padata which cpuset it whishes
> > to use for parallelization. I could try to push the cpumask
> > handling down to padata if you want to limit this to the core
> > kernel.
> 
> /me more confused now.. cpu_active isn't in any way shape or form
> related to cpusets.

When looking at this, it seems that cpu_online_mask is the mask
we should use instead of cpu_active_mask. I'll go in for a closer
look at the end of the week.

> > 
> > The padata code is generic and not limited to crypto, you can find
> > a documentation at Documentation/padata.txt.
> 
> Would be nice to have a short blurb in kernel/padata.c with a reference
> to that Documentation stuff for more in-depth bits.

Indeed, I'll take care for this.

Re: cpu_active vs pcrypt & padata

[Jonathan Corbet]

On Fri, 16 Mar 2012 11:13:08 +0100
Peter Zijlstra <peterz@infradead.org> wrote:

> Also, wth is all this kernel/padata.c stuff? There's next to no useful
> comment in there and the only consumer seems to be pcrypt, does that
> really need to be in kernel/ ?

FWIW I wrote that stuff up when it went in...

	https://lwn.net/Articles/382257/

Might be helpful in case anybody else thinks they might want to use this
facility.  Maybe.

jon