Re: Multi Core Support for compression in compression.c

[Hugo Mills <hugo@carfax.org.uk>]

[-- Attachment #1: Type: text/plain, Size: 3389 bytes --]

On Sun, Jul 27, 2014 at 11:21:53PM -0400, Nick Krause wrote:
> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
> > On 07/27/2014 04:47 PM, Nick Krause wrote:
> >> This may be a bad idea , but compression in brtfs seems to be only
> >> using one core to compress.
> >> Depending on the CPU used and the amount of cores in the CPU we can
> >> make this much faster
> >> with multiple cores. This seems bad by my reading at least I would
> >> recommend for writing compression
> >> we write a function to use a certain amount of cores based on the load
> >> of the system's CPU not using
> >> more then 75% of the system's CPU resources as my system when idle has
> >> never needed more
> >> then one core of my i5 2500k to run when with interrupts for opening
> >> eclipse are running. For reading
> >> compression on good core seems fine to me as testing other compression
> >> software for reads , it's
> >> way less CPU intensive.
> >> Cheers Nick
> > We would probably get a bigger benefit from taking an approach like
> > SquashFS has recently added, that is, allowing multi-threaded
> > decompression fro reads, and decompressing directly into the pagecache.
> >  Such an approach would likely make zlib compression much more scalable
> > on large systems.
> >
> >
> 
> Austin,
> That seems better then my idea as you seem to be more up to date on
> brtfs devolopment.
> If you and the other developers of brtfs are interested in adding this
> as a feature please let
> me known as I would like to help improve brtfs as the file system as
> an idea is great just
> seems like it needs a lot of work :).

   Yes, it probably does need a lot of work. This is (at least one
reason) why it's not been done yet. If you want to work on doing this,
then please do. However, don't expect anyone else to give you a
detailed plan of what code to write. Don't expect anyone else to write
the code for you. You will have to come up with your own ideas as to
how to implement it, and actually do it yourself, including building
it, and testing it.

   That's not to say that you are on your own, though. People will
help -- provided that you aren't asking them to do all the work. You
are not an empty vessel to be filled with the wisdom of the ancients.
This means that *you* have to take action. You have to take yourself
as far as you can in learning how things work. When you get stuck,
work out what it is that you don't know, and then ask about that one
thing. This makes it easier to answer, it shows that you're putting in
effort on your side, and it means that you *actually learn things*.
Questions like "what function should I be modifying?", or "how do you
want me to do this?" show that you haven't put in even the smallest
piece of effort, and will be ignored (f you're lucky). Questions like
"I'm trying to implement a crumble filter, but in the mix_breadcrumbs
function, how does it take account of the prestressed_yoghurt field?"
show that you've read and understood at least some of the code, and
have thought about what it's doing.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Alert status mauve ocelot: Slight chance of brimstone. Be ---    
                   prepared to make a nice cup of tea.                   

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]