Multi Core Support for compression in compression.c

Multi Core Support for compression in compression.c

[Nick Krause]

This may be a bad idea , but compression in brtfs seems to be only
using one core to compress.
Depending on the CPU used and the amount of cores in the CPU we can
make this much faster
with multiple cores. This seems bad by my reading at least I would
recommend for writing compression
we write a function to use a certain amount of cores based on the load
of the system's CPU not using
more then 75% of the system's CPU resources as my system when idle has
never needed more
then one core of my i5 2500k to run when with interrupts for opening
eclipse are running. For reading
compression on good core seems fine to me as testing other compression
software for reads , it's
way less CPU intensive.
Cheers Nick

Re: Multi Core Support for compression in compression.c

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 1104 bytes --]

On 07/27/2014 04:47 PM, Nick Krause wrote:
> This may be a bad idea , but compression in brtfs seems to be only
> using one core to compress.
> Depending on the CPU used and the amount of cores in the CPU we can
> make this much faster
> with multiple cores. This seems bad by my reading at least I would
> recommend for writing compression
> we write a function to use a certain amount of cores based on the load
> of the system's CPU not using
> more then 75% of the system's CPU resources as my system when idle has
> never needed more
> then one core of my i5 2500k to run when with interrupts for opening
> eclipse are running. For reading
> compression on good core seems fine to me as testing other compression
> software for reads , it's
> way less CPU intensive.
> Cheers Nick
We would probably get a bigger benefit from taking an approach like
SquashFS has recently added, that is, allowing multi-threaded
decompression fro reads, and decompressing directly into the pagecache.
 Such an approach would likely make zlib compression much more scalable
on large systems.



[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 07/27/2014 04:47 PM, Nick Krause wrote:
>> This may be a bad idea , but compression in brtfs seems to be only
>> using one core to compress.
>> Depending on the CPU used and the amount of cores in the CPU we can
>> make this much faster
>> with multiple cores. This seems bad by my reading at least I would
>> recommend for writing compression
>> we write a function to use a certain amount of cores based on the load
>> of the system's CPU not using
>> more then 75% of the system's CPU resources as my system when idle has
>> never needed more
>> then one core of my i5 2500k to run when with interrupts for opening
>> eclipse are running. For reading
>> compression on good core seems fine to me as testing other compression
>> software for reads , it's
>> way less CPU intensive.
>> Cheers Nick
> We would probably get a bigger benefit from taking an approach like
> SquashFS has recently added, that is, allowing multi-threaded
> decompression fro reads, and decompressing directly into the pagecache.
>  Such an approach would likely make zlib compression much more scalable
> on large systems.
>
>

Austin,
That seems better then my idea as you seem to be more up to date on
brtfs devolopment.
If you and the other developers of brtfs are interested in adding this
as a feature please let
me known as I would like to help improve brtfs as the file system as
an idea is great just
seems like it needs a lot of work :).
Nick

Re: Multi Core Support for compression in compression.c

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 3389 bytes --]

On Sun, Jul 27, 2014 at 11:21:53PM -0400, Nick Krause wrote:
> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
> > On 07/27/2014 04:47 PM, Nick Krause wrote:
> >> This may be a bad idea , but compression in brtfs seems to be only
> >> using one core to compress.
> >> Depending on the CPU used and the amount of cores in the CPU we can
> >> make this much faster
> >> with multiple cores. This seems bad by my reading at least I would
> >> recommend for writing compression
> >> we write a function to use a certain amount of cores based on the load
> >> of the system's CPU not using
> >> more then 75% of the system's CPU resources as my system when idle has
> >> never needed more
> >> then one core of my i5 2500k to run when with interrupts for opening
> >> eclipse are running. For reading
> >> compression on good core seems fine to me as testing other compression
> >> software for reads , it's
> >> way less CPU intensive.
> >> Cheers Nick
> > We would probably get a bigger benefit from taking an approach like
> > SquashFS has recently added, that is, allowing multi-threaded
> > decompression fro reads, and decompressing directly into the pagecache.
> >  Such an approach would likely make zlib compression much more scalable
> > on large systems.
> >
> >
> 
> Austin,
> That seems better then my idea as you seem to be more up to date on
> brtfs devolopment.
> If you and the other developers of brtfs are interested in adding this
> as a feature please let
> me known as I would like to help improve brtfs as the file system as
> an idea is great just
> seems like it needs a lot of work :).

   Yes, it probably does need a lot of work. This is (at least one
reason) why it's not been done yet. If you want to work on doing this,
then please do. However, don't expect anyone else to give you a
detailed plan of what code to write. Don't expect anyone else to write
the code for you. You will have to come up with your own ideas as to
how to implement it, and actually do it yourself, including building
it, and testing it.

   That's not to say that you are on your own, though. People will
help -- provided that you aren't asking them to do all the work. You
are not an empty vessel to be filled with the wisdom of the ancients.
This means that *you* have to take action. You have to take yourself
as far as you can in learning how things work. When you get stuck,
work out what it is that you don't know, and then ask about that one
thing. This makes it easier to answer, it shows that you're putting in
effort on your side, and it means that you *actually learn things*.
Questions like "what function should I be modifying?", or "how do you
want me to do this?" show that you haven't put in even the smallest
piece of effort, and will be ignored (f you're lucky). Questions like
"I'm trying to implement a crumble filter, but in the mix_breadcrumbs
function, how does it take account of the prestressed_yoghurt field?"
show that you've read and understood at least some of the code, and
have thought about what it's doing.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
    --- Alert status mauve ocelot: Slight chance of brimstone. Be ---    
                   prepared to make a nice cup of tea.                   

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: Multi Core Support for compression in compression.c

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 2296 bytes --]

On 07/27/2014 11:21 PM, Nick Krause wrote:
> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>> This may be a bad idea , but compression in brtfs seems to be only
>>> using one core to compress.
>>> Depending on the CPU used and the amount of cores in the CPU we can
>>> make this much faster
>>> with multiple cores. This seems bad by my reading at least I would
>>> recommend for writing compression
>>> we write a function to use a certain amount of cores based on the load
>>> of the system's CPU not using
>>> more then 75% of the system's CPU resources as my system when idle has
>>> never needed more
>>> then one core of my i5 2500k to run when with interrupts for opening
>>> eclipse are running. For reading
>>> compression on good core seems fine to me as testing other compression
>>> software for reads , it's
>>> way less CPU intensive.
>>> Cheers Nick
>> We would probably get a bigger benefit from taking an approach like
>> SquashFS has recently added, that is, allowing multi-threaded
>> decompression fro reads, and decompressing directly into the pagecache.
>>  Such an approach would likely make zlib compression much more scalable
>> on large systems.
>>
>>
> 
> Austin,
> That seems better then my idea as you seem to be more up to date on
> brtfs devolopment.
> If you and the other developers of brtfs are interested in adding this
> as a feature please let
> me known as I would like to help improve brtfs as the file system as
> an idea is great just
> seems like it needs a lot of work :).
> Nick
I wouldn't say that I am a BTRFS developer (power user maybe?), but I
would definitely say that parallelizing compression on writes would be a
good idea too (especially for things like lz4, which IIRC is either in
3.16 or in the queue for 3.17).  Both options would be a lot of work,
but almost any performance optimization would.  I would almost say that
it would provide a bigger performance improvement to get BTRFS to
intelligently stripe reads and writes (at the moment, any given worker
thread only dispatches one write or read to a single device at a time,
and any given write() or read() syscall gets handled by only one worker).


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 07/27/2014 11:21 PM, Nick Krause wrote:
>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>> <ahferroin7@gmail.com> wrote:
>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>> This may be a bad idea , but compression in brtfs seems to be only
>>>> using one core to compress.
>>>> Depending on the CPU used and the amount of cores in the CPU we can
>>>> make this much faster
>>>> with multiple cores. This seems bad by my reading at least I would
>>>> recommend for writing compression
>>>> we write a function to use a certain amount of cores based on the load
>>>> of the system's CPU not using
>>>> more then 75% of the system's CPU resources as my system when idle has
>>>> never needed more
>>>> then one core of my i5 2500k to run when with interrupts for opening
>>>> eclipse are running. For reading
>>>> compression on good core seems fine to me as testing other compression
>>>> software for reads , it's
>>>> way less CPU intensive.
>>>> Cheers Nick
>>> We would probably get a bigger benefit from taking an approach like
>>> SquashFS has recently added, that is, allowing multi-threaded
>>> decompression fro reads, and decompressing directly into the pagecache.
>>>  Such an approach would likely make zlib compression much more scalable
>>> on large systems.
>>>
>>>
>>
>> Austin,
>> That seems better then my idea as you seem to be more up to date on
>> brtfs devolopment.
>> If you and the other developers of brtfs are interested in adding this
>> as a feature please let
>> me known as I would like to help improve brtfs as the file system as
>> an idea is great just
>> seems like it needs a lot of work :).
>> Nick
> I wouldn't say that I am a BTRFS developer (power user maybe?), but I
> would definitely say that parallelizing compression on writes would be a
> good idea too (especially for things like lz4, which IIRC is either in
> 3.16 or in the queue for 3.17).  Both options would be a lot of work,
> but almost any performance optimization would.  I would almost say that
> it would provide a bigger performance improvement to get BTRFS to
> intelligently stripe reads and writes (at the moment, any given worker
> thread only dispatches one write or read to a single device at a time,
> and any given write() or read() syscall gets handled by only one worker).
>

I will look into this idea and see if I can do this for writes.
Regards Nick

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com> wrote:
> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>> This may be a bad idea , but compression in brtfs seems to be only
>>>>> using one core to compress.
>>>>> Depending on the CPU used and the amount of cores in the CPU we can
>>>>> make this much faster
>>>>> with multiple cores. This seems bad by my reading at least I would
>>>>> recommend for writing compression
>>>>> we write a function to use a certain amount of cores based on the load
>>>>> of the system's CPU not using
>>>>> more then 75% of the system's CPU resources as my system when idle has
>>>>> never needed more
>>>>> then one core of my i5 2500k to run when with interrupts for opening
>>>>> eclipse are running. For reading
>>>>> compression on good core seems fine to me as testing other compression
>>>>> software for reads , it's
>>>>> way less CPU intensive.
>>>>> Cheers Nick
>>>> We would probably get a bigger benefit from taking an approach like
>>>> SquashFS has recently added, that is, allowing multi-threaded
>>>> decompression fro reads, and decompressing directly into the pagecache.
>>>>  Such an approach would likely make zlib compression much more scalable
>>>> on large systems.
>>>>
>>>>
>>>
>>> Austin,
>>> That seems better then my idea as you seem to be more up to date on
>>> brtfs devolopment.
>>> If you and the other developers of brtfs are interested in adding this
>>> as a feature please let
>>> me known as I would like to help improve brtfs as the file system as
>>> an idea is great just
>>> seems like it needs a lot of work :).
>>> Nick
>> I wouldn't say that I am a BTRFS developer (power user maybe?), but I
>> would definitely say that parallelizing compression on writes would be a
>> good idea too (especially for things like lz4, which IIRC is either in
>> 3.16 or in the queue for 3.17).  Both options would be a lot of work,
>> but almost any performance optimization would.  I would almost say that
>> it would provide a bigger performance improvement to get BTRFS to
>> intelligently stripe reads and writes (at the moment, any given worker
>> thread only dispatches one write or read to a single device at a time,
>> and any given write() or read() syscall gets handled by only one worker).
>>
>
> I will look into this idea and see if I can do this for writes.
> Regards Nick

Austin,
Seems since we don't want to release the cache for inodes in order to
improve writes if
are going to use the page cache. We seem to be doing this for writes in
end_compressed_bio_write for standard pages and in end_compressed_bio_write.
If we want to cache write pages why are we removing then ? Seems like this needs
to be removed in order to start off.
Regards Nick

Re: Multi Core Support for compression in compression.c

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 3800 bytes --]

On 2014-07-28 11:57, Nick Krause wrote:
> On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com>
> wrote:
>> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn 
>> <ahferroin7@gmail.com> wrote:
>>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn 
>>>> <ahferroin7@gmail.com> wrote:
>>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>>> This may be a bad idea , but compression in brtfs seems
>>>>>> to be only using one core to compress. Depending on the
>>>>>> CPU used and the amount of cores in the CPU we can make
>>>>>> this much faster with multiple cores. This seems bad by
>>>>>> my reading at least I would recommend for writing
>>>>>> compression we write a function to use a certain amount
>>>>>> of cores based on the load of the system's CPU not using 
>>>>>> more then 75% of the system's CPU resources as my system
>>>>>> when idle has never needed more then one core of my i5
>>>>>> 2500k to run when with interrupts for opening eclipse are
>>>>>> running. For reading compression on good core seems fine
>>>>>> to me as testing other compression software for reads ,
>>>>>> it's way less CPU intensive. Cheers Nick
>>>>> We would probably get a bigger benefit from taking an
>>>>> approach like SquashFS has recently added, that is,
>>>>> allowing multi-threaded decompression fro reads, and
>>>>> decompressing directly into the pagecache. Such an approach
>>>>> would likely make zlib compression much more scalable on
>>>>> large systems.
>>>>> 
>>>>> 
>>>> 
>>>> Austin, That seems better then my idea as you seem to be more
>>>> up to date on brtfs devolopment. If you and the other
>>>> developers of brtfs are interested in adding this as a
>>>> feature please let me known as I would like to help improve
>>>> brtfs as the file system as an idea is great just seems like
>>>> it needs a lot of work :). Nick
>>> I wouldn't say that I am a BTRFS developer (power user maybe?),
>>> but I would definitely say that parallelizing compression on
>>> writes would be a good idea too (especially for things like
>>> lz4, which IIRC is either in 3.16 or in the queue for 3.17).
>>> Both options would be a lot of work, but almost any performance
>>> optimization would.  I would almost say that it would provide a
>>> bigger performance improvement to get BTRFS to intelligently
>>> stripe reads and writes (at the moment, any given worker thread
>>> only dispatches one write or read to a single device at a
>>> time, and any given write() or read() syscall gets handled by
>>> only one worker).
>>> 
>> 
>> I will look into this idea and see if I can do this for writes. 
>> Regards Nick
> 
> Austin, Seems since we don't want to release the cache for inodes
> in order to improve writes if are going to use the page cache. We
> seem to be doing this for writes in end_compressed_bio_write for
> standard pages and in end_compressed_bio_write. If we want to cache
> write pages why are we removing then ? Seems like this needs to be
> removed in order to start off. Regards Nick
> 
I'm not entirely sure, it's been a while since I went exploring in the
page-cache code.  My guess is that there is some reason that you and I
aren't seeing that we are trying for write-around semantics, maybe one
of the people who originally wrote this code could weigh in?  Part of
this might be to do with the fact that normal page-cache semantics
don't always work as expected with COW filesystems (cause a write goes
to a different block on the device than a read before the write would
have gone to).  It might be easier to parallelize reads first, and
then work from that (and most workloads would probably benefit more
from the parallelized reads).


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Mon, Jul 28, 2014 at 12:19 PM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2014-07-28 11:57, Nick Krause wrote:
>> On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com>
>> wrote:
>>> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>>>>> <ahferroin7@gmail.com> wrote:
>>>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>>>> This may be a bad idea , but compression in brtfs seems
>>>>>>> to be only using one core to compress. Depending on the
>>>>>>> CPU used and the amount of cores in the CPU we can make
>>>>>>> this much faster with multiple cores. This seems bad by
>>>>>>> my reading at least I would recommend for writing
>>>>>>> compression we write a function to use a certain amount
>>>>>>> of cores based on the load of the system's CPU not using
>>>>>>> more then 75% of the system's CPU resources as my system
>>>>>>> when idle has never needed more then one core of my i5
>>>>>>> 2500k to run when with interrupts for opening eclipse are
>>>>>>> running. For reading compression on good core seems fine
>>>>>>> to me as testing other compression software for reads ,
>>>>>>> it's way less CPU intensive. Cheers Nick
>>>>>> We would probably get a bigger benefit from taking an
>>>>>> approach like SquashFS has recently added, that is,
>>>>>> allowing multi-threaded decompression fro reads, and
>>>>>> decompressing directly into the pagecache. Such an approach
>>>>>> would likely make zlib compression much more scalable on
>>>>>> large systems.
>>>>>>
>>>>>>
>>>>>
>>>>> Austin, That seems better then my idea as you seem to be more
>>>>> up to date on brtfs devolopment. If you and the other
>>>>> developers of brtfs are interested in adding this as a
>>>>> feature please let me known as I would like to help improve
>>>>> brtfs as the file system as an idea is great just seems like
>>>>> it needs a lot of work :). Nick
>>>> I wouldn't say that I am a BTRFS developer (power user maybe?),
>>>> but I would definitely say that parallelizing compression on
>>>> writes would be a good idea too (especially for things like
>>>> lz4, which IIRC is either in 3.16 or in the queue for 3.17).
>>>> Both options would be a lot of work, but almost any performance
>>>> optimization would.  I would almost say that it would provide a
>>>> bigger performance improvement to get BTRFS to intelligently
>>>> stripe reads and writes (at the moment, any given worker thread
>>>> only dispatches one write or read to a single device at a
>>>> time, and any given write() or read() syscall gets handled by
>>>> only one worker).
>>>>
>>>
>>> I will look into this idea and see if I can do this for writes.
>>> Regards Nick
>>
>> Austin, Seems since we don't want to release the cache for inodes
>> in order to improve writes if are going to use the page cache. We
>> seem to be doing this for writes in end_compressed_bio_write for
>> standard pages and in end_compressed_bio_write. If we want to cache
>> write pages why are we removing then ? Seems like this needs to be
>> removed in order to start off. Regards Nick
>>
> I'm not entirely sure, it's been a while since I went exploring in the
> page-cache code.  My guess is that there is some reason that you and I
> aren't seeing that we are trying for write-around semantics, maybe one
> of the people who originally wrote this code could weigh in?  Part of
> this might be to do with the fact that normal page-cache semantics
> don't always work as expected with COW filesystems (cause a write goes
> to a different block on the device than a read before the write would
> have gone to).  It might be easier to parallelize reads first, and
> then work from that (and most workloads would probably benefit more
> from the parallelized reads).
>
I will look into this later today and work on it then.
Regards Nick

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Mon, Jul 28, 2014 at 2:36 PM, Nick Krause <xerofoify@gmail.com> wrote:
> On Mon, Jul 28, 2014 at 12:19 PM, Austin S Hemmelgarn
> <ahferroin7@gmail.com> wrote:
>> On 2014-07-28 11:57, Nick Krause wrote:
>>> On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com>
>>> wrote:
>>>> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
>>>> <ahferroin7@gmail.com> wrote:
>>>>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>>>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>>>>>> <ahferroin7@gmail.com> wrote:
>>>>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>>>>> This may be a bad idea , but compression in brtfs seems
>>>>>>>> to be only using one core to compress. Depending on the
>>>>>>>> CPU used and the amount of cores in the CPU we can make
>>>>>>>> this much faster with multiple cores. This seems bad by
>>>>>>>> my reading at least I would recommend for writing
>>>>>>>> compression we write a function to use a certain amount
>>>>>>>> of cores based on the load of the system's CPU not using
>>>>>>>> more then 75% of the system's CPU resources as my system
>>>>>>>> when idle has never needed more then one core of my i5
>>>>>>>> 2500k to run when with interrupts for opening eclipse are
>>>>>>>> running. For reading compression on good core seems fine
>>>>>>>> to me as testing other compression software for reads ,
>>>>>>>> it's way less CPU intensive. Cheers Nick
>>>>>>> We would probably get a bigger benefit from taking an
>>>>>>> approach like SquashFS has recently added, that is,
>>>>>>> allowing multi-threaded decompression fro reads, and
>>>>>>> decompressing directly into the pagecache. Such an approach
>>>>>>> would likely make zlib compression much more scalable on
>>>>>>> large systems.
>>>>>>>
>>>>>>>
>>>>>>
>>>>>> Austin, That seems better then my idea as you seem to be more
>>>>>> up to date on brtfs devolopment. If you and the other
>>>>>> developers of brtfs are interested in adding this as a
>>>>>> feature please let me known as I would like to help improve
>>>>>> brtfs as the file system as an idea is great just seems like
>>>>>> it needs a lot of work :). Nick
>>>>> I wouldn't say that I am a BTRFS developer (power user maybe?),
>>>>> but I would definitely say that parallelizing compression on
>>>>> writes would be a good idea too (especially for things like
>>>>> lz4, which IIRC is either in 3.16 or in the queue for 3.17).
>>>>> Both options would be a lot of work, but almost any performance
>>>>> optimization would.  I would almost say that it would provide a
>>>>> bigger performance improvement to get BTRFS to intelligently
>>>>> stripe reads and writes (at the moment, any given worker thread
>>>>> only dispatches one write or read to a single device at a
>>>>> time, and any given write() or read() syscall gets handled by
>>>>> only one worker).
>>>>>
>>>>
>>>> I will look into this idea and see if I can do this for writes.
>>>> Regards Nick
>>>
>>> Austin, Seems since we don't want to release the cache for inodes
>>> in order to improve writes if are going to use the page cache. We
>>> seem to be doing this for writes in end_compressed_bio_write for
>>> standard pages and in end_compressed_bio_write. If we want to cache
>>> write pages why are we removing then ? Seems like this needs to be
>>> removed in order to start off. Regards Nick
>>>
>> I'm not entirely sure, it's been a while since I went exploring in the
>> page-cache code.  My guess is that there is some reason that you and I
>> aren't seeing that we are trying for write-around semantics, maybe one
>> of the people who originally wrote this code could weigh in?  Part of
>> this might be to do with the fact that normal page-cache semantics
>> don't always work as expected with COW filesystems (cause a write goes
>> to a different block on the device than a read before the write would
>> have gone to).  It might be easier to parallelize reads first, and
>> then work from that (and most workloads would probably benefit more
>> from the parallelized reads).
>>
> I will look into this later today and work on it then.
> Regards Nick

Seems the best way to do is to create a kernel thread per core like in NFS and
depending on the load of the system use these threads.
Regards Nick

Re: Multi Core Support for compression in compression.c

[Austin S Hemmelgarn]

[-- Attachment #1: Type: text/plain, Size: 4739 bytes --]

On 2014-07-29 13:08, Nick Krause wrote:
> On Mon, Jul 28, 2014 at 2:36 PM, Nick Krause <xerofoify@gmail.com> wrote:
>> On Mon, Jul 28, 2014 at 12:19 PM, Austin S Hemmelgarn
>> <ahferroin7@gmail.com> wrote:
>>> On 2014-07-28 11:57, Nick Krause wrote:
>>>> On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com>
>>>> wrote:
>>>>> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
>>>>> <ahferroin7@gmail.com> wrote:
>>>>>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>>>>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>>>>>>> <ahferroin7@gmail.com> wrote:
>>>>>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>>>>>> This may be a bad idea , but compression in brtfs seems
>>>>>>>>> to be only using one core to compress. Depending on the
>>>>>>>>> CPU used and the amount of cores in the CPU we can make
>>>>>>>>> this much faster with multiple cores. This seems bad by
>>>>>>>>> my reading at least I would recommend for writing
>>>>>>>>> compression we write a function to use a certain amount
>>>>>>>>> of cores based on the load of the system's CPU not using
>>>>>>>>> more then 75% of the system's CPU resources as my system
>>>>>>>>> when idle has never needed more then one core of my i5
>>>>>>>>> 2500k to run when with interrupts for opening eclipse are
>>>>>>>>> running. For reading compression on good core seems fine
>>>>>>>>> to me as testing other compression software for reads ,
>>>>>>>>> it's way less CPU intensive. Cheers Nick
>>>>>>>> We would probably get a bigger benefit from taking an
>>>>>>>> approach like SquashFS has recently added, that is,
>>>>>>>> allowing multi-threaded decompression fro reads, and
>>>>>>>> decompressing directly into the pagecache. Such an approach
>>>>>>>> would likely make zlib compression much more scalable on
>>>>>>>> large systems.
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>> Austin, That seems better then my idea as you seem to be more
>>>>>>> up to date on brtfs devolopment. If you and the other
>>>>>>> developers of brtfs are interested in adding this as a
>>>>>>> feature please let me known as I would like to help improve
>>>>>>> brtfs as the file system as an idea is great just seems like
>>>>>>> it needs a lot of work :). Nick
>>>>>> I wouldn't say that I am a BTRFS developer (power user maybe?),
>>>>>> but I would definitely say that parallelizing compression on
>>>>>> writes would be a good idea too (especially for things like
>>>>>> lz4, which IIRC is either in 3.16 or in the queue for 3.17).
>>>>>> Both options would be a lot of work, but almost any performance
>>>>>> optimization would.  I would almost say that it would provide a
>>>>>> bigger performance improvement to get BTRFS to intelligently
>>>>>> stripe reads and writes (at the moment, any given worker thread
>>>>>> only dispatches one write or read to a single device at a
>>>>>> time, and any given write() or read() syscall gets handled by
>>>>>> only one worker).
>>>>>>
>>>>>
>>>>> I will look into this idea and see if I can do this for writes.
>>>>> Regards Nick
>>>>
>>>> Austin, Seems since we don't want to release the cache for inodes
>>>> in order to improve writes if are going to use the page cache. We
>>>> seem to be doing this for writes in end_compressed_bio_write for
>>>> standard pages and in end_compressed_bio_write. If we want to cache
>>>> write pages why are we removing then ? Seems like this needs to be
>>>> removed in order to start off. Regards Nick
>>>>
>>> I'm not entirely sure, it's been a while since I went exploring in the
>>> page-cache code.  My guess is that there is some reason that you and I
>>> aren't seeing that we are trying for write-around semantics, maybe one
>>> of the people who originally wrote this code could weigh in?  Part of
>>> this might be to do with the fact that normal page-cache semantics
>>> don't always work as expected with COW filesystems (cause a write goes
>>> to a different block on the device than a read before the write would
>>> have gone to).  It might be easier to parallelize reads first, and
>>> then work from that (and most workloads would probably benefit more
>>> from the parallelized reads).
>>>
>> I will look into this later today and work on it then.
>> Regards Nick
> 
> Seems the best way to do is to create a kernel thread per core like in NFS and
> depending on the load of the system use these threads.
> Regards Nick
> 
It might be more work now, but it would probably be better in the long
run to do it using kernel workqueues, as they would provide better
support for suspend/hibernate/resume, and then you wouldn't need to
worry about scheduling or how many CPU cores are in the system.


[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/pkcs7-signature, Size: 2967 bytes --]

Re: Multi Core Support for compression in compression.c

[Nick Krause]

On Tue, Jul 29, 2014 at 1:14 PM, Austin S Hemmelgarn
<ahferroin7@gmail.com> wrote:
> On 2014-07-29 13:08, Nick Krause wrote:
>> On Mon, Jul 28, 2014 at 2:36 PM, Nick Krause <xerofoify@gmail.com> wrote:
>>> On Mon, Jul 28, 2014 at 12:19 PM, Austin S Hemmelgarn
>>> <ahferroin7@gmail.com> wrote:
>>>> On 2014-07-28 11:57, Nick Krause wrote:
>>>>> On Mon, Jul 28, 2014 at 11:13 AM, Nick Krause <xerofoify@gmail.com>
>>>>> wrote:
>>>>>> On Mon, Jul 28, 2014 at 6:10 AM, Austin S Hemmelgarn
>>>>>> <ahferroin7@gmail.com> wrote:
>>>>>>> On 07/27/2014 11:21 PM, Nick Krause wrote:
>>>>>>>> On Sun, Jul 27, 2014 at 10:56 PM, Austin S Hemmelgarn
>>>>>>>> <ahferroin7@gmail.com> wrote:
>>>>>>>>> On 07/27/2014 04:47 PM, Nick Krause wrote:
>>>>>>>>>> This may be a bad idea , but compression in brtfs seems
>>>>>>>>>> to be only using one core to compress. Depending on the
>>>>>>>>>> CPU used and the amount of cores in the CPU we can make
>>>>>>>>>> this much faster with multiple cores. This seems bad by
>>>>>>>>>> my reading at least I would recommend for writing
>>>>>>>>>> compression we write a function to use a certain amount
>>>>>>>>>> of cores based on the load of the system's CPU not using
>>>>>>>>>> more then 75% of the system's CPU resources as my system
>>>>>>>>>> when idle has never needed more then one core of my i5
>>>>>>>>>> 2500k to run when with interrupts for opening eclipse are
>>>>>>>>>> running. For reading compression on good core seems fine
>>>>>>>>>> to me as testing other compression software for reads ,
>>>>>>>>>> it's way less CPU intensive. Cheers Nick
>>>>>>>>> We would probably get a bigger benefit from taking an
>>>>>>>>> approach like SquashFS has recently added, that is,
>>>>>>>>> allowing multi-threaded decompression fro reads, and
>>>>>>>>> decompressing directly into the pagecache. Such an approach
>>>>>>>>> would likely make zlib compression much more scalable on
>>>>>>>>> large systems.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>> Austin, That seems better then my idea as you seem to be more
>>>>>>>> up to date on brtfs devolopment. If you and the other
>>>>>>>> developers of brtfs are interested in adding this as a
>>>>>>>> feature please let me known as I would like to help improve
>>>>>>>> brtfs as the file system as an idea is great just seems like
>>>>>>>> it needs a lot of work :). Nick
>>>>>>> I wouldn't say that I am a BTRFS developer (power user maybe?),
>>>>>>> but I would definitely say that parallelizing compression on
>>>>>>> writes would be a good idea too (especially for things like
>>>>>>> lz4, which IIRC is either in 3.16 or in the queue for 3.17).
>>>>>>> Both options would be a lot of work, but almost any performance
>>>>>>> optimization would.  I would almost say that it would provide a
>>>>>>> bigger performance improvement to get BTRFS to intelligently
>>>>>>> stripe reads and writes (at the moment, any given worker thread
>>>>>>> only dispatches one write or read to a single device at a
>>>>>>> time, and any given write() or read() syscall gets handled by
>>>>>>> only one worker).
>>>>>>>
>>>>>>
>>>>>> I will look into this idea and see if I can do this for writes.
>>>>>> Regards Nick
>>>>>
>>>>> Austin, Seems since we don't want to release the cache for inodes
>>>>> in order to improve writes if are going to use the page cache. We
>>>>> seem to be doing this for writes in end_compressed_bio_write for
>>>>> standard pages and in end_compressed_bio_write. If we want to cache
>>>>> write pages why are we removing then ? Seems like this needs to be
>>>>> removed in order to start off. Regards Nick
>>>>>
>>>> I'm not entirely sure, it's been a while since I went exploring in the
>>>> page-cache code.  My guess is that there is some reason that you and I
>>>> aren't seeing that we are trying for write-around semantics, maybe one
>>>> of the people who originally wrote this code could weigh in?  Part of
>>>> this might be to do with the fact that normal page-cache semantics
>>>> don't always work as expected with COW filesystems (cause a write goes
>>>> to a different block on the device than a read before the write would
>>>> have gone to).  It might be easier to parallelize reads first, and
>>>> then work from that (and most workloads would probably benefit more
>>>> from the parallelized reads).
>>>>
>>> I will look into this later today and work on it then.
>>> Regards Nick
>>
>> Seems the best way to do is to create a kernel thread per core like in NFS and
>> depending on the load of the system use these threads.
>> Regards Nick
>>
> It might be more work now, but it would probably be better in the long
> run to do it using kernel workqueues, as they would provide better
> support for suspend/hibernate/resume, and then you wouldn't need to
> worry about scheduling or how many CPU cores are in the system.
>

Seems better then my ideas , I will need to work on this later as for now I have
some reading on the Linux networking stack.
Regards Nick