Work Queue for btrfs compression writes

Work Queue for btrfs compression writes

[Nick Krause]

Hey Guys ,
I am new to   reading  and writing  kernel code.I got interested in
writing code for btrfs as it seems to
need more work then other file systems and this seems other then
drivers, a good use of time on my part.
I interested in helping improving the compression of btrfs by using  a
set of threads using work queues like XFS
or reads and keeping the page cache after reading compressed blocks as
these seem to be a great way to improve
on  compression performance mostly with large partitions of compressed
data. I am not asking you to write the code
for me but as I am new a little guidance and help would be greatly
appreciated as this seems like too much work for just a newbie.
Thanks A lot,
Nick

Re: Work Queue for btrfs compression writes

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2432 bytes --]

On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
> Hey Guys ,
> I am new to   reading  and writing  kernel code.I got interested in
> writing code for btrfs as it seems to
> need more work then other file systems and this seems other then
> drivers, a good use of time on my part.
> I interested in helping improving the compression of btrfs by using  a
> set of threads using work queues like XFS
> or reads and keeping the page cache after reading compressed blocks as
> these seem to be a great way to improve
> on  compression performance mostly with large partitions of compressed
> data. I am not asking you to write the code
> for me but as I am new a little guidance and help would be greatly
> appreciated as this seems like too much work for just a newbie.

 * Documentation/workqueue.txt (in general, grep in Documentation
   usually throws up something useful)

 * grep -r alloc_workqueue fs/ shows a lot of uses (including in
   btrfs), so it should be fairly easy to see how to create and manage
   a workqueue.

   I suspect that this may be a medium-sized project, rather than a
small one. My gut feeling (based on limited experience) is that the
fallocate extensions project would be considerably simpler.

   I also noticed from the public reply to the private mail (don't do
this without getting permission from the other person) that you posted
to LKML in this thread (don't switch mailing lists mid-thread) that
you anticipated having problems testing with limited disks -- what you
will find is that testing new kernel code is something that you don't
do on your main development OS installation. Instead, you will need
either a scratch machine that you can easily update, or one or more
virtual machines. qemu/kvm is good for this, because it has a mode
that bypasses the BIOS and bootloader emulation, and just directly
runs a kernel from a file on the host machine. This is fast. You can
pass large sparse files to the VM to act as scratch disks, plus keep
another smaller file for the guest OS (and a copy of it so that you
can throw one away and make another one quickly and easily).

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
  --- You've read the project plan.  Forget that. We're going to Do ---  
                      Stuff and Have Fun doing it.                       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 811 bytes --]

Re: Work Queue for btrfs compression writes

[Theodore Ts'o]

On Wed, Jul 30, 2014 at 10:38:21AM +0100, Hugo Mills wrote:
> qemu/kvm is good for this, because it has a mode
> that bypasses the BIOS and bootloader emulation, and just directly
> runs a kernel from a file on the host machine. This is fast. You can
> pass large sparse files to the VM to act as scratch disks, plus keep
> another smaller file for the guest OS (and a copy of it so that you
> can throw one away and make another one quickly and easily).

Nick,

The xfstests-bld/kvm-xfstests git tree I pointed out to you has an
example of a test infrastructure which does this for ext4.  It's been
on my todo list to support other file systems, and I have some plans
for how to do thi, but it's been low on my priority list.  If someone
in the btrfs development community is interested in working with me on
this, they should contact me.

Or the btrfs developers can create their own automated systems which
then gets promulgated, but either way, it's something that I strongly
recommend.  The set of test configs that ext4 developers would run
significantly increased after I cleaned up my test scripts and made it
something that other people could use, and it means that I can ask
people who are implementing new features that they run a complete
regression test run, with many different file systems features enabled
and disabled, so they find the problems before they propose a patch
for merging.

Cheers,

					- Ted

Re: Work Queue for btrfs compression writes

[Peter Hurley]

On 07/30/2014 10:13 AM, Theodore Ts'o wrote:
> On Wed, Jul 30, 2014 at 10:38:21AM +0100, Hugo Mills wrote:
>> qemu/kvm is good for this, because it has a mode
>> that bypasses the BIOS and bootloader emulation, and just directly
>> runs a kernel from a file on the host machine. This is fast. You can
>> pass large sparse files to the VM to act as scratch disks, plus keep
>> another smaller file for the guest OS (and a copy of it so that you
>> can throw one away and make another one quickly and easily).
> 
> Nick,
> 
> The xfstests-bld/kvm-xfstests git tree I pointed out to you has an
> example of a test infrastructure which does this for ext4.  It's been
> on my todo list to support other file systems, and I have some plans
> for how to do thi, but it's been low on my priority list.  If someone
> in the btrfs development community is interested in working with me on
> this, they should contact me.

Ted,

Where is that git tree? I've been planning to set up a unit test and
regression suite for tty/serial, and wouldn't mind cribbing the
infrastructure from someone's existing work.

Regards,
Peter Hurley

Re: Work Queue for btrfs compression writes

[Theodore Ts'o]

On Wed, Jul 30, 2014 at 10:36:57AM -0400, Peter Hurley wrote:
> 
> Where is that git tree? I've been planning to set up a unit test and
> regression suite for tty/serial, and wouldn't mind cribbing the
> infrastructure from someone's existing work.

https://git.kernel.org/cgit/fs/ext2/xfstests-bld.git/

Re: Work Queue for btrfs compression writes

[ashford]

> On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
>> Hey Guys ,
>> I interested in helping improving the compression of btrfs by using  a
>> set of threads using work queues like XFS
>> or reads and keeping the page cache after reading compressed blocks as
>> these seem to be a great way to improve
>> on  compression performance mostly with large partitions of compressed
>> data.
>
>    I suspect that this may be a medium-sized project, rather than a
> small one. My gut feeling (based on limited experience) is that the
> fallocate extensions project would be considerably simpler.
>
>    Hugo.

I may be in error, but I believe this may be a bit more complex than just
routing all reads and writes through a decompression/compression work
queue.  On the write side, it might be better to compress the synchronous
requests in-line, instead of going through a work queue.  Similarly, on
the read side, it might be better to decompress user-requested data (and
metadata) in-line, and have any system-generated read-ahead data be
decompressed in a work queue (the same work queue?).

I believe that one of the significant concerns of the above is that the
compression and decompression routines will have to be verified to be
thread-safe.  If BTRFS is using the same compression/decompression
routines that other file-systems use, then the core code is almost
certainly thread-safe.  Any BTRFS wrapper code will have to be verified.

Peter Ashford

Re: Work Queue for btrfs compression writes

[Nick Krause]

On Wed, Jul 30, 2014 at 11:36 AM,  <ashford@whisperpc.com> wrote:
>> On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
>>> Hey Guys ,
>>> I interested in helping improving the compression of btrfs by using  a
>>> set of threads using work queues like XFS
>>> or reads and keeping the page cache after reading compressed blocks as
>>> these seem to be a great way to improve
>>> on  compression performance mostly with large partitions of compressed
>>> data.
>>
>>    I suspect that this may be a medium-sized project, rather than a
>> small one. My gut feeling (based on limited experience) is that the
>> fallocate extensions project would be considerably simpler.
>>
>>    Hugo.
>
> I may be in error, but I believe this may be a bit more complex than just
> routing all reads and writes through a decompression/compression work
> queue.  On the write side, it might be better to compress the synchronous
> requests in-line, instead of going through a work queue.  Similarly, on
> the read side, it might be better to decompress user-requested data (and
> metadata) in-line, and have any system-generated read-ahead data be
> decompressed in a work queue (the same work queue?).
>
> I believe that one of the significant concerns of the above is that the
> compression and decompression routines will have to be verified to be
> thread-safe.  If BTRFS is using the same compression/decompression
> routines that other file-systems use, then the core code is almost
> certainly thread-safe.  Any BTRFS wrapper code will have to be verified.
>
> Peter Ashford
>


Peter ,
Seems that the code from my reading is that the code is using the same
compression
routines as the other file systems. You seem to have some ideas on how
to write this
or improve it at least, if you want you can send me a list of ideas
that you have.
Cheers Nick

Re: Work Queue for btrfs compression writes

[ashford]

Nick,

> On Wed, Jul 30, 2014 at 11:36 AM,  <ashford@whisperpc.com> wrote:
>>> On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
>>>> Hey Guys ,
>>>> I interested in helping improving the compression of btrfs by using  a
>>>> set of threads using work queues like XFS
>>>> or reads and keeping the page cache after reading compressed blocks as
>>>> these seem to be a great way to improve
>>>> on  compression performance mostly with large partitions of compressed
>>>> data.
>>>
>>>    I suspect that this may be a medium-sized project, rather than a
>>> small one. My gut feeling (based on limited experience) is that the
>>> fallocate extensions project would be considerably simpler.
>>>
>>>    Hugo.
>>
>> I may be in error, but I believe this may be a bit more complex than
>> just
>> routing all reads and writes through a decompression/compression work
>> queue.  On the write side, it might be better to compress the
>> synchronous
>> requests in-line, instead of going through a work queue.  Similarly, on
>> the read side, it might be better to decompress user-requested data (and
>> metadata) in-line, and have any system-generated read-ahead data be
>> decompressed in a work queue (the same work queue?).
>>
>> I believe that one of the significant concerns of the above is that the
>> compression and decompression routines will have to be verified to be
>> thread-safe.  If BTRFS is using the same compression/decompression
>> routines that other file-systems use, then the core code is almost
>> certainly thread-safe.  Any BTRFS wrapper code will have to be verified.
>
> Seems that the code from my reading is that the code is using the same
> compression
> routines as the other file systems. You seem to have some ideas on how
> to write this
> or improve it at least, if you want you can send me a list of ideas
> that you have.

Sorry.  It's been almost 20 years since I did any serious kernel work, and
that kernel (not Linux) didn't even know how to page.  There have been too
many changes since then for me to be able to climb back in with under a
year of hard work.  Even then, I would select a simpler task than this to
start on.

What I DO remember is that the layer you're working on is fairly low in
the stack.  The application, kernel and file-system have already had a
chance to prioritize the I/O requests as synchronous or asynchronous. 
That prioritization must be properly handled, or the overall stability of
BTRFS will be at risk.

Good luck.

Peter Ashford

Re: Work Queue for btrfs compression writes

[Nick Krause]

On Wed, Jul 30, 2014 at 2:31 PM,  <ashford@whisperpc.com> wrote:
> Nick,
>
>> On Wed, Jul 30, 2014 at 11:36 AM,  <ashford@whisperpc.com> wrote:
>>>> On Tue, Jul 29, 2014 at 11:54:20PM -0400, Nick Krause wrote:
>>>>> Hey Guys ,
>>>>> I interested in helping improving the compression of btrfs by using  a
>>>>> set of threads using work queues like XFS
>>>>> or reads and keeping the page cache after reading compressed blocks as
>>>>> these seem to be a great way to improve
>>>>> on  compression performance mostly with large partitions of compressed
>>>>> data.
>>>>
>>>>    I suspect that this may be a medium-sized project, rather than a
>>>> small one. My gut feeling (based on limited experience) is that the
>>>> fallocate extensions project would be considerably simpler.
>>>>
>>>>    Hugo.
>>>
>>> I may be in error, but I believe this may be a bit more complex than
>>> just
>>> routing all reads and writes through a decompression/compression work
>>> queue.  On the write side, it might be better to compress the
>>> synchronous
>>> requests in-line, instead of going through a work queue.  Similarly, on
>>> the read side, it might be better to decompress user-requested data (and
>>> metadata) in-line, and have any system-generated read-ahead data be
>>> decompressed in a work queue (the same work queue?).
>>>
>>> I believe that one of the significant concerns of the above is that the
>>> compression and decompression routines will have to be verified to be
>>> thread-safe.  If BTRFS is using the same compression/decompression
>>> routines that other file-systems use, then the core code is almost
>>> certainly thread-safe.  Any BTRFS wrapper code will have to be verified.
>>
>> Seems that the code from my reading is that the code is using the same
>> compression
>> routines as the other file systems. You seem to have some ideas on how
>> to write this
>> or improve it at least, if you want you can send me a list of ideas
>> that you have.
>
> Sorry.  It's been almost 20 years since I did any serious kernel work, and
> that kernel (not Linux) didn't even know how to page.  There have been too
> many changes since then for me to be able to climb back in with under a
> year of hard work.  Even then, I would select a simpler task than this to
> start on.
>
> What I DO remember is that the layer you're working on is fairly low in
> the stack.  The application, kernel and file-system have already had a
> chance to prioritize the I/O requests as synchronous or asynchronous.
> That prioritization must be properly handled, or the overall stability of
> BTRFS will be at risk.
>
> Good luck.
>
> Peter Ashford
>
Peter,
Seems like a challenge and I like challenges.
Regards Nick

Re: Work Queue for btrfs compression writes

[Chris Mason]

On 07/29/2014 11:54 PM, Nick Krause wrote:
> Hey Guys ,
> I am new to   reading  and writing  kernel code.I got interested in
> writing code for btrfs as it seems to
> need more work then other file systems and this seems other then
> drivers, a good use of time on my part.
> I interested in helping improving the compression of btrfs by using  a
> set of threads using work queues like XFS
> or reads and keeping the page cache after reading compressed blocks as
> these seem to be a great way to improve
> on  compression performance mostly with large partitions of compressed
> data. I am not asking you to write the code
> for me but as I am new a little guidance and help would be greatly
> appreciated as this seems like too much work for just a newbie.

[ Back from vacation ]

Reading through the thread, I don't see anyone mentioning that btrfs
already funnels most compression through helper threads in the kernel
workqueues.

There is also an ordering component that submits the compressed bios to
disk (for writes) in the same order they were created.  This lets us
scatter compression across N cpus, but not introduce seeks if they make
progress at different rates.

-chris