Re: [PATCHv9 0/7] add compressing abstraction and multi stream support

[Minchan Kim <minchan@kernel.org>]

Hello Sergey,

Sorry for the late.

Today, I tested this patch and confirm that it's really good.
I send result for the record.

In x86(4core and x2 hyper threading, i7, 2.8GHz), I did parallel 4 dd
test with 200m file like below

dd if=./test200m.file of=mnt/file1 bs=512k count=1024 oflag=direct &
dd if=./test200m.file of=mnt/file2 bs=512k count=1024 oflag=direct &
dd if=./test200m.file of=mnt/file3 bs=512k count=1024 oflag=direct &
dd if=./test200m.file of=mnt/file4 bs=512k count=1024 oflag=direct &
wait all backgroud job

The result is 

When 1 max_comp_streams, elapsed time is 36.26s
When 8 max_comp_streams, elapsed time is 4.09s

In ARM(4core, ARMv7, 1.5GHz), I did iozone test.

      1 max_comp_streams,      8 max_comp_streams
==Initial  write      ==Initial  write
records:   10         records:   10
avg:       141964.05  avg:       239632.54
std:       3532.11    (2.49%)    std:       43863.53  (18.30%)
max:       147900.45  max:       319046.62
min:       135363.73  min:       178929.40
==Rewrite  ==Rewrite
records:   10         records:   10
avg:       144757.46  avg:       247410.80
std:       4019.19    (2.78%)    std:       37378.42  (15.11%)
max:       150757.72  max:       293284.84
min:       135127.55  min:       188984.27
==Read     ==Read
records:   10         records:   10
avg:       208325.22  avg:       202894.24
std:       57072.62   (27.40%)   std:       41099.56  (20.26%)
max:       293428.96  max:       289581.12
min:       79445.37   min:       157478.27
==Re-read  ==Re-read
records:   10         records:   10
avg:       204750.36  avg:       237406.96
std:       36959.99   (18.05%)   std:       41518.36  (17.49%)
max:       268399.89  max:       286898.13
min:       154831.28  min:       160326.88
==Reverse  Read       ==Reverse  Read
records:   10         records:   10
avg:       215043.10  avg:       208946.35
std:       31239.60   (14.53%)   std:       38859.74  (18.60%)
max:       251564.57  max:       284481.31
min:       154719.20  min:       155024.33
==Stride   read       ==Stride   read
records:   10         records:   10
avg:       227246.54  avg:       198925.10
std:       31105.89   (13.69%)   std:       30721.86  (15.44%)
max:       290020.34  max:       227178.70
min:       157399.46  min:       153592.91
==Random   read       ==Random   read
records:   10         records:   10
avg:       238239.81  avg:       216298.41
std:       37276.91   (15.65%)   std:       38194.73  (17.66%)
max:       291416.20  max:       286345.37
min:       152734.23  min:       151871.52
==Mixed    workload   ==Mixed    workload
records:   10         records:   10
avg:       208434.11  avg:       234355.66
std:       31385.40   (15.06%)   std:       22064.02  (9.41%)
max:       253990.11  max:       270412.58
min:       162041.47  min:       186052.12
==Random   write      ==Random   write
records:   10         records:   10
avg:       142172.54  avg:       290231.28
std:       6233.67    (4.38%)    std:       46462.35  (16.01%)
max:       150652.40  max:       338096.54
min:       130584.14  min:       183253.25
==Pwrite   ==Pwrite
records:   10         records:   10
avg:       141247.91  avg:       267085.70
std:       6756.08    (4.78%)    std:       40019.39  (14.98%)
max:       150239.13  max:       335512.33
min:       130456.98  min:       180832.45
==Pread    ==Pread
records:   10         records:   10
avg:       214990.26  avg:       208730.94
std:       40701.79   (18.93%)   std:       50797.78  (24.34%)
max:       287060.54  max:       300675.25
min:       157642.17  min:       156763.98

So, all write test both x86 and ARM is really huge win
and I couldn't find any regression!

Thanks for nice work.

For all patchset,

Acked-by: Minchan Kim <minchan@kernel.org>

On Fri, Feb 28, 2014 at 08:52:00PM +0300, Sergey Senozhatsky wrote:
> This patchset introduces zcomp compression backend abstraction
> adding ability to support compression algorithms other than LZO;
> support for multi compression streams, making parallel compressions
> possible; adds support for LZ4 compression algorithm.
> 
> v8->v9 (reviewed by Andrew Morton):
> -- add LZ4 backend (+iozone test vs LZO)
> -- merge patches 'zram: document max_comp_streams' and 'zram: add multi
>    stream functionality'
> -- do not extern backend struct from source file
> -- use find()/release() naming instead of get()/put()
> -- minor code, commit messages and code comments `nitpicks'
> -- removed Acked-by Minchan Kim from first two patches, because I've
>    changed them a bit.
> 
> v7->v8 (reviewed by Minchan Kim):
> -- merge patches 'add multi stream functionality' and 'enable multi
>    stream compression support in zram'
> -- return status code from set_max_streams knob and print message on
>    error
> -- do not use atomic type for ->avail_strm
> -- return back: allocate by default only one stream for multi stream backend
> -- wake sleeping write in zcomp_strm_multi_put() only if we put stream
>    to idle list
> -- minor code `nitpicks'
> 
> v6->v7 (reviewed by Minchan Kim):
> -- enable multi and single stream support out of the box (drop
>    ZRAM_MULTI_STREAM config option)
> -- add set_max_stream knob, so we can adjust max number of compression
>    streams in runtime (for multi stream backend at the moment)
> -- minor code `nitpicks'
> 
> v5->v6 (reviewed by Minchan Kim):
> -- handle single compression stream case separately, using mutex locking,
>    to address perfomance regression
> -- handle multi compression stream using spin lock and wait_event()/wake_up()
> -- make multi compression stream support configurable (ZRAM_MULTI_STREAM
>    config option)
> 
> v4->v5 (reviewed by Minchan Kim):
> -- renamed zcomp buffer_lock; removed src len and dst len from
>    compress() and decompress(); not using term `buffer' and
>    `workmem' in code and documentation; define compress() and
>    decompress() functions for LZO backend; not using goto's;
>    do not put idle zcomp_strm to idle list tail.
> 
> v3->v4 (reviewed by Minchan Kim):
> -- renamed compression backend and working memory structs as requested
>    by Minchan Kim; fixed several issues noted by Minchan Kim.
> 
> Sergey Senozhatsky (7):
>   zram: introduce compressing backend abstraction
>   zram: use zcomp compressing backends
>   zram: factor out single stream compression
>   zram: add multi stream functionality
>   zram: add set_max_streams knob
>   zram: make compression algorithm selection possible
>   zram: add lz4 algorithm backend
> 
>  Documentation/ABI/testing/sysfs-block-zram |  17 +-
>  Documentation/blockdev/zram.txt            |  45 +++-
>  drivers/block/zram/Kconfig                 |  10 +
>  drivers/block/zram/Makefile                |   4 +-
>  drivers/block/zram/zcomp.c                 | 349 +++++++++++++++++++++++++++++
>  drivers/block/zram/zcomp.h                 |  68 ++++++
>  drivers/block/zram/zcomp_lz4.c             |  47 ++++
>  drivers/block/zram/zcomp_lz4.h             |  17 ++
>  drivers/block/zram/zcomp_lzo.c             |  47 ++++
>  drivers/block/zram/zcomp_lzo.h             |  17 ++
>  drivers/block/zram/zram_drv.c              | 131 ++++++++---
>  drivers/block/zram/zram_drv.h              |  11 +-
>  12 files changed, 715 insertions(+), 48 deletions(-)
>  create mode 100644 drivers/block/zram/zcomp.c
>  create mode 100644 drivers/block/zram/zcomp.h
>  create mode 100644 drivers/block/zram/zcomp_lz4.c
>  create mode 100644 drivers/block/zram/zcomp_lz4.h
>  create mode 100644 drivers/block/zram/zcomp_lzo.c
>  create mode 100644 drivers/block/zram/zcomp_lzo.h
> 
> -- 
> 1.9.0.359.g5e34a15
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

-- 
Kind regards,
Minchan Kim