qemu-devel.nongnu.org archive mirror
 help / color / mirror / Atom feed
From: Max Reitz <mreitz@redhat.com>
To: qemu-block@nongnu.org
Cc: Kevin Wolf <kwolf@redhat.com>,
	Peter Maydell <peter.maydell@linaro.org>,
	qemu-devel@nongnu.org, Max Reitz <mreitz@redhat.com>
Subject: [PULL 30/34] block: introduce compress filter driver
Date: Mon,  6 Jan 2020 15:42:02 +0100	[thread overview]
Message-ID: <20200106144206.698920-31-mreitz@redhat.com> (raw)
In-Reply-To: <20200106144206.698920-1-mreitz@redhat.com>

From: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>

Allow writing all the data compressed through the filter driver.
The written data will be aligned by the cluster size.
Based on the QEMU current implementation, that data can be written to
unallocated clusters only. May be used for a backup job.

Suggested-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-id: 1575288906-551879-2-git-send-email-andrey.shinkevich@virtuozzo.com
[mreitz: Replace NULL bdrv_get_format_name() by "(no format)"]
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/Makefile.objs     |   1 +
 block/filter-compress.c | 168 ++++++++++++++++++++++++++++++++++++++++
 qapi/block-core.json    |  10 ++-
 3 files changed, 175 insertions(+), 4 deletions(-)
 create mode 100644 block/filter-compress.c

diff --git a/block/Makefile.objs b/block/Makefile.objs
index e394fe0b6c..330529b0b7 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -43,6 +43,7 @@ block-obj-y += crypto.o
 
 block-obj-y += aio_task.o
 block-obj-y += backup-top.o
+block-obj-y += filter-compress.o
 
 common-obj-y += stream.o
 
diff --git a/block/filter-compress.c b/block/filter-compress.c
new file mode 100644
index 0000000000..60137fb680
--- /dev/null
+++ b/block/filter-compress.c
@@ -0,0 +1,168 @@
+/*
+ * Compress filter block driver
+ *
+ * Copyright (c) 2019 Virtuozzo International GmbH
+ *
+ * Author:
+ *   Andrey Shinkevich <andrey.shinkevich@virtuozzo.com>
+ *   (based on block/copy-on-read.c by Max Reitz)
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 or
+ * (at your option) any later version of the License.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "block/block_int.h"
+#include "qemu/module.h"
+#include "qapi/error.h"
+
+
+static int compress_open(BlockDriverState *bs, QDict *options, int flags,
+                         Error **errp)
+{
+    bs->file = bdrv_open_child(NULL, options, "file", bs, &child_file, false,
+                               errp);
+    if (!bs->file) {
+        return -EINVAL;
+    }
+
+    if (!bs->file->bs->drv || !block_driver_can_compress(bs->file->bs->drv)) {
+        error_setg(errp,
+                   "Compression is not supported for underlying format: %s",
+                   bdrv_get_format_name(bs->file->bs) ?: "(no format)");
+
+        return -ENOTSUP;
+    }
+
+    bs->supported_write_flags = BDRV_REQ_WRITE_UNCHANGED |
+        (BDRV_REQ_FUA & bs->file->bs->supported_write_flags);
+
+    bs->supported_zero_flags = BDRV_REQ_WRITE_UNCHANGED |
+        ((BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP | BDRV_REQ_NO_FALLBACK) &
+            bs->file->bs->supported_zero_flags);
+
+    return 0;
+}
+
+
+static int64_t compress_getlength(BlockDriverState *bs)
+{
+    return bdrv_getlength(bs->file->bs);
+}
+
+
+static int coroutine_fn compress_co_preadv_part(BlockDriverState *bs,
+                                                uint64_t offset, uint64_t bytes,
+                                                QEMUIOVector *qiov,
+                                                size_t qiov_offset,
+                                                int flags)
+{
+    return bdrv_co_preadv_part(bs->file, offset, bytes, qiov, qiov_offset,
+                               flags);
+}
+
+
+static int coroutine_fn compress_co_pwritev_part(BlockDriverState *bs,
+                                                 uint64_t offset,
+                                                 uint64_t bytes,
+                                                 QEMUIOVector *qiov,
+                                                 size_t qiov_offset, int flags)
+{
+    return bdrv_co_pwritev_part(bs->file, offset, bytes, qiov, qiov_offset,
+                                flags | BDRV_REQ_WRITE_COMPRESSED);
+}
+
+
+static int coroutine_fn compress_co_pwrite_zeroes(BlockDriverState *bs,
+                                                  int64_t offset, int bytes,
+                                                  BdrvRequestFlags flags)
+{
+    return bdrv_co_pwrite_zeroes(bs->file, offset, bytes, flags);
+}
+
+
+static int coroutine_fn compress_co_pdiscard(BlockDriverState *bs,
+                                             int64_t offset, int bytes)
+{
+    return bdrv_co_pdiscard(bs->file, offset, bytes);
+}
+
+
+static void compress_refresh_limits(BlockDriverState *bs, Error **errp)
+{
+    BlockDriverInfo bdi;
+    int ret;
+
+    if (!bs->file) {
+        return;
+    }
+
+    ret = bdrv_get_info(bs->file->bs, &bdi);
+    if (ret < 0 || bdi.cluster_size == 0) {
+        return;
+    }
+
+    bs->bl.request_alignment = bdi.cluster_size;
+}
+
+
+static void compress_eject(BlockDriverState *bs, bool eject_flag)
+{
+    bdrv_eject(bs->file->bs, eject_flag);
+}
+
+
+static void compress_lock_medium(BlockDriverState *bs, bool locked)
+{
+    bdrv_lock_medium(bs->file->bs, locked);
+}
+
+
+static bool compress_recurse_is_first_non_filter(BlockDriverState *bs,
+                                                 BlockDriverState *candidate)
+{
+    return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate);
+}
+
+
+static BlockDriver bdrv_compress = {
+    .format_name                        = "compress",
+
+    .bdrv_open                          = compress_open,
+    .bdrv_child_perm                    = bdrv_filter_default_perms,
+
+    .bdrv_getlength                     = compress_getlength,
+
+    .bdrv_co_preadv_part                = compress_co_preadv_part,
+    .bdrv_co_pwritev_part               = compress_co_pwritev_part,
+    .bdrv_co_pwrite_zeroes              = compress_co_pwrite_zeroes,
+    .bdrv_co_pdiscard                   = compress_co_pdiscard,
+    .bdrv_refresh_limits                = compress_refresh_limits,
+
+    .bdrv_eject                         = compress_eject,
+    .bdrv_lock_medium                   = compress_lock_medium,
+
+    .bdrv_co_block_status               = bdrv_co_block_status_from_file,
+
+    .bdrv_recurse_is_first_non_filter   = compress_recurse_is_first_non_filter,
+
+    .has_variable_length                = true,
+    .is_filter                          = true,
+};
+
+static void bdrv_compress_init(void)
+{
+    bdrv_register(&bdrv_compress);
+}
+
+block_init(bdrv_compress_init);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 839b10b3f0..7ff5e5edaf 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2884,15 +2884,16 @@
 # @copy-on-read: Since 3.0
 # @blklogwrites: Since 3.0
 # @blkreplay: Since 4.2
+# @compress: Since 5.0
 #
 # Since: 2.9
 ##
 { 'enum': 'BlockdevDriver',
   'data': [ 'blkdebug', 'blklogwrites', 'blkreplay', 'blkverify', 'bochs',
-            'cloop', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps', 'gluster',
-            'host_cdrom', 'host_device', 'http', 'https', 'iscsi', 'luks',
-            'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels', 'qcow',
-            'qcow2', 'qed', 'quorum', 'raw', 'rbd',
+            'cloop', 'compress', 'copy-on-read', 'dmg', 'file', 'ftp', 'ftps',
+            'gluster', 'host_cdrom', 'host_device', 'http', 'https', 'iscsi',
+            'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
+            'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
             { 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
             'sheepdog',
             'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat', 'vxhs' ] }
@@ -4060,6 +4061,7 @@
       'blkreplay':  'BlockdevOptionsBlkreplay',
       'bochs':      'BlockdevOptionsGenericFormat',
       'cloop':      'BlockdevOptionsGenericFormat',
+      'compress':   'BlockdevOptionsGenericFormat',
       'copy-on-read':'BlockdevOptionsGenericFormat',
       'dmg':        'BlockdevOptionsGenericFormat',
       'file':       'BlockdevOptionsFile',
-- 
2.24.1



  parent reply	other threads:[~2020-01-06 15:04 UTC|newest]

Thread overview: 36+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-01-06 14:41 [PULL 00/34] Block patches Max Reitz
2020-01-06 14:41 ` [PULL 01/34] block: Add bdrv_qapi_perm_to_blk_perm() Max Reitz
2020-01-06 14:41 ` [PULL 02/34] block: Use bdrv_qapi_perm_to_blk_perm() Max Reitz
2020-01-06 14:41 ` [PULL 03/34] blkdebug: Allow taking/unsharing permissions Max Reitz
2020-01-06 14:41 ` [PULL 04/34] iotests: Add @error to wait_until_completed Max Reitz
2020-01-06 14:41 ` [PULL 05/34] iotests: Add test for failing mirror complete Max Reitz
2020-01-06 14:41 ` [PULL 06/34] throttle-groups: fix memory leak in throttle_group_set_limit: Max Reitz
2020-01-06 14:41 ` [PULL 07/34] qcow2-bitmaps: fix qcow2_can_store_new_dirty_bitmap Max Reitz
2020-01-06 14:41 ` [PULL 08/34] iotests: s/qocw2/qcow2/ Max Reitz
2020-01-06 14:41 ` [PULL 09/34] iotests/qcow2.py: Add dump-header-exts Max Reitz
2020-01-06 14:41 ` [PULL 10/34] iotests/qcow2.py: Split feature fields into bits Max Reitz
2020-01-06 14:41 ` [PULL 11/34] iotests: Add _filter_json_filename Max Reitz
2020-01-06 14:41 ` [PULL 12/34] iotests: Filter refcount_order in 036 Max Reitz
2020-01-06 14:41 ` [PULL 13/34] iotests: Replace IMGOPTS by _unsupported_imgopts Max Reitz
2020-01-06 14:41 ` [PULL 14/34] iotests: Drop compat=1.1 in 050 Max Reitz
2020-01-06 14:41 ` [PULL 15/34] iotests: Let _make_test_img parse its parameters Max Reitz
2020-01-06 14:41 ` [PULL 16/34] iotests: Add -o and --no-opts to _make_test_img Max Reitz
2020-01-06 14:41 ` [PULL 17/34] iotests: Inject space into -ocompat=0.10 in 051 Max Reitz
2020-01-06 14:41 ` [PULL 18/34] iotests: Replace IMGOPTS= by -o Max Reitz
2020-01-06 14:41 ` [PULL 19/34] iotests: Replace IMGOPTS='' by --no-opts Max Reitz
2020-01-06 14:41 ` [PULL 20/34] iotests: Drop IMGOPTS use in 267 Max Reitz
2020-01-06 14:41 ` [PULL 21/34] iotests: Avoid qemu-img create Max Reitz
2020-01-06 14:41 ` [PULL 22/34] iotests: Use _rm_test_img for deleting test images Max Reitz
2020-01-06 14:41 ` [PULL 23/34] iotests: Avoid cp/mv of " Max Reitz
2020-01-06 14:41 ` [PULL 24/34] iotests: Make 091 work with data_file Max Reitz
2020-01-06 14:41 ` [PULL 25/34] iotests: Make 110 " Max Reitz
2020-01-06 14:41 ` [PULL 26/34] iotests: Make 137 " Max Reitz
2020-01-06 14:41 ` [PULL 27/34] iotests: Make 198 " Max Reitz
2020-01-06 14:42 ` [PULL 28/34] iotests: Disable data_file where it cannot be used Max Reitz
2020-01-06 14:42 ` [PULL 29/34] iotests: Allow check -o data_file Max Reitz
2020-01-06 14:42 ` Max Reitz [this message]
2020-01-06 14:42 ` [PULL 31/34] qcow2: Allow writing compressed data of multiple clusters Max Reitz
2020-01-06 14:42 ` [PULL 32/34] tests/qemu-iotests: add case to write " Max Reitz
2020-01-06 14:42 ` [PULL 33/34] tests/qemu-iotests: Update tests to recent desugarized -accel option Max Reitz
2020-01-06 14:42 ` [PULL 34/34] backup-top: Begin drain earlier Max Reitz
2020-01-06 18:22 ` [PULL 00/34] Block patches Peter Maydell

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20200106144206.698920-31-mreitz@redhat.com \
    --to=mreitz@redhat.com \
    --cc=kwolf@redhat.com \
    --cc=peter.maydell@linaro.org \
    --cc=qemu-block@nongnu.org \
    --cc=qemu-devel@nongnu.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).