linux-raid.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Ram Pai <linuxram@us.ibm.com>
To: LKML <linux-kernel@vger.kernel.org>,
	linux-raid@vger.kernel.org, dm-devel@redhat.com,
	linux-doc@vger.kernel.org
Cc: shli@kernel.org, snitzer@redhat.com, agk@redhat.com, corbet@lwn.net
Subject: [RFC PATCH 07/16] DM: Optimize memory allocated to hold compressed buffer.
Date: Mon, 15 Aug 2016 10:36:44 -0700	[thread overview]
Message-ID: <1471282613-31006-8-git-send-email-linuxram@us.ibm.com> (raw)
In-Reply-To: <1471282613-31006-1-git-send-email-linuxram@us.ibm.com>

On an average the compressed size is less than 50% of the original buffer.  Use
this knowledge to optimize the amount of space allocated to hold the compressed
buffer. If the allocated size is determined to be insufficient than reallocate
the required size.

Signed-off-by: Ram Pai <linuxram@us.ibm.com>
---
 drivers/md/dm-inplace-compress.c |   39 ++++++++++++++++++++++++++++++++++++++
 drivers/md/dm-inplace-compress.h |   11 ++++++++++
 2 files changed, 50 insertions(+), 0 deletions(-)

diff --git a/drivers/md/dm-inplace-compress.c b/drivers/md/dm-inplace-compress.c
index 5c39169..fe4a4c1 100644
--- a/drivers/md/dm-inplace-compress.c
+++ b/drivers/md/dm-inplace-compress.c
@@ -19,10 +19,12 @@ static struct dm_icomp_compressor_data compressors[] = {
 	[DMCP_COMP_ALG_LZO] = {
 		.name = "lzo",
 		.comp_len = lzo_comp_len,
+		.max_comp_len = lzo_max_comp_len,
 	},
 	[DMCP_COMP_ALG_842] = {
 		.name = "842",
 		.comp_len = nx842_comp_len,
+		.max_comp_len = nx842_max_comp_len,
 	},
 };
 static int default_compressor = -1;
@@ -848,6 +850,14 @@ static inline int dm_icomp_compressor_len(struct dm_icomp_info *info, int len)
 	return len;
 }
 
+static inline int dm_icomp_compressor_maxlen(struct dm_icomp_info *info,
+		int len)
+{
+	if (compressors[info->comp_alg].max_comp_len)
+		return compressors[info->comp_alg].max_comp_len(len);
+	return len;
+}
+
 /*
  * caller should set region.sector, region.count. bi_rw. IO always to/from
  * comp_data
@@ -919,6 +929,25 @@ static void dm_icomp_bio_copy(struct bio *bio, off_t bio_off, void *buf,
 	}
 }
 
+static int dm_icomp_mod_to_max_io_range(struct dm_icomp_info *info,
+			 struct dm_icomp_io_range *io)
+{
+	unsigned int maxlen = dm_icomp_compressor_maxlen(info, io->decomp_len);
+
+	if (maxlen <= io->comp_len)
+		return -ENOSPC;
+	io->io_req.mem.ptr.addr = io->comp_data =
+		dm_icomp_krealloc(io->comp_data, maxlen,
+			io->comp_len, GFP_NOIO);
+	if (!io->comp_data) {
+		DMWARN("UNFORTUNE allocation failure ");
+		io->comp_len = 0;
+		return -ENOSPC;
+	}
+	io->comp_len = maxlen;
+	return 0;
+}
+
 /*
  * return value:
  * < 0 : error
@@ -940,7 +969,17 @@ static int dm_icomp_io_range_compress(struct dm_icomp_info *info,
 	ret = crypto_comp_compress(tfm, decomp_data, decomp_len,
 		io->comp_data, &actual_comp_len);
 
+	if (ret || actual_comp_len > io->comp_len) {
+		ret = dm_icomp_mod_to_max_io_range(info, io);
+		if (!ret) {
+			actual_comp_len = io->comp_len;
+			ret = crypto_comp_compress(tfm, decomp_data, decomp_len,
+				io->comp_data, &actual_comp_len);
+		}
+	}
+
 	put_cpu();
+
 	if (ret < 0)
 		DMWARN("CO Error %d ", ret);
 
diff --git a/drivers/md/dm-inplace-compress.h b/drivers/md/dm-inplace-compress.h
index b61ff0d..86c0ce6 100644
--- a/drivers/md/dm-inplace-compress.h
+++ b/drivers/md/dm-inplace-compress.h
@@ -17,15 +17,26 @@ struct dm_icomp_super_block {
 struct dm_icomp_compressor_data {
 	char *name;
 	int (*comp_len)(int comp_len);
+	int (*max_comp_len)(int comp_len);
 };
 
 static inline int lzo_comp_len(int comp_len)
 {
+	return lzo1x_worst_compress(comp_len) >> 1;
+}
+
+static inline int lzo_max_comp_len(int comp_len)
+{
 	return lzo1x_worst_compress(comp_len);
 }
 
 static inline int nx842_comp_len(int comp_len)
 {
+	return (comp_len>>4)*7; /* less than half: 7/16 */
+}
+
+static inline int nx842_max_comp_len(int comp_len)
+{
 	return comp_len;
 }
 
-- 
1.7.1

  parent reply	other threads:[~2016-08-15 17:36 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-15 17:36 [RFC PATCH 00/16] dm-inplace-compression block device Ram Pai
2016-08-15 17:36 ` [RFC PATCH 01/16] DM: dm-inplace-compress: an inplace compressed DM target Ram Pai
2016-08-15 17:36 ` [RFC PATCH 02/16] DM: Ability to choose the compressor Ram Pai
2016-08-15 17:36 ` [RFC PATCH 03/16] DM: Error if enough space is not available Ram Pai
2016-08-15 17:36 ` [RFC PATCH 04/16] DM: Ensure that the read request is within the device range Ram Pai
2016-08-15 17:36 ` [RFC PATCH 05/16] DM: allocation/free helper routines Ram Pai
2016-08-15 17:36 ` [RFC PATCH 06/16] DM: separate out compression and decompression routines Ram Pai
2016-08-15 17:36 ` Ram Pai [this message]
2016-08-15 17:36 ` [RFC PATCH 08/16] DM: Tag a magicmarker at the end of each compressed segment Ram Pai
2016-08-15 17:36 ` [RFC PATCH 09/16] DM: Delay allocation of decompression buffer during read Ram Pai
2016-08-15 17:36 ` [RFC PATCH 10/16] DM: Try to use the bio buffer for decompression instead of allocating one Ram Pai
2016-08-15 17:36 ` [RFC PATCH 11/16] DM: Try to avoid temporary buffer allocation to hold compressed data Ram Pai
2016-08-15 17:36 ` [RFC PATCH 12/16] DM: release unneeded buffer as soon as possible Ram Pai
2016-08-15 17:36 ` [RFC PATCH 13/16] DM: macros to set and get the state of the request Ram Pai
2016-08-15 17:36 ` [RFC PATCH 14/16] DM: Wasted bio copy Ram Pai
2016-08-15 17:36 ` [RFC PATCH 15/16] DM: Add sysfs parameters to track total memory saved and allocated Ram Pai
2016-08-15 17:36 ` [RFC PATCH 16/16] DM: add documentation for dm-inplace-compress Ram Pai

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1471282613-31006-8-git-send-email-linuxram@us.ibm.com \
    --to=linuxram@us.ibm.com \
    --cc=agk@redhat.com \
    --cc=corbet@lwn.net \
    --cc=dm-devel@redhat.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-raid@vger.kernel.org \
    --cc=shli@kernel.org \
    --cc=snitzer@redhat.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).