From: Timofey Titovets <nefelim4ag@gmail.com>
To: linux-btrfs@vger.kernel.org
Cc: Timofey Titovets <nefelim4ag@gmail.com>
Subject: [PATCH v8 2/6] Btrfs: heuristic workspace add bucket and sample items, macros
Date: Thu, 28 Sep 2017 17:33:37 +0300 [thread overview]
Message-ID: <20170928143341.24491-3-nefelim4ag@gmail.com> (raw)
In-Reply-To: <20170928143341.24491-1-nefelim4ag@gmail.com>
Added macros:
- For future sampling algo
- For bucket size
Heuristic workspace:
- Add bucket for storing byte type counters
- Add sample array for storing partial copy of
input data range
- Add counter for store current sample size to workspace
Signed-off-by: Timofey Titovets <nefelim4ag@gmail.com>
---
fs/btrfs/compression.c | 57 +++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 56 insertions(+), 1 deletion(-)
diff --git a/fs/btrfs/compression.c b/fs/btrfs/compression.c
index c3624e8e3919..1715655d050e 100644
--- a/fs/btrfs/compression.c
+++ b/fs/btrfs/compression.c
@@ -691,7 +691,50 @@ blk_status_t btrfs_submit_compressed_read(struct inode *inode, struct bio *bio,
}
+/*
+ * Heuristic use systematic sampling to collect data from
+ * input data range, so some constant needed to control algo
+ *
+ * @SAMPLING_READ_SIZE - how many bytes will be copied from on each sample
+ * @SAMPLING_INTERVAL - period that used to iterate over input data range
+ */
+#define SAMPLING_READ_SIZE 16
+#define SAMPLING_INTERVAL 256
+
+/*
+ * For statistic analize of input data range
+ * consider that data consists from bytes
+ * so this is Galois Field with 256 objects
+ * each object have attribute count, i.e. how many times
+ * that object detected in sample
+ */
+#define BUCKET_SIZE 256
+
+/*
+ * The size of the sample is based on a statistical sampling rule of thumb.
+ * That common to perform sampling tests as long as number of elements in
+ * each cell is at least five.
+ *
+ * Instead of five, for now choose 32 value to obtain more accurate results.
+ * If the data contains the maximum number of symbols, which is 256,
+ * lets obtain a sample size bound of 8192.
+ *
+ * So sample at most 8KB of data per data range: 16 consecutive
+ * bytes from up to 512 locations.
+ */
+#define MAX_SAMPLE_SIZE (BTRFS_MAX_UNCOMPRESSED * \
+ SAMPLING_READ_SIZE / SAMPLING_INTERVAL)
+
+
+struct bucket_item {
+ u32 count;
+};
+
struct heuristic_ws {
+ /* Partial copy of input data */
+ u8 *sample;
+ /* Bucket store counter for each byte type */
+ struct bucket_item *bucket;
struct list_head list;
};
@@ -701,6 +744,8 @@ static void free_heuristic_ws(struct list_head *ws)
workspace = list_entry(ws, struct heuristic_ws, list);
+ kvfree(workspace->sample);
+ kfree(workspace->bucket);
kfree(workspace);
}
@@ -711,9 +756,19 @@ static struct list_head *alloc_heuristic_ws(void){
if (!ws)
return ERR_PTR(-ENOMEM);
- INIT_LIST_HEAD(&ws->list);
+ ws->sample = kvmalloc(MAX_SAMPLE_SIZE, GFP_KERNEL);
+ if (!ws->sample)
+ goto fail;
+
+ ws->bucket = kcalloc(BUCKET_SIZE, sizeof(*ws->bucket), GFP_KERNEL);
+ if (!ws->bucket)
+ goto fail;
+ INIT_LIST_HEAD(&ws->list);
return &ws->list;
+fail:
+ free_heuristic_ws(&ws->list);
+ return ERR_PTR(-ENOMEM);
}
struct workspaces_list {
--
2.14.2
next prev parent reply other threads:[~2017-09-28 14:33 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-09-28 14:33 [PATCH v8 0/6] Btrfs: populate heuristic with code Timofey Titovets
2017-09-28 14:33 ` [PATCH v8 1/6] Btrfs: compression.c separated heuristic/compression workspaces Timofey Titovets
2017-09-28 14:33 ` Timofey Titovets [this message]
2017-09-28 14:33 ` [PATCH v8 3/6] Btrfs: implement heuristic sampling logic Timofey Titovets
2017-09-28 14:33 ` [PATCH v8 4/6] Btrfs: heuristic add detection of repeated data patterns Timofey Titovets
2017-09-28 14:33 ` [PATCH v8 5/6] Btrfs: heuristic add byte set calculation Timofey Titovets
2017-09-28 14:33 ` [PATCH v8 6/6] Btrfs: heuristic add byte core " Timofey Titovets
2017-09-29 16:22 ` [PATCH v8 0/6] Btrfs: populate heuristic with code David Sterba
2017-10-19 15:39 ` David Sterba
2017-10-19 22:48 ` Timofey Titovets
2017-10-20 13:45 ` David Sterba
2017-10-22 13:44 ` Timofey Titovets
2017-10-23 18:36 ` Timofey Titovets
2017-10-24 19:23 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170928143341.24491-3-nefelim4ag@gmail.com \
--to=nefelim4ag@gmail.com \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).