Re: [PATCH v2] btrfs: add mount option to set commit interval

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Mike Audia]

> ----- Original Message -----
> From: Mike Audia
> Sent: 08/02/13 05:12 PM
> To: David Sterba, linux-btrfs@vger.kernel.org
> Subject: Re: [PATCH v2] btrfs: add mount option to set commit interval
>
> > v2:
> > * Stefan pointed out a missing break and that /proc/mounts does not show
> >  the option.
> > * fixed missing goto and extra printk parameter after failed match_int

Another newbie question is which version of the kernel do I need to have in order to cleanly apply this patch?  I am finding that it fails to apply to the current stable kernel code (as of now it is v3.10.4) which makes me think your patch has to be applied to a newer one?  Are you patching against the linux git tree meaning I have to use the 3.11 series to try your code?

Is this the correct git repo: git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Here is the error in patching observed patching against 3.10.4 as described:

patching file fs/btrfs/ctree.h
Hunk #1 succeeded at 1338 (offset -12 lines).
Hunk #2 succeeded at 1929 (offset -40 lines).
patching file fs/btrfs/disk-io.c
Hunk #1 succeeded at 1713 (offset -24 lines).
Hunk #2 FAILED at 1749.
Hunk #3 succeeded at 2172 with fuzz 1 (offset -17 lines).
1 out of 3 hunks FAILED -- saving rejects to file fs/btrfs/disk-io.c.rej
patching file fs/btrfs/super.c
Hunk #1 succeeded at 319 (offset -2 lines).
Hunk #2 succeeded at 360 (offset -2 lines).
Hunk #3 succeeded at 626 (offset -2 lines).
Hunk #4 succeeded at 968 (offset 1 line).

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Zach Brown]

> Another newbie question is which version of the kernel do I need to
> have in order to cleanly apply this patch?  I am finding that it fails
> to apply to the current stable kernel code (as of now it is v3.10.4)
> which makes me think your patch has to be applied to a newer one?  Are
> you patching against the linux git tree meaning I have to use the 3.11
> series to try your code?

Try Josef's btrfs-next repo:

  https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories#Integration_repository_.28btrfs-next.29

- z

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Mike Audia]

> > Another newbie question is which version of the kernel do I need to
> > have in order to cleanly apply this patch?  I am finding that it fails
> > to apply to the current stable kernel code (as of now it is v3.10.4)
> > which makes me think your patch has to be applied to a newer one?  Are
> > you patching against the linux git tree meaning I have to use the 3.11
> > series to try your code?
> 
> Try Josef's btrfs-next repo:
> 
>  https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories#Integration_repository_.28btrfs-next.29

OK!  I can patch successfully into that git repo:

% cd /tmp/work
% git clone git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
% cd btrfs
% patch -Np1 -i btrfs_add_mount_option_to_set_commit_interval.patch
patching file fs/btrfs/ctree.h
patching file fs/btrfs/disk-io.c
patching file fs/btrfs/super.c
Hunk #3 succeeded at 647 (offset 19 lines).
Hunk #4 succeeded at 1006 with fuzz 1 (offset 39 lines).

If I am not mistaken, btrfs-next is the entire kernel's code?  The wiki suggests running anything compiled therein from the build dir.  If I want to compile this into the official 3.10.4 tree, how can I do it?

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 2041 bytes --]

On Sat, Aug 03, 2013 at 07:39:01AM -0400, Mike Audia wrote:
> > > Another newbie question is which version of the kernel do I need to
> > > have in order to cleanly apply this patch?  I am finding that it fails
> > > to apply to the current stable kernel code (as of now it is v3.10.4)
> > > which makes me think your patch has to be applied to a newer one?  Are
> > > you patching against the linux git tree meaning I have to use the 3.11
> > > series to try your code?
> > 
> > Try Josef's btrfs-next repo:
> > 
> >  https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories#Integration_repository_.28btrfs-next.29
> 
> OK!  I can patch successfully into that git repo:
> 
> % cd /tmp/work
> % git clone git://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
> % cd btrfs
> % patch -Np1 -i btrfs_add_mount_option_to_set_commit_interval.patch
> patching file fs/btrfs/ctree.h
> patching file fs/btrfs/disk-io.c
> patching file fs/btrfs/super.c
> Hunk #3 succeeded at 647 (offset 19 lines).
> Hunk #4 succeeded at 1006 with fuzz 1 (offset 39 lines).

> If I am not mistaken, btrfs-next is the entire kernel's code?  The
> wiki suggests running anything compiled therein from the build dir.

   That'll be for the userspace tools, not the kernel. Obviously, one
doesn't tend to run kernels from the command line. :)

>  If I want to compile this into the official 3.10.4 tree, how can I
> do it?

   Add the "official" kernel repo as a remote to the same git repo
(with git remote add), fetch that repo, create a new branch to work
in, based on the btrfs-next branch, then merge in the other branch (or
vice-versa).

   Note that btrfs-next is usually based on the latest released kernel
anyway, so that's likely to be largely superfluous.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 65E74AC0 from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
     --- A gentleman doesn't do damage unless he's paid for it. ---      

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Mike Audia]

>  Add the "official" kernel repo as a remote to the same git repo
> (with git remote add), fetch that repo, create a new branch to work
> in, based on the btrfs-next branch, then merge in the other branch (or
> vice-versa).
> 
>  Note that btrfs-next is usually based on the latest released kernel
> anyway, so that's likely to be largely superfluous.

Thank you for the reply.  I am afraid what you've outlined is beyond my abilities.  I guess the patch is built against something newer than 3.10.5 which was just released yesterday since I get the similar rejection when patching into it:

patching file fs/btrfs/ctree.h
Hunk #1 succeeded at 1338 (offset -12 lines).
Hunk #2 succeeded at 1929 (offset -40 lines).
patching file fs/btrfs/disk-io.c
Hunk #1 succeeded at 1713 (offset -24 lines).
Hunk #2 FAILED at 1749.
Hunk #3 succeeded at 2172 with fuzz 1 (offset -17 lines).
1 out of 3 hunks FAILED -- saving rejects to file fs/btrfs/disk-io.c.rej
patching file fs/btrfs/super.c
Hunk #1 succeeded at 319 (offset -2 lines).
Hunk #2 succeeded at 360 (offset -2 lines).
Hunk #3 succeeded at 626 (offset -2 lines).
Hunk #4 succeeded at 968 (offset 1 line).

Is is safe for me to manually fix that particular hunk?

Re: [PATCH v2] btrfs: add mount option to set commit interval

[Mike Audia]

> v2:
> * Stefan pointed out a missing break and that /proc/mounts does not show
>  the option.
> * fixed missing goto and extra printk parameter after failed match_int

Forgive my naive question: I am using gmx's webclient which doesn't seem to identify your patch as an attachment.  I can simply copy/paste into vim and apply the patch.  Is there are more elegant way to obtain the code, a git tree perhaps or ...?

Thank you.

Re: [PATCH v2] btrfs: add mount option to set commit interval

[David Sterba]

On Fri, Aug 02, 2013 at 05:12:49PM -0400, Mike Audia wrote:
> > v2:
> > * Stefan pointed out a missing break and that /proc/mounts does not show
> >  the option.
> > * fixed missing goto and extra printk parameter after failed match_int
> 
> Forgive my naive question: I am using gmx's webclient which doesn't
> seem to identify your patch as an attachment.  I can simply copy/paste
> into vim and apply the patch.  Is there are more elegant way to obtain
> the code, a git tree perhaps or ...?

The standard way in linux kernel community is to send patches in emails
inline. If you can't apply the patches from the mail, a git-friendly way
of applying patches is the patchwork.kernel.org interface, see

https://btrfs.wiki.kernel.org/index.php/Btrfs_source_repositories#Patches_sent_to_mailinglist

how to use it.

david

[PATCH] btrfs: add mount option to set commit interval

[David Sterba]

I'ts hardcoded to 30 seconds which is fine for most users. Higher values
defer data being synced to permanent storage with obvious consequences
when the system crashes. The upper bound is not forced, but a warning is
printed if it's more than 300 seconds (5 minutes).

Signed-off-by: David Sterba <dsterba@suse.cz>
---
 fs/btrfs/ctree.h   |  3 +++
 fs/btrfs/disk-io.c |  6 ++++--
 fs/btrfs/super.c   | 23 +++++++++++++++++++++++
 3 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index d6dd49b..6f7b607 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1338,6 +1338,7 @@ struct btrfs_fs_info {
 	u64 last_trans_log_full_commit;
 	unsigned long mount_opt;
 	unsigned long compress_type:4;
+	int commit_interval;
 	/*
 	 * It is a suggestive number, the read side is safe even it gets a
 	 * wrong number because we will write out the data into a regular
@@ -1928,6 +1929,8 @@ struct btrfs_ioctl_defrag_range_args {
 #define BTRFS_MOUNT_CHECK_INTEGRITY_INCLUDING_EXTENT_DATA (1 << 21)
 #define BTRFS_MOUNT_PANIC_ON_FATAL_ERROR	(1 << 22)
 
+#define BTRFS_DEFAULT_COMMIT_INTERVAL	(30)
+
 #define btrfs_clear_opt(o, opt)		((o) &= ~BTRFS_MOUNT_##opt)
 #define btrfs_set_opt(o, opt)		((o) |= BTRFS_MOUNT_##opt)
 #define btrfs_raw_test_opt(o, opt)	((o) & BTRFS_MOUNT_##opt)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index b0292b3..d366043 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1714,7 +1714,7 @@ static int transaction_kthread(void *arg)
 
 	do {
 		cannot_commit = false;
-		delay = HZ * 30;
+		delay = HZ * root->fs_info->commit_interval;
 		mutex_lock(&root->fs_info->transaction_kthread_mutex);
 
 		spin_lock(&root->fs_info->trans_lock);
@@ -1726,7 +1726,8 @@ static int transaction_kthread(void *arg)
 
 		now = get_seconds();
 		if (!cur->blocked &&
-		    (now < cur->start_time || now - cur->start_time < 30)) {
+		    (now < cur->start_time ||
+		     now - cur->start_time < root->fs_info->commit_interval)) {
 			spin_unlock(&root->fs_info->trans_lock);
 			delay = HZ * 5;
 			goto sleep;
@@ -2173,6 +2174,7 @@ int open_ctree(struct super_block *sb,
 	fs_info->trans_no_join = 0;
 	fs_info->free_chunk_space = 0;
 	fs_info->tree_mod_log = RB_ROOT;
+	fs_info->commit_interval = BTRFS_DEFAULT_COMMIT_INTERVAL;
 
 	/* readahead state */
 	INIT_RADIX_TREE(&fs_info->reada_tree, GFP_NOFS & ~__GFP_WAIT);
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index f0857e0..4cc950f 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -319,6 +319,7 @@ enum {
 	Opt_no_space_cache, Opt_recovery, Opt_skip_balance,
 	Opt_check_integrity, Opt_check_integrity_including_extent_data,
 	Opt_check_integrity_print_mask, Opt_fatal_errors,
+	Opt_commit_interval,
 	Opt_err,
 };
 
@@ -359,6 +360,7 @@ static match_table_t tokens = {
 	{Opt_check_integrity_including_extent_data, "check_int_data"},
 	{Opt_check_integrity_print_mask, "check_int_print_mask=%d"},
 	{Opt_fatal_errors, "fatal_errors=%s"},
+	{Opt_commit_interval, "commit=%d"},
 	{Opt_err, NULL},
 };
 
@@ -624,6 +626,27 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 				goto out;
 			}
 			break;
+		case Opt_commit_interval:
+			intarg = 0;
+			ret = match_int(&args[0], &intarg);
+			if (ret < 0) {
+				printk(KERN_ERR
+					"btrfs: invalid commit interval\n", ret);
+				ret = -EINVAL;
+			}
+			if (intarg > 0) {
+				if (intarg > 300) {
+					printk(KERN_WARNING
+					    "btrfs: excessive commit interval %d\n",
+							intarg);
+				}
+				info->commit_interval = intarg;
+			} else {
+				printk(KERN_INFO
+				    "btrfs: using default commit interval %ds\n",
+				    BTRFS_DEFAULT_COMMIT_INTERVAL);
+				info->commit_interval = BTRFS_DEFAULT_COMMIT_INTERVAL;
+			}
 		case Opt_err:
 			printk(KERN_INFO "btrfs: unrecognized mount option "
 			       "'%s'\n", p);
-- 
1.8.1.4

[PATCH v2] btrfs: add mount option to set commit interval

[David Sterba]

I'ts hardcoded to 30 seconds which is fine for most users. Higher values
defer data being synced to permanent storage with obvious consequences
when the system crashes. The upper bound is not forced, but a warning is
printed if it's more than 300 seconds (5 minutes).

Signed-off-by: David Sterba <dsterba@suse.cz>
---

v2:
* Stefan pointed out a missing break and that /proc/mounts does not show
  the option.
* fixed missing goto and extra printk parameter after failed match_int

 fs/btrfs/ctree.h   |  3 +++
 fs/btrfs/disk-io.c |  6 ++++--
 fs/btrfs/super.c   | 27 +++++++++++++++++++++++++++
 3 files changed, 34 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index e795bf1..edeb058 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1350,6 +1350,7 @@ struct btrfs_fs_info {
 	u64 last_trans_log_full_commit;
 	unsigned long mount_opt;
 	unsigned long compress_type:4;
+	int commit_interval;
 	/*
 	 * It is a suggestive number, the read side is safe even it gets a
 	 * wrong number because we will write out the data into a regular
@@ -1968,6 +1969,8 @@ struct btrfs_ioctl_defrag_range_args {
 #define BTRFS_MOUNT_CHECK_INTEGRITY_INCLUDING_EXTENT_DATA (1 << 21)
 #define BTRFS_MOUNT_PANIC_ON_FATAL_ERROR	(1 << 22)
 
+#define BTRFS_DEFAULT_COMMIT_INTERVAL	(30)
+
 #define btrfs_clear_opt(o, opt)		((o) &= ~BTRFS_MOUNT_##opt)
 #define btrfs_set_opt(o, opt)		((o) |= BTRFS_MOUNT_##opt)
 #define btrfs_raw_test_opt(o, opt)	((o) & BTRFS_MOUNT_##opt)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 6b092a1..48390b1 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -1737,7 +1737,7 @@ static int transaction_kthread(void *arg)
 
 	do {
 		cannot_commit = false;
-		delay = HZ * 30;
+		delay = HZ * root->fs_info->commit_interval;
 		mutex_lock(&root->fs_info->transaction_kthread_mutex);
 
 		spin_lock(&root->fs_info->trans_lock);
@@ -1749,7 +1749,8 @@ static int transaction_kthread(void *arg)
 
 		now = get_seconds();
 		if (cur->state < TRANS_STATE_BLOCKED &&
-		    (now < cur->start_time || now - cur->start_time < 30)) {
+		    (now < cur->start_time ||
+		     now - cur->start_time < root->fs_info->commit_interval)) {
 			spin_unlock(&root->fs_info->trans_lock);
 			delay = HZ * 5;
 			goto sleep;
@@ -2189,6 +2190,7 @@ int open_ctree(struct super_block *sb,
 	fs_info->defrag_inodes = RB_ROOT;
 	fs_info->free_chunk_space = 0;
 	fs_info->tree_mod_log = RB_ROOT;
+	fs_info->commit_interval = BTRFS_DEFAULT_COMMIT_INTERVAL;
 
 	/* readahead state */
 	INIT_RADIX_TREE(&fs_info->reada_tree, GFP_NOFS & ~__GFP_WAIT);
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index 8eb6191..cfb916d 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -321,6 +321,7 @@ enum {
 	Opt_no_space_cache, Opt_recovery, Opt_skip_balance,
 	Opt_check_integrity, Opt_check_integrity_including_extent_data,
 	Opt_check_integrity_print_mask, Opt_fatal_errors,
+	Opt_commit_interval,
 	Opt_err,
 };
 
@@ -361,6 +362,7 @@ static match_table_t tokens = {
 	{Opt_check_integrity_including_extent_data, "check_int_data"},
 	{Opt_check_integrity_print_mask, "check_int_print_mask=%d"},
 	{Opt_fatal_errors, "fatal_errors=%s"},
+	{Opt_commit_interval, "commit=%d"},
 	{Opt_err, NULL},
 };
 
@@ -626,6 +628,29 @@ int btrfs_parse_options(struct btrfs_root *root, char *options)
 				goto out;
 			}
 			break;
+		case Opt_commit_interval:
+			intarg = 0;
+			ret = match_int(&args[0], &intarg);
+			if (ret < 0) {
+				printk(KERN_ERR
+					"btrfs: invalid commit interval\n");
+				ret = -EINVAL;
+				goto out;
+			}
+			if (intarg > 0) {
+				if (intarg > 300) {
+					printk(KERN_WARNING
+					    "btrfs: excessive commit interval %d\n",
+							intarg);
+				}
+				info->commit_interval = intarg;
+			} else {
+				printk(KERN_INFO
+				    "btrfs: using default commit interval %ds\n",
+				    BTRFS_DEFAULT_COMMIT_INTERVAL);
+				info->commit_interval = BTRFS_DEFAULT_COMMIT_INTERVAL;
+			}
+			break;
 		case Opt_err:
 			printk(KERN_INFO "btrfs: unrecognized mount option "
 			       "'%s'\n", p);
@@ -942,6 +967,8 @@ static int btrfs_show_options(struct seq_file *seq, struct dentry *dentry)
 		seq_puts(seq, ",skip_balance");
 	if (btrfs_test_opt(root, PANIC_ON_FATAL_ERROR))
 		seq_puts(seq, ",fatal_errors=panic");
+	if (info->commit_interval != BTRFS_DEFAULT_COMMIT_INTERVAL)
+		seq_printf(seq, ",commit=%d", info->commit_interval);
 	return 0;
 }
 
-- 
1.8.1.4