* [PATCH] Btrfs: release metadata from global reserve if we have to fallback for unlink
From: Josef Bacik @ 2011-11-01 17:52 UTC (permalink / raw)
To: linux-btrfs
I fixed a problem where we weren't reserving space for an orphan item when we
had to fallback to using the global reserve for an unlink, but I introduced
another problem. I was migrating the bytes from the transaction reserve to the
global reserve and then releasing from the global reserve in
btrfs_end_transaction(). The problem with this is that a migrate will jack up
the size for the destination, but leave the size alone for the source, with the
idea that you can do a release normally on the source and it all washes out, and
then you can do a release again on the destination and it works out right. My
way was skipping the release on the trans_block_rsv which still had the jacked
up size from our original reservation. So instead release manually from the
global reserve if this transaction was using it, and then set the
trans->block_rsv back to the trans_block_rsv so that btrfs_end_transaction
cleans everything up properly. With this patch xfstest 83 doesn't emit warnings
about leaking space. Thanks,
Signed-off-by: Josef Bacik <josef@redhat.com>
---
fs/btrfs/inode.c | 5 ++++-
1 files changed, 4 insertions(+), 1 deletions(-)
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index f12747c..3dd2d5d 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -2900,7 +2900,7 @@ out:
if (!err)
err = btrfs_block_rsv_migrate(trans->block_rsv,
&root->fs_info->global_block_rsv,
- btrfs_calc_trans_metadata_size(root, 1));
+ trans->bytes_reserved);
if (err) {
btrfs_end_transaction(trans, root);
@@ -2916,6 +2916,9 @@ static void __unlink_end_trans(struct btrfs_trans_handle *trans,
struct btrfs_root *root)
{
if (trans->block_rsv == &root->fs_info->global_block_rsv) {
+ btrfs_block_rsv_release(root, trans->block_rsv,
+ trans->bytes_reserved);
+ trans->block_rsv = &root->fs_info->trans_block_rsv;
BUG_ON(!root->fs_info->enospc_unlink);
root->fs_info->enospc_unlink = 0;
}
--
1.7.5.2
^ permalink raw reply related
* Re: BUG at fs/btrfs/tree-log.c:3106 (Re: new integration-danger branch pushed out (kernel))
From: David Sterba @ 2011-11-01 16:39 UTC (permalink / raw)
To: liubo2009, Chris Mason, linux-btrfs
In-Reply-To: <20111101103637.GD12759@twin.jikos.cz>
On Tue, Nov 01, 2011 at 11:36:37AM +0100, David Sterba wrote:
> xfstests/214
>
> [ 3142.654774] ------------[ cut here ]------------
> [ 3142.656017] kernel BUG at fs/btrfs/tree-log.c:3106!
> [ 3142.656017] invalid opcode: 0000 [#1] SMP
> [ 3142.656017] CPU 0
> [ 3142.656017] Modules linked in: loop btrfs aoe
> [ 3142.656017]
> [ 3142.656017] Pid: 27328, comm: xfs_io Tainted: G W 3.1.0-default+ #64 Intel Corporation Santa Rosa platform/Matanzas
> [ 3142.656017] RIP: 0010:[<ffffffffa006fe74>] [<ffffffffa006fe74>] btrfs_log_inode+0x554/0x600 [btrfs]
> [ 3142.656017] RSP: 0018:ffff88005c801d58 EFLAGS: 00010282
> [ 3142.656017] RAX: 00000000ffffffef RBX: ffff88003ef39e20 RCX: 0000000000000006
> [ 3142.656017] RDX: 0000000000000001 RSI: ffff8800643aa170 RDI: 0000000000000286
> [ 3142.656017] RBP: ffff88005c801e48 R08: 0000000000000001 R09: 0000000000000001
> [ 3142.656017] R10: 0000000000000001 R11: 0000000000000002 R12: ffff88003405fe70
> [ 3142.656017] R13: ffff88005c801db8 R14: 0000000000000003 R15: ffff8800340604b0
> [ 3142.656017] FS: 00007f7034667700(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
> [ 3142.656017] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3142.656017] CR2: 00000000006b1fb0 CR3: 00000000778df000 CR4: 00000000000006f0
> [ 3142.656017] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 3142.656017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 3142.656017] Process xfs_io (pid: 27328, threadinfo ffff88005c800000, task ffff8800643a9ac0)
> [ 3142.656017] Stack:
> [ 3142.656017] 0000000000000000 0000000000000000 ffff8800340602b0 ffff880072efe000
> [ 3142.656017] 0000000000000000 00000000000004c8 000000005c801dd8 ffff8800340604b0
> [ 3142.656017] ffff88003ef39eb0 ffff88000dfb4420 0000000000000019 0000000000000308
> [ 3142.656017] Call Trace:
> [ 3142.656017] [<ffffffffa0070d28>] btrfs_log_inode_parent+0x198/0x480 [btrfs]
> [ 3142.656017] [<ffffffff8115941c>] ? dget_parent+0x1c/0xd0
> [ 3142.656017] [<ffffffffa0071059>] btrfs_log_dentry_safe+0x49/0x70 [btrfs]
> [ 3142.656017] [<ffffffffa0049406>] btrfs_sync_file+0x146/0x1d0 [btrfs]
> [ 3142.656017] [<ffffffff81143e80>] ? fget_raw+0x210/0x210
> [ 3142.656017] [<ffffffff81171029>] do_fsync+0x59/0x80
> [ 3142.656017] [<ffffffff81171340>] sys_fsync+0x10/0x20
> [ 3142.656017] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
> [ 3142.656017] Code: 60 ff ff ff 45 89 f1 48 8b bd 58 ff ff ff 4c 89 e1 4c 89 fe 89 14 24 48 8b 95 50 ff ff ff e8 44 cf ff ff 85 c0 0f 84 04 fe ff ff <0f> 0b 48 89 df e8 f2 62 fa ff 48 8b bd 50 ff ff ff e8 e6 62 fa
> [ 3142.656017] RIP [<ffffffffa006fe74>] btrfs_log_inode+0x554/0x600 [btrfs]
> [ 3142.656017] RSP <ffff88005c801d58>
> [ 3142.995743] ---[ end trace 81d2bc6c70d7ca63 ]---
for the record, exactly same crash at xfstets/256 (run from 215- on),
with "rm -f /mnt/a1/25365.falloc" stuck (D) at vfs_unlink
david
^ permalink raw reply
* btrfs-progs
From: Andrew Benton @ 2011-11-01 16:34 UTC (permalink / raw)
To: linux-btrfs
Hello,
I've just pulled btrfs-progs from the new git repo
git://git.kernel.org/pub/scm/linux/kernel/git/mason/btrfs-progs.git
However, when I come to make it fails like so:
gcc -Wp,-MMD,./.btrfsctl.o.d,-MT,btrfsctl.o -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -g -Werror -Os -c btrfsctl.c
gcc -g -Werror -Os -o btrfsctl btrfsctl.o ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o root-tree.o dir-item.o file-item.o inode-item.o inode-map.o crc32c.o rbtree.o extent-cache.o extent_io.o volumes.o utils.o btrfs-list.o btrfslabel.o -luuid
ls mkfs.c
mkfs.c
gcc -Wp,-MMD,./.mkfs.o.d,-MT,mkfs.o -Wall -D_FILE_OFFSET_BITS=64 -D_FORTIFY_SOURCE=2 -g -Werror -Os -c mkfs.c
mkfs.c:39:24: fatal error: attr/xattr.h: No such file or directory
compilation terminated.
make: *** [mkfs.o] Error 1
andy@eccles:~/btrfs-progs$
Is attr a required dependency of btrfs-progs now?
If I sed -i 's#attr/#sys/#' mkfs.c then it compiles and installs (I've
not tried to use any of the tools yet). Is it wise to do this? I don't
use any extended attributes, I don't have extended attributes enabled
in my kernel config, I have no wish to install libattr.
Please CC me as I'm not subscribed,
Andy
^ permalink raw reply
* Re: BUG at fs/btrfs/tree-log.c:3106 (Re: new integration-danger branch pushed out (kernel))
From: David Sterba @ 2011-11-01 16:16 UTC (permalink / raw)
To: Chris Mason, liubo2009, linux-btrfs
In-Reply-To: <20111101105650.GB9554@shiny>
On Tue, Nov 01, 2011 at 06:56:50AM -0400, Chris Mason wrote:
> On Tue, Nov 01, 2011 at 11:36:37AM +0100, David Sterba wrote:
> > xfstests/214
> >
> > [ 3142.654774] ------------[ cut here ]------------
> > [ 3142.656017] kernel BUG at fs/btrfs/tree-log.c:3106!
>
> What kind of hardware are you running this on? I'm passing 214 here.
this is some old piece of hw, intel 2x 2.2Ghz CPU, 2GB memory, ICH8, SATA
controller is 82801HBM/HEM.
partition setup:
SCRATCH
Label: none uuid: b3aabfae-69f6-4b9b-9dc0-e1ec3472183c
Total devices 1 FS bytes used 11.32MB
devid 1 size 104.00MB used 104.00MB path /dev/sda9
TEST
Label: none uuid: d0e2fc5f-8a72-4c54-bee3-3c8ae1766ade
Total devices 4 FS bytes used 15.16MB
devid 4 size 10.00GB used 1.01GB path /dev/sda8
devid 3 size 10.00GB used 1.01GB path /dev/sda7
devid 2 size 10.00GB used 2.00GB path /dev/sda6
devid 1 size 10.00GB used 2.02GB path /dev/sda5
mkfs was default.
the second box, which reproduced this same BUG, is 8 CPU AMD,
partition setup is:
Label: none uuid: 686f0422-2bc5-4265-a4a7-f17f21cdcfd0
Total devices 1 FS bytes used 11.31MB
devid 1 size 104.00MB used 104.00MB path /dev/sda9
Label: none uuid: c17552f9-4727-411d-9b75-5d84a1c5ba66
Total devices 4 FS bytes used 13.18MB
devid 4 size 30.00GB used 30.00GB path /dev/sda8
devid 3 size 30.00GB used 30.00GB path /dev/sda7
devid 2 size 30.00GB used 30.00GB path /dev/sda6
devid 1 size 30.00GB used 30.00GB path /dev/sda5
and the crashdump is
[ 5326.176937] ------------[ cut here ]------------
[ 5326.180791] kernel BUG at fs/btrfs/tree-log.c:3106!
[ 5326.180791] invalid opcode: 0000 [#1] SMP
[ 5326.180791] CPU 5
[ 5326.180791] Modules linked in: btrfs zlib_deflate crc32c libcrc32c edd nfs lockd fscache auth_rpcgss nfs_acl sunrpc af_packet bridge ipv6 stp llc cpufreq_conservative cpufreq_userspace cpufreq_powersave powernow_k8 mperf microcode fuse loop dm_mod kvm_amd kvm tpm_infineon tg3 amd64_edac_mod edac_core shpchp edac_mce_amd i2c_piix4 i2c_core pcspkr tpm_tis tpm tpm_bios pci_hotplug sg sr_mod k10temp cdrom rtc_cmos button mptctl ext3 jbd mbcache ohci_hcd ehci_hcd usbcore sd_mod crc_t10dif fan thermal processor thermal_sys hwmon ide_pci_generic serverworks ide_core ata_generic sata_svw pata_serverworks libata mptsas mptscsih mptbase scsi_transport_sas scsi_mod [last unloaded: floppy]
[ 5326.180791]
[ 5326.180791] Pid: 17588, comm: xfs_io Tainted: G W 3.1.0-default+ #3
[ 5326.180791] RIP: 0010:[<ffffffffa05e0504>] [<ffffffffa05e0504>] btrfs_log_inode+0x4c4/0x680 [btrfs]
[ 5326.180791] RSP: 0018:ffff880054857d18 EFLAGS: 00010282
[ 5326.180791] RAX: 00000000ffffffef RBX: 0000000000000353 RCX: ffff88011d703300
[ 5326.180791] RDX: 000000000000002e RSI: 0000000000000001 RDI: 0000000000000286
[ 5326.180791] RBP: ffff880054857e38 R08: ffff880054857aa8 R09: 0000000000001000
[ 5326.180791] R10: 00000000000000da R11: 0000000000000000 R12: ffff8801297eb400
[ 5326.180791] R13: 0000000000000003 R14: ffff880024d781e0 R15: ffff88011861f1d8
[ 5326.180791] FS: 00007f2c3354e700(0000) GS:ffff88012fc80000(0000) knlGS:0000000000000000
[ 5326.180791] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 5326.180791] CR2: 00007f4d712f8c30 CR3: 0000000127a31000 CR4: 00000000000006e0
[ 5326.180791] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 5326.180791] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 5326.180791] Process xfs_io (pid: 17588, threadinfo ffff880054856000, task ffff880076a26300)
[ 5326.180791] Stack:
[ 5326.180791] ffff880000000000 0007ffffffffffff ffff880054857d78 00000000810e85e0
[ 5326.180791] ffff88007ba7f000 ffff8801101e8108 0000000000000000 0000000000000c23
[ 5326.180791] 0000000000000c23 ffff88011861f0e0 ffff880110030ac0 0000001c00000000
[ 5326.180791] Call Trace:
[ 5326.180791] [<ffffffff810e9db3>] ? filemap_fdatawait_range+0x93/0x150
[ 5326.180791] [<ffffffffa05e080c>] btrfs_log_inode_parent+0x14c/0x2d0 [btrfs]
[ 5326.180791] [<ffffffffa05ac55f>] ? join_transaction+0x1f/0x250 [btrfs]
[ 5326.180791] [<ffffffffa05e0a44>] btrfs_log_dentry_safe+0x44/0x70 [btrfs]
[ 5326.180791] [<ffffffffa05badbf>] btrfs_sync_file+0x16f/0x1c0 [btrfs]
[ 5326.180791] [<ffffffff81172283>] vfs_fsync_range+0x23/0x30
[ 5326.180791] [<ffffffff811722a7>] vfs_fsync+0x17/0x20
[ 5326.180791] [<ffffffff811724b5>] do_fsync+0x35/0x60
[ 5326.180791] [<ffffffff8117250b>] sys_fsync+0xb/0x10
[ 5326.180791] [<ffffffff814232d2>] system_call_fastpath+0x16/0x1b
[ 5326.180791] Code: 3c ff ff ff 45 89 e9 48 8b bd 08 ff ff ff 4c 89 f1 4c 89 fe 89 14 24 48 8b 95 30 ff ff ff e8 e4 f6 ff ff 85 c0 0f 84 c5 fc ff ff <0f> 0b eb fe 0f 1f 84 00 00 00 00 00 45 85 ed 74 33 8b 85 fc fe
[ 5326.180791] RIP [<ffffffffa05e0504>] btrfs_log_inode+0x4c4/0x680 [btrfs]
[ 5326.180791] RSP <ffff880054857d18>
[ 5326.482211] ---[ end trace 0a323c63265b6de4 ]---
[while writing this mail, testbox1 just tripped over the BUG again]
In both cases, the xfstests suite was run on clean mkfs and from test 001 on.
I see that process "rm -f /mnt/sda5/ouch2" is stuck at
$ cat /proc/17594/stack
[<ffffffff81152ae1>] vfs_unlink+0x61/0x120
[<ffffffff81156723>] do_unlinkat+0x1d3/0x1e0
[<ffffffff811568ad>] sys_unlinkat+0x1d/0x40
[<ffffffff814232d2>] system_call_fastpath+0x16/0x1b
[<ffffffffffffffff>] 0xffffffffffffffff
So, bug is reproducible quite reliably.
david
^ permalink raw reply
* Re: [RFC PATCH 2/2] Btrfs: introduce free space cluster for each node
From: Chris Mason @ 2011-11-01 16:04 UTC (permalink / raw)
To: Miao Xie; +Cc: Linux Btrfs
In-Reply-To: <4EAFA21F.5010200@cn.fujitsu.com>
On Tue, Nov 01, 2011 at 03:39:11PM +0800, Miao Xie wrote:
> Any Comment?
This is definitely interesting, in terms of trying to avoid btree
fragmentation and improve the performance of the allocator.
But I'm worried about what happens to performance as the FS fills up?
What kind of benchmarks have you done on larger filesystems that are
almost full?
-chris
^ permalink raw reply
* Re: Where can I find symlink's target?
From: Hugo Mills @ 2011-11-01 16:03 UTC (permalink / raw)
To: email; +Cc: linux-btrfs
In-Reply-To: <edb53b9dd6dcf054485278225f6ee728@joachim-neu.de>
[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]
On Tue, Nov 01, 2011 at 02:29:50PM +0000, email@joachim-neu.de wrote:
>
> Hello,
>
> I'm taking a look at the B-tree level of btrfs. I have a dir item
> which is a symlink and I'm wondering where I can get the symlink's
> target from.
I'm not 100% certain about the btrfs implementation from memory,
but doesn't a symlink simply have file contents that give the link's
target? For something as small as a symlink, this will probably be an
inline extent.
> I hope anyone can tell me? Is there a bloody technical documentation
> of btrfs somewhere out there that can answer me questions like this?
It's all documented in some detail on the wiki[1], which is
currently down after the kernel.org hack. We're expecting it to be
back by the end of this week. (And if it's not, we're told we'll get a
dump of the database, so we can have it back up somewhere else shortly
after that).
Hugo.
[1] http://btrfs.wiki.kernel.org/
--
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
--- Is it true that "last known good" on Windows XP ---
boots into CP/M?
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 190 bytes --]
^ permalink raw reply
* Re: Where can I find symlink's target?
From: David Sterba @ 2011-11-01 15:56 UTC (permalink / raw)
To: email; +Cc: linux-btrfs
In-Reply-To: <edb53b9dd6dcf054485278225f6ee728@joachim-neu.de>
On Tue, Nov 01, 2011 at 02:29:50PM +0000, email@joachim-neu.de wrote:
> I'm taking a look at the B-tree level of btrfs. I have a dir item which
> is a symlink and I'm wondering where I can get the symlink's target
> from.
The symlink target is stored as inline data, inside the leaf, see
btrfs_symlink how it's constructed.
> I hope anyone can tell me? Is there a bloody technical documentation of
> btrfs somewhere out there that can answer me questions like this?
Nope, I wish there were. Some bits were at the wiki.
david
^ permalink raw reply
* Re: Unable to mount (or, why not to work late at night).
From: Dave @ 2011-11-01 15:06 UTC (permalink / raw)
To: Ken D'Ambrosio; +Cc: Fajar A. Nugraha, linux-btrfs
In-Reply-To: <7d421a89143f5c4811d64e08c6820c29@www.jots.org>
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
On Thu, Oct 27, 2011 at 08:32:09PM -0400, Ken D'Ambrosio wrote:
> root@ubuntu:/tmp/btrfs-progs# ./btrfs-zero-log /dev/sda1
> parent transid verify failed on 100695031808 wanted 217713 found 217732
> parent transid verify failed on 100695031808 wanted 217713 found 217732
> parent transid verify failed on 100695031808 wanted 217713 found 217732
> btrfs-zero-log: disk-io.c:413: find_and_setup_root: Assertion '!(!root->node)'
> failed.
This is the same bullshit I dealt with back in August. In short, your data is
likely fine. btrfs is preventing you from accessing your data because there is
something wrong with the csum tree. My solution was to create a custom kernel
which ignored all checksums. I was then able to mount the filesystem read-only
and copy data to another volume.
http://thread.gmane.org/gmane.comp.file-systems.btrfs/12344
I've had this happen to me on two occasions. The first time was after a hard
reboot. The second was on a totally different machine at a different geographic
location, which occurred after nothing more than a reboot.
- --
- -=[dave]=-
Entropy isn't what it used to be.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
iF4EAREIAAYFAk6wCtwACgkQXM0u5ajNnCj7bQD+LWnK+rF6t3ajkeFfFYfap8eA
eaeRWtH/uVkO3RMTFEMA/1H9aBSDWxfhtFd6a11KRYm5MYYonYtfMtpDqxgl7H0k
=MNIf
-----END PGP SIGNATURE-----
^ permalink raw reply
* Where can I find symlink's target?
From: email @ 2011-11-01 14:29 UTC (permalink / raw)
To: linux-btrfs
Hello,
I'm taking a look at the B-tree level of btrfs. I have a dir item which
is a symlink and I'm wondering where I can get the symlink's target
from.
I hope anyone can tell me? Is there a bloody technical documentation of
btrfs somewhere out there that can answer me questions like this?
Thanks!
Joachim
^ permalink raw reply
* Re: ENOSPC on almost empty FS
From: Mitch Harder @ 2011-11-01 14:01 UTC (permalink / raw)
To: linux-btrfs
In-Reply-To: <slrnjavbrs.81n.stephane.chazelas@spam.is.invalid>
On Tue, Nov 1, 2011 at 3:36 AM, Stephane CHAZELAS
<stephane_chazelas@yahoo.fr> wrote:
> Hiya,
>
> trying to restore a FS from a backup (tgz) on a freshly made
> btrfs this morning, I got ENOSPCs after about 100MB out of 4GB
> have been extracted. strace indicates that the ENOSPC are upon
> the open(O_WRONLY).
>
> Restoring with:
>
> mkfs.btrfs /dev/mapper/VG_USB-root
> mount -o compress-force,ssd $_ /mnt
> cd /mnt
> pv ~/backup.tgz | gunzip | sudo bsdtar -xpSf - --numeric-owner
>
>
> That's on a LVM LV with the PV on a USB key.
>
> If I supspend the job and resume it, then the ENOSPCs go away.
>
> The only way I could restore the backup was via rate limiting
> the untar:
>
> zcat ~/backup.tgz | pv -L 3000000 | sudo bsdtar -xpSf - --numeric-owner
>
> That 3MB/s wasn't even enough, as 3 files triggered a ENOSPC,
> but I did untar them separately afterwards.
>
> That's with debian's 3.0.0-1 amd64 kernel.
>
> Is that expected behavior due to the way allocation works in
> btrfs?
>
I have also run into similar behavior while restoring a volume with tar.
I believe this is fixed by one of Josef's recent patches, but it might
be one of the patches queued for 3.2.
^ permalink raw reply
* Warnings and crash
From: David Sterba @ 2011-11-01 12:56 UTC (permalink / raw)
To: Chris Mason, linux-btrfs
In-Reply-To: <20111101024957.GH4468@shiny>
the related source pointers of the warnings and crash (detailed below):
[ 1186.540073] WARNING: at fs/btrfs/extent-tree.c:5837 btrfs_alloc_free_block+0x337/0x350 [btrfs]()
[ 1186.541076] WARNING: at fs/btrfs/extent-tree.c:4617 __btrfs_free_extent+0x60d/0x6c0 [btrfs]()
[ 1186.541234] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 1186.541237] IP: [<ffffffffa005b881>] map_private_extent_buffer+0x21/0x1f0 [btrfs]
fs/btrfs/extent-tree.c:5837
This not a new warning, also known as 'the warn in use_block_rsv', happened
during xfstests/224.
The ratelimited printk patch (btrfs: ratelimit WARN_ON in use_block_rsv) is
part of the btrfs-next-stable branch pulled in, though is still meant as a
safety not to hang the box completely (and should go away in the future).
[ 1186.529033] btrfs: block rsv returned -28
[ 1186.534256] ------------[ cut here ]------------
[ 1186.540073] WARNING: at fs/btrfs/extent-tree.c:5837 btrfs_alloc_free_block+0x337/0x350 [btrfs]()
[ 1186.540075] Hardware name: Santa Rosa platform
[ 1186.540077] Modules linked in: btrfs aoe
[ 1186.540082] Pid: 5007, comm: btrfs-transacti Not tainted 3.1.0-default+ #64
[ 1186.540084] Call Trace:
[ 1186.540091] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1186.540095] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1186.540104] [<ffffffffa002a9d7>] btrfs_alloc_free_block+0x337/0x350 [btrfs]
[ 1186.540109] [<ffffffff810913dd>] ? trace_hardirqs_off+0xd/0x10
[ 1186.540113] [<ffffffff810098d5>] ? native_sched_clock+0x15/0x70
[ 1186.540118] [<ffffffff810844ff>] ? local_clock+0x6f/0x80
[ 1186.540126] [<ffffffffa0017162>] __btrfs_cow_block+0x152/0x800 [btrfs]
[ 1186.540130] [<ffffffff810844ff>] ? local_clock+0x6f/0x80
[ 1186.540133] [<ffffffff810916fd>] ? lock_release_holdtime+0x3d/0x1a0
[ 1186.540141] [<ffffffffa0017ae1>] btrfs_cow_block+0x2d1/0x410 [btrfs]
[ 1186.540150] [<ffffffffa001cb88>] btrfs_search_slot+0x728/0xe70 [btrfs]
[ 1186.540159] [<ffffffffa0022f0b>] lookup_inline_extent_backref+0x8b/0x480 [btrfs]
[ 1186.540168] [<ffffffffa00245f0>] __btrfs_free_extent+0xd0/0x6c0 [btrfs]
[ 1186.540172] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.540182] [<ffffffffa00287cf>] run_clustered_refs+0x21f/0x850 [btrfs]
[ 1186.540192] [<ffffffffa0028ec8>] btrfs_run_delayed_refs+0xc8/0x220 [btrfs]
[ 1186.540195] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.540205] [<ffffffffa0029178>] btrfs_write_dirty_block_groups+0x158/0x540 [btrfs]
[ 1186.540216] [<ffffffffa0038ee5>] commit_cowonly_roots+0x115/0x1e0 [btrfs]
[ 1186.540228] [<ffffffffa0039ee7>] btrfs_commit_transaction+0x477/0x8f0 [btrfs]
[ 1186.540240] [<ffffffffa00391a9>] ? join_transaction+0x69/0x2b0 [btrfs]
[ 1186.540243] [<ffffffff8107d480>] ? wake_up_bit+0x40/0x40
[ 1186.540254] [<ffffffffa0032a1b>] transaction_kthread+0x28b/0x2a0 [btrfs]
[ 1186.540264] [<ffffffffa0032790>] ? btree_readpage+0x30/0x30 [btrfs]
[ 1186.540267] [<ffffffff8107cf16>] kthread+0xa6/0xb0
[ 1186.540272] [<ffffffff81a1ea04>] kernel_thread_helper+0x4/0x10
[ 1186.540276] [<ffffffff81a15334>] ? retint_restore_args+0x13/0x13
[ 1186.540279] [<ffffffff8107ce70>] ? __init_kthread_worker+0x70/0x70
[ 1186.540282] [<ffffffff81a1ea00>] ? gs_change+0x13/0x13
[ 1186.540284] ---[ end trace 6fa3da5fd65420af ]---
[ 1186.540433] btrfs: block rsv returned -28
ratelimited, I see just 2 occurences before, followed by
[ 1186.541076] WARNING: at fs/btrfs/extent-tree.c:4617 __btrfs_free_extent+0x60d/0x6c0 [btrfs]()
[ 1186.541079] Hardware name: Santa Rosa platform
[ 1186.541080] Modules linked in: btrfs aoe
[ 1186.541084] Pid: 5007, comm: btrfs-transacti Tainted: G W 3.1.0-default+ #64
[ 1186.541086] Call Trace:
[ 1186.541089] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1186.541093] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1186.541101] [<ffffffffa0024b2d>] __btrfs_free_extent+0x60d/0x6c0 [btrfs]
[ 1186.541105] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541114] [<ffffffffa0028949>] run_clustered_refs+0x399/0x850 [btrfs]
[ 1186.541126] [<ffffffffa0079800>] ? btrfs_find_ref_cluster+0xb0/0x1a0 [btrfs]
[ 1186.541136] [<ffffffffa0028ec8>] btrfs_run_delayed_refs+0xc8/0x220 [btrfs]
[ 1186.541139] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541149] [<ffffffffa0029178>] btrfs_write_dirty_block_groups+0x158/0x540 [btrfs]
[ 1186.541160] [<ffffffffa0038ee5>] commit_cowonly_roots+0x115/0x1e0 [btrfs]
[ 1186.541172] [<ffffffffa0039ee7>] btrfs_commit_transaction+0x477/0x8f0 [btrfs]
[ 1186.541183] [<ffffffffa00391a9>] ? join_transaction+0x69/0x2b0 [btrfs]
[ 1186.541186] [<ffffffff8107d480>] ? wake_up_bit+0x40/0x40
[ 1186.541196] [<ffffffffa0032a1b>] transaction_kthread+0x28b/0x2a0 [btrfs]
[ 1186.541207] [<ffffffffa0032790>] ? btree_readpage+0x30/0x30 [btrfs]
[ 1186.541210] [<ffffffff8107cf16>] kthread+0xa6/0xb0
[ 1186.541213] [<ffffffff81a1ea04>] kernel_thread_helper+0x4/0x10
[ 1186.541216] [<ffffffff81a15334>] ? retint_restore_args+0x13/0x13
[ 1186.541219] [<ffffffff8107ce70>] ? __init_kthread_worker+0x70/0x70
[ 1186.541222] [<ffffffff81a1ea00>] ? gs_change+0x13/0x13
[ 1186.541224] ---[ end trace 6fa3da5fd65420b1 ]---
[ 1186.541076] WARNING: at fs/btrfs/extent-tree.c:4617 __btrfs_free_extent+0x60d/0x6c0 [btrfs]()
[ 1186.541079] Hardware name: Santa Rosa platform
[ 1186.541080] Modules linked in: btrfs aoe
[ 1186.541084] Pid: 5007, comm: btrfs-transacti Tainted: G W 3.1.0-default+ #64
[ 1186.541086] Call Trace:
[ 1186.541089] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1186.541093] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1186.541101] [<ffffffffa0024b2d>] __btrfs_free_extent+0x60d/0x6c0 [btrfs]
[ 1186.541105] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541114] [<ffffffffa0028949>] run_clustered_refs+0x399/0x850 [btrfs]
[ 1186.541126] [<ffffffffa0079800>] ? btrfs_find_ref_cluster+0xb0/0x1a0 [btrfs]
[ 1186.541136] [<ffffffffa0028ec8>] btrfs_run_delayed_refs+0xc8/0x220 [btrfs]
[ 1186.541139] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541149] [<ffffffffa0029178>] btrfs_write_dirty_block_groups+0x158/0x540 [btrfs]
[ 1186.541160] [<ffffffffa0038ee5>] commit_cowonly_roots+0x115/0x1e0 [btrfs]
[ 1186.541172] [<ffffffffa0039ee7>] btrfs_commit_transaction+0x477/0x8f0 [btrfs]
[ 1186.541183] [<ffffffffa00391a9>] ? join_transaction+0x69/0x2b0 [btrfs]
[ 1186.541186] [<ffffffff8107d480>] ? wake_up_bit+0x40/0x40
[ 1186.541196] [<ffffffffa0032a1b>] transaction_kthread+0x28b/0x2a0 [btrfs]
[ 1186.541207] [<ffffffffa0032790>] ? btree_readpage+0x30/0x30 [btrfs]
[ 1186.541210] [<ffffffff8107cf16>] kthread+0xa6/0xb0
[ 1186.541213] [<ffffffff81a1ea04>] kernel_thread_helper+0x4/0x10
[ 1186.541216] [<ffffffff81a15334>] ? retint_restore_args+0x13/0x13
[ 1186.541219] [<ffffffff8107ce70>] ? __init_kthread_worker+0x70/0x70
[ 1186.541222] [<ffffffff81a1ea00>] ? gs_change+0x13/0x13
[ 1186.541224] ---[ end trace 6fa3da5fd65420b1 ]---
4615 } else {
4616 btrfs_print_leaf(extent_root, path->nodes[0]);
4617 WARN_ON(1);
4618 printk(KERN_ERR "btrfs unable to find ref byte nr %llu "
4619 "parent %llu root %llu owner %llu offset %llu\n",
4620 (unsigned long long)bytenr,
4621 (unsigned long long)parent,
4622 (unsigned long long)root_objectid,
4623 (unsigned long long)owner_objectid,
4624 (unsigned long long)owner_offset);
4625 }
[ 1186.541227] btrfs unable to find ref byte nr 428474368 parent 0 root 7 owner 0 offset 0
[ 1186.541234] BUG: unable to handle kernel NULL pointer dereference at (null)
[ 1186.541237] IP: [<ffffffffa005b881>] map_private_extent_buffer+0x21/0x1f0 [btrfs]
[ 1186.541250] PGD 51ece067 PUD 6425d067 PMD 0
[ 1186.541253] Oops: 0000 [#1] SMP
[ 1186.541257] CPU 0
[ 1186.541258] Modules linked in: btrfs aoe
[ 1186.541261]
[ 1186.541263] Pid: 5007, comm: btrfs-transacti Tainted: G W 3.1.0-default+ #64 Intel Corporation Santa Rosa platform/Matanzas
[ 1186.541268] RIP: 0010:[<ffffffffa005b881>] [<ffffffffa005b881>] map_private_extent_buffer+0x21/0x1f0 [btrfs]
[ 1186.541281] RSP: 0018:ffff8800795d1960 EFLAGS: 00010286
[ 1186.541283] RAX: 0000000000000000 RBX: 0000000000000065 RCX: ffff8800795d19d0
[ 1186.541285] RDX: 0000000000000004 RSI: 000000000000007a RDI: 0000000000000000
[ 1186.541288] RBP: ffff8800795d19b0 R08: ffff8800795d19c8 R09: ffff8800795d19c0
[ 1186.541290] R10: 0000000000000001 R11: 0000000000000001 R12: 000000000000007a
[ 1186.541292] R13: 0000000000000000 R14: 0000000000000065 R15: 0000000000000000
[ 1186.541295] FS: 0000000000000000(0000) GS:ffff88007c000000(0000) knlGS:0000000000000000
[ 1186.541297] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 1186.541300] CR2: 0000000000000000 CR3: 000000006a578000 CR4: 00000000000006f0
[ 1186.541302] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 1186.541304] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 1186.541307] Process btrfs-transacti (pid: 5007, threadinfo ffff8800795d0000, task ffff88007a00c880)
[ 1186.541309] Stack:
[ 1186.541310] ffff8800795d1970 00ffffff00000003 ffff8800795d19b0 ffff880078da6520
[ 1186.541314] 0000000000000000 0000000000000065 000000000000007a 0000000000000000
[ 1186.541318] 0000000000000065 0000000000000000 ffff8800795d1a00 ffffffffa0050dfc
[ 1186.541322] Call Trace:
[ 1186.541335] [<ffffffffa0050dfc>] btrfs_item_size+0x3c/0x90 [btrfs]
[ 1186.541344] [<ffffffffa0024755>] __btrfs_free_extent+0x235/0x6c0 [btrfs]
[ 1186.541348] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541357] [<ffffffffa0028949>] run_clustered_refs+0x399/0x850 [btrfs]
[ 1186.541369] [<ffffffffa0079800>] ? btrfs_find_ref_cluster+0xb0/0x1a0 [btrfs]
[ 1186.541378] [<ffffffffa0028ec8>] btrfs_run_delayed_refs+0xc8/0x220 [btrfs]
[ 1186.541382] [<ffffffff813628be>] ? do_raw_spin_unlock+0x5e/0xb0
[ 1186.541391] [<ffffffffa0029178>] btrfs_write_dirty_block_groups+0x158/0x540 [btrfs]
[ 1186.541402] [<ffffffffa0038ee5>] commit_cowonly_roots+0x115/0x1e0 [btrfs]
[ 1186.541414] [<ffffffffa0039ee7>] btrfs_commit_transaction+0x477/0x8f0 [btrfs]
[ 1186.541425] [<ffffffffa00391a9>] ? join_transaction+0x69/0x2b0 [btrfs]
[ 1186.541428] [<ffffffff8107d480>] ? wake_up_bit+0x40/0x40
[ 1186.541439] [<ffffffffa0032a1b>] transaction_kthread+0x28b/0x2a0 [btrfs]
[ 1186.541449] [<ffffffffa0032790>] ? btree_readpage+0x30/0x30 [btrfs]
[ 1186.541452] [<ffffffff8107cf16>] kthread+0xa6/0xb0
[ 1186.541456] [<ffffffff81a1ea04>] kernel_thread_helper+0x4/0x10
[ 1186.541459] [<ffffffff81a15334>] ? retint_restore_args+0x13/0x13
[ 1186.541462] [<ffffffff8107ce70>] ? __init_kthread_worker+0x70/0x70
[ 1186.541464] [<ffffffff81a1ea00>] ? gs_change+0x13/0x13
[ 1186.541466] Code: 55 90 e9 b6 fc ff ff 0f 1f 00 55 48 89 e5 48 83 ec 50 48 89 5d d8 4c 89 65 e0 4c 89 6d e8 4c 89 75 f0 4c 89 7d f8 66 66 66 66 90 <4c> 8b 37 49 89 cd 4c 89 cb 41 81 e6 ff 0f 00 00 4a 8d 04 36 48
[ 1186.541493] RIP [<ffffffffa005b881>] map_private_extent_buffer+0x21/0x1f0 [btrfs]
[ 1186.541506] RSP <ffff8800795d1960>
[ 1186.541507] CR2: 0000000000000000
[ 1187.258581] ---[ end trace 6fa3da5fd65420b2 ]---
david
^ permalink raw reply
* Re: [PATCH 17/21] Btrfs: allow for pausing restriper
From: Arne Jansen @ 2011-11-01 11:46 UTC (permalink / raw)
To: Ilya Dryomov; +Cc: linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <1314129722-31601-18-git-send-email-idryomov@gmail.com>
On 23.08.2011 22:01, Ilya Dryomov wrote:
> Implement an ioctl for pausing restriper. This pauses the relocation,
> but restripe is still considered to be "in progress": restriper item is
> not deleted, other volume operations cannot be started, etc. If paused
> in the middle of profile changing operation we will continue making
> allocations with the target profile.
>
> Add a hook to close_ctree() to be able to pause restriper and free it's
> data structures on unmount. (It's safe to unmount when restriper is in
> 'paused' state, we will resume with the same parameters on the next
> mount)
>
> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
> ---
> fs/btrfs/disk-io.c | 3 +++
> fs/btrfs/ioctl.c | 2 ++
> fs/btrfs/ioctl.h | 1 +
> fs/btrfs/volumes.c | 44 ++++++++++++++++++++++++++++++++++++++++++--
> fs/btrfs/volumes.h | 2 ++
> 5 files changed, 50 insertions(+), 2 deletions(-)
>
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index 662a6e6..7db5c50 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -2542,6 +2542,9 @@ int close_ctree(struct btrfs_root *root)
> fs_info->closing = 1;
> smp_mb();
>
> + /* pause restriper and free restripe_ctl */
> + btrfs_pause_restripe(root->fs_info, 1);
> +
> btrfs_scrub_cancel(root);
>
> /* wait for any defraggers to finish */
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index d8bdb67..61978ac 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -2921,6 +2921,8 @@ static long btrfs_ioctl_restripe_ctl(struct btrfs_root *root,
> switch (cmd) {
> case BTRFS_RESTRIPE_CTL_CANCEL:
> return btrfs_cancel_restripe(root->fs_info);
> + case BTRFS_RESTRIPE_CTL_PAUSE:
> + return btrfs_pause_restripe(root->fs_info, 0);
> }
>
> return -EINVAL;
> diff --git a/fs/btrfs/ioctl.h b/fs/btrfs/ioctl.h
> index 4f6ead5..e468d5b 100644
> --- a/fs/btrfs/ioctl.h
> +++ b/fs/btrfs/ioctl.h
> @@ -110,6 +110,7 @@ struct btrfs_ioctl_fs_info_args {
> };
>
> #define BTRFS_RESTRIPE_CTL_CANCEL 1
> +#define BTRFS_RESTRIPE_CTL_PAUSE 2
>
> struct btrfs_restripe_args {
> __u64 profiles;
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index cd43368..65deaa7 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -2555,7 +2555,8 @@ static int __btrfs_restripe(struct btrfs_root *dev_root)
> while (1) {
> struct btrfs_fs_info *fs_info = dev_root->fs_info;
>
> - if (test_bit(RESTRIPE_CANCEL_REQ, &fs_info->restripe_state)) {
> + if (test_bit(RESTRIPE_CANCEL_REQ, &fs_info->restripe_state) ||
> + test_bit(RESTRIPE_PAUSE_REQ, &fs_info->restripe_state)) {
> ret = -ECANCELED;
> goto error;
> }
> @@ -2730,7 +2731,9 @@ do_restripe:
> mutex_lock(&fs_info->restripe_mutex);
> clear_bit(RESTRIPE_RUNNING, &fs_info->restripe_state);
>
> - if (test_bit(RESTRIPE_CANCEL_REQ, &fs_info->restripe_state)) {
> + if (test_bit(RESTRIPE_CANCEL_REQ, &fs_info->restripe_state) ||
> + (!test_bit(RESTRIPE_PAUSE_REQ, &fs_info->restripe_state) &&
> + !test_bit(RESTRIPE_CANCEL_REQ, &fs_info->restripe_state))) {
> mutex_lock(&fs_info->volume_mutex);
>
> unset_restripe_control(fs_info);
> @@ -2858,6 +2861,43 @@ out:
> return ret;
> }
I don't see a difference in what CANCEL_REQ and PAUSE_REQ do, so it seems
one of them would be enough.
>
> +int btrfs_pause_restripe(struct btrfs_fs_info *fs_info, int unset)
> +{
> + int ret = 0;
> +
> + mutex_lock(&fs_info->restripe_mutex);
> + if (!fs_info->restripe_ctl) {
> + ret = -ENOTCONN;
> + goto out;
> + }
> +
> + /* only running restripe can be paused */
> + if (!test_bit(RESTRIPE_RUNNING, &fs_info->restripe_state)) {
> + ret = -ENOTCONN;
> + goto out_unset;
> + }
> +
> + set_bit(RESTRIPE_PAUSE_REQ, &fs_info->restripe_state);
> + while (test_bit(RESTRIPE_RUNNING, &fs_info->restripe_state)) {
> + mutex_unlock(&fs_info->restripe_mutex);
> + wait_event(fs_info->restripe_wait,
> + !test_bit(RESTRIPE_RUNNING,
> + &fs_info->restripe_state));
> + mutex_lock(&fs_info->restripe_mutex);
> + }
> + clear_bit(RESTRIPE_PAUSE_REQ, &fs_info->restripe_state);
> +
> +out_unset:
> + if (unset) {
> + mutex_lock(&fs_info->volume_mutex);
> + unset_restripe_control(fs_info);
> + mutex_unlock(&fs_info->volume_mutex);
> + }
> +out:
> + mutex_unlock(&fs_info->restripe_mutex);
> + return ret;
> +}
> +
This looks very similar to cancel_restripe. It should be easy to
merge them to one function without making a mess out of it.
> /*
> * shrinking a device means finding all of the device extents past
> * the new size, and then following the back refs to the chunks.
> diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
> index dd1fa7f..b8c234a 100644
> --- a/fs/btrfs/volumes.h
> +++ b/fs/btrfs/volumes.h
> @@ -204,6 +204,7 @@ struct map_lookup {
> */
> #define RESTRIPE_RUNNING 0
> #define RESTRIPE_CANCEL_REQ 1
> +#define RESTRIPE_PAUSE_REQ 2
>
> struct btrfs_restripe_args;
> struct restripe_control {
> @@ -261,6 +262,7 @@ int btrfs_balance(struct btrfs_root *dev_root);
> int btrfs_restripe(struct restripe_control *rctl, int resume);
> int btrfs_recover_restripe(struct btrfs_root *tree_root);
> int btrfs_cancel_restripe(struct btrfs_fs_info *fs_info);
> +int btrfs_pause_restripe(struct btrfs_fs_info *fs_info, int unset);
> int btrfs_chunk_readonly(struct btrfs_root *root, u64 chunk_offset);
> int find_free_dev_extent(struct btrfs_trans_handle *trans,
> struct btrfs_device *device, u64 num_bytes,
^ permalink raw reply
* Re: [PATCH 05/21] Btrfs: add basic restriper infrastructure
From: Arne Jansen @ 2011-11-01 11:08 UTC (permalink / raw)
To: Ilya Dryomov, linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <20111101110734.GE12759@twin.jikos.cz>
On 01.11.2011 12:07, David Sterba wrote:
> On Tue, Nov 01, 2011 at 11:08:38AM +0100, Arne Jansen wrote:
>>> +/*
>>> + * Should be called with restripe_mutex held
>>> + */
>>> +int btrfs_restripe(struct restripe_control *rctl)
>>> +{
> ...
>>> + if (rctl->data.target & BTRFS_BLOCK_GROUP_DUP) {
>>> + printk(KERN_ERR "btrfs: dup for data is not allowed\n");
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>
>> It would be good to get these error messages somehow to the user,
>> or at least give the user a hint to look in dmesg.
>
> the restriper command ends with EINVAL which is in most cases returned
> as a result of the ioctl and progs counterpart will
>
> 1117 fprintf(stderr, "ERROR: error during restriping '%s' "
> 1118 "- %s\n", path, strerror(e));
> 1119 return 19;
>
> the hint should go there imho.
Though it would still be much nicer to get a proper error message to the
user directly.
>
>
> david
^ permalink raw reply
* Re: [PATCH 05/21] Btrfs: add basic restriper infrastructure
From: David Sterba @ 2011-11-01 11:07 UTC (permalink / raw)
To: Arne Jansen; +Cc: Ilya Dryomov, linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <4EAFC526.4040701@gmx.net>
On Tue, Nov 01, 2011 at 11:08:38AM +0100, Arne Jansen wrote:
> > +/*
> > + * Should be called with restripe_mutex held
> > + */
> > +int btrfs_restripe(struct restripe_control *rctl)
> > +{
...
> > + if (rctl->data.target & BTRFS_BLOCK_GROUP_DUP) {
> > + printk(KERN_ERR "btrfs: dup for data is not allowed\n");
> > + ret = -EINVAL;
> > + goto out;
> > + }
>
> It would be good to get these error messages somehow to the user,
> or at least give the user a hint to look in dmesg.
the restriper command ends with EINVAL which is in most cases returned
as a result of the ioctl and progs counterpart will
1117 fprintf(stderr, "ERROR: error during restriping '%s' "
1118 "- %s\n", path, strerror(e));
1119 return 19;
the hint should go there imho.
david
^ permalink raw reply
* Re: [PATCH 15/21] Btrfs: recover restripe on mount
From: Arne Jansen @ 2011-11-01 10:57 UTC (permalink / raw)
To: Ilya Dryomov; +Cc: linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <1314129722-31601-16-git-send-email-idryomov@gmail.com>
On 23.08.2011 22:01, Ilya Dryomov wrote:
> On mount, if restripe item is found, resume restripe in a separate
> kernel thread.
>
> Try to be smart to continue roughly where previous balance (or convert)
> was interrupted. For chunk types that were being converted to some
> profile we turn on soft convert, in case of a simple balance we turn on
> usage filter and relocate only less-than-90%-full chunks of that type.
> These are just heuristics but they help quite a bit, and can be improved
> in future.
>
Instead of trying to find out where you left off, can't you just save a
pointer in your restripe_item every time a chunk finished?
^ permalink raw reply
* Re: BUG at fs/btrfs/tree-log.c:3106 (Re: new integration-danger branch pushed out (kernel))
From: Chris Mason @ 2011-11-01 10:56 UTC (permalink / raw)
To: liubo2009, linux-btrfs
In-Reply-To: <20111101103637.GD12759@twin.jikos.cz>
On Tue, Nov 01, 2011 at 11:36:37AM +0100, David Sterba wrote:
> xfstests/214
>
> [ 3142.654774] ------------[ cut here ]------------
> [ 3142.656017] kernel BUG at fs/btrfs/tree-log.c:3106!
What kind of hardware are you running this on? I'm passing 214 here.
-chris
^ permalink raw reply
* Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones
From: Johannes Weiner @ 2011-11-01 10:55 UTC (permalink / raw)
To: Wu Fengguang
Cc: Michal Hocko, Andrew Morton, Mel Gorman, Christoph Hellwig,
Dave Chinner, Jan Kara, Rik van Riel, Minchan Kim, Chris Mason,
Theodore Ts'o, Andreas Dilger, Li, Shaohua, xfs@oss.sgi.com,
linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <20111031113321.GA30890@localhost>
On Mon, Oct 31, 2011 at 07:33:21PM +0800, Wu Fengguang wrote:
> > //regression
> > 3) much increased cpu %user and %system for btrfs
>
> Sorry I find out that the CPU time regressions for btrfs are caused by
> some additional trace events enabled on btrfs (for debugging an
> unrelated btrfs hang bug) which results in 7 times more trace event
> lines:
>
> 2701238 /export/writeback/thresh=1000M/btrfs-1dd-4k-8p-2941M-1000M:10-3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+
> 19054054 /export/writeback/thresh=1000M/btrfs-1dd-4k-8p-2941M-1000M:10-3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+
>
> So no real regressions.
Phew :-)
> Besides, the patchset also performs good on random writes:
>
> 3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+ 3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+
> ------------------------ ------------------------
> 1.65 -5.1% 1.57 MMAP-RANDWRITE-4K/btrfs-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X
> 18.65 -6.4% 17.46 MMAP-RANDWRITE-4K/ext3-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X
> 2.09 +1.2% 2.12 MMAP-RANDWRITE-4K/ext4-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X
> 2.49 -0.3% 2.48 MMAP-RANDWRITE-4K/xfs-fio_fat_mmap_randwrite_4k-4k-8p-4096M-20:10-X
> 51.35 +0.0% 51.36 MMAP-RANDWRITE-64K/btrfs-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X
> 45.20 +0.5% 45.43 MMAP-RANDWRITE-64K/ext3-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X
> 44.77 +0.7% 45.10 MMAP-RANDWRITE-64K/ext4-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X
> 45.11 +2.5% 46.23 MMAP-RANDWRITE-64K/xfs-fio_fat_mmap_randwrite_64k-64k-8p-4096M-20:10-X
> 211.31 +0.2% 211.74 TOTAL write_bw
Hmm, mmapped IO page allocations are not annotated yet, so I expect
this to be just runtime variations?
> And writes to USB key:
>
> 3.1.0-rc9-ioless-full-nfs-wq5-next-20111014+ 3.1.0-rc9-ioless-full-per-zone-dirty-next-20111014+
> ------------------------ ------------------------
> 5.94 +0.8% 5.99 UKEY-thresh=1G/btrfs-1dd-4k-8p-4096M-1024M:10-X
> 2.64 -0.8% 2.62 UKEY-thresh=1G/ext3-10dd-4k-8p-4096M-1024M:10-X
> 5.10 +0.3% 5.12 UKEY-thresh=1G/ext3-1dd-4k-8p-4096M-1024M:10-X
> 3.26 -0.8% 3.24 UKEY-thresh=1G/ext3-2dd-4k-8p-4096M-1024M:10-X
> 5.63 -0.5% 5.60 UKEY-thresh=1G/ext4-10dd-4k-8p-4096M-1024M:10-X
> 6.04 -0.1% 6.04 UKEY-thresh=1G/ext4-1dd-4k-8p-4096M-1024M:10-X
> 5.90 -0.2% 5.88 UKEY-thresh=1G/ext4-2dd-4k-8p-4096M-1024M:10-X
> 2.45 +22.6% 3.00 UKEY-thresh=1G/xfs-10dd-4k-8p-4096M-1024M:10-X
> 6.18 -0.4% 6.16 UKEY-thresh=1G/xfs-1dd-4k-8p-4096M-1024M:10-X
> 4.81 +0.0% 4.81 UKEY-thresh=1G/xfs-2dd-4k-8p-4096M-1024M:10-X
> 47.94 +1.1% 48.45 TOTAL write_bw
>
> In summary, I see no problem at all in these trivial writeback tests.
>
> Tested-by: Wu Fengguang <fengguang.wu@intel.com>
Thanks!
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* Re: new integration-danger branch pushed out (kernel)
From: Chris Mason @ 2011-11-01 10:55 UTC (permalink / raw)
To: Liu Bo; +Cc: linux-btrfs
In-Reply-To: <4EAFBA4C.1080201@cn.fujitsu.com>
On Tue, Nov 01, 2011 at 05:22:20PM +0800, Liu Bo wrote:
> Hi,
>
> On 11/01/2011 10:49 AM, Chris Mason wrote:
> > Hi everyone,
> >
> > I've pushed out a new integration branch, but it is not for general use.
> >
> > I'm still going through and hammering on the new logging code code to
> > make sure it is properly backwards compatible with older kernels and
> > that it hasn't introduced any new bugs.
> >
> > *** Please do not use this code on anything other than test filesystems that
> > can easily be reformatted. ***
> >
> > In testing the new sub-transid logging code, I've spent a lot of time
> > hunting logging bugs, and writing new tests to try and make sure the
> > logging code is working correctly in general. Between that and some new
> > tests from Arne, I've had to make some pretty big changes to the code.
> >
>
> I'm giving a hard read on the new code, and I notice a slight change:
Thanks a lot for reading it carefully.
> @@ -3006,14 +3179,11 @@ out:
> static int inode_in_log(struct btrfs_trans_handle *trans,
> struct inode *inode)
> {
> - struct btrfs_root *root = BTRFS_I(inode)->root;
> int ret = 0;
>
> - mutex_lock(&root->log_mutex);
> - if (BTRFS_I(inode)->logged_trans == trans->transid &&
> - BTRFS_I(inode)->last_sub_trans <= root->last_log_commit)
> + if (BTRFS_I(inode)->logged_trans >= trans->transaction->transid &&
> + BTRFS_I(inode)->last_trans < BTRFS_I(inode)->logged_trans)
> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> ret = 1;
> - mutex_unlock(&root->log_mutex);
> return ret;
> }
>
> since we've moved the "sub_transid++" update to btrfs_sync_log(), the ending of log,
> say there is a case like this:
>
> transaction ------------------------------------------ start
> |
> sub_trans A | --- start
> |
> | --- modify inode X, and set X's last_trans to sub_transid
> | |
> | | --- fsync inode X, and set X's logged_trans to sub_transid
> | |
> | | --- in btrfs_sync_log(), sub_transid++
> | |
> \|/ | --- end
> |
> sub_trans A+1 | --- start
> |
> | --- no change to inode X
> |
> | --- fsync inode X
> |
> | --- in inode_in_log(), X's last_trans is equal to X's logged_trans,
> unchanged X is just in the log, but we need to log X again.
>
> ...... ......
>
> transaction ------------------------------------------ end
>
> we do not want this, does we? Or am I missing something?
Correct, this is less optimal. I was finding races where the transid
increasing at the start of the log trans would lead to one process
missing file and directory updates done by another process.
So I made the patch slightly more conservative. We're still logging
about 75% less than we used to in every workload I've tried, but we can
definitely refine this in future commits.
Along the way I fixed an old bug in directory logging that meant we
almost always force a full commit for directory fsyncs. The fix means
we use the directory log more often now, which is faster in some cases
and slower in others.
It also means a bug with delayed metadata insertion triggers more easily
during log replay (we need to flush the delayed ops during log replay).
The bug was hidden before because the directory log wasn't being used
much.
-chris
^ permalink raw reply
* Re: [patch 3/5] mm: try to distribute dirty pages fairly across zones
From: Johannes Weiner @ 2011-11-01 10:52 UTC (permalink / raw)
To: Wu Fengguang
Cc: Michal Hocko, Andrew Morton, Mel Gorman, Christoph Hellwig,
Dave Chinner, Jan Kara, Rik van Riel, Minchan Kim, Chris Mason,
Theodore Ts'o, Andreas Dilger, Li, Shaohua, xfs@oss.sgi.com,
linux-btrfs@vger.kernel.org, linux-ext4@vger.kernel.org,
linux-mm@kvack.org, linux-fsdevel@vger.kernel.org,
linux-kernel@vger.kernel.org
In-Reply-To: <20111028203944.GB20607@localhost>
On Sat, Oct 29, 2011 at 04:39:44AM +0800, Wu Fengguang wrote:
> [restore CC list]
>
> > > I'm trying to understand where the performance gain comes from.
> > >
> > > I noticed that in all cases, before/after patchset, nr_vmscan_write are all zero.
> > >
> > > nr_vmscan_immediate_reclaim is significantly reduced though:
> >
> > That's a good thing, it means we burn less CPU time on skipping
> > through dirty pages on the LRU.
> >
> > Until a certain priority level, the dirty pages encountered on the LRU
> > list are marked PageReclaim and put back on the list, this is the
> > nr_vmscan_immediate_reclaim number. And only below that priority, we
> > actually ask the FS to write them, which is nr_vmscan_write.
>
> Yes, it is.
>
> > I suspect this is where the performance improvement comes from: we
> > find clean pages for reclaim much faster.
>
> That explains how it could reduce CPU overheads. However the dd's are
> throttled anyway, so I still don't understand how the speedup of dd page
> allocations improve the _IO_ performance.
They are throttled in balance_dirty_pages() when there are too many
dirty pages. But they are also 'throttled' in direct reclaim when
there are too many clean + dirty pages. Wild guess: speeding up
direct reclaim allows dirty pages to be generated faster and the
writer can better saturate the BDI?
Not all filesystems ignore all VM writepage requests, either. xfs
e.g. ignores only direct reclaim but honors requests from kswapd.
ext4 honors writepage whenever it pleases. On those, I can imagine
the reduced writepage intereference to help. But that can not be the
only reason as btrfs ignores writepage from the reclaim in general and
still sees improvement.
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
^ permalink raw reply
* BUG at fs/btrfs/tree-log.c:3106 (Re: new integration-danger branch pushed out (kernel))
From: David Sterba @ 2011-11-01 10:36 UTC (permalink / raw)
To: liubo2009, Chris Mason, linux-btrfs
In-Reply-To: <20111101024957.GH4468@shiny>
xfstests/214
[ 3142.654774] ------------[ cut here ]------------
[ 3142.656017] kernel BUG at fs/btrfs/tree-log.c:3106!
[ 3142.656017] invalid opcode: 0000 [#1] SMP
[ 3142.656017] CPU 0
[ 3142.656017] Modules linked in: loop btrfs aoe
[ 3142.656017]
[ 3142.656017] Pid: 27328, comm: xfs_io Tainted: G W 3.1.0-default+ #64 Intel Corporation Santa Rosa platform/Matanzas
[ 3142.656017] RIP: 0010:[<ffffffffa006fe74>] [<ffffffffa006fe74>] btrfs_log_inode+0x554/0x600 [btrfs]
[ 3142.656017] RSP: 0018:ffff88005c801d58 EFLAGS: 00010282
[ 3142.656017] RAX: 00000000ffffffef RBX: ffff88003ef39e20 RCX: 0000000000000006
[ 3142.656017] RDX: 0000000000000001 RSI: ffff8800643aa170 RDI: 0000000000000286
[ 3142.656017] RBP: ffff88005c801e48 R08: 0000000000000001 R09: 0000000000000001
[ 3142.656017] R10: 0000000000000001 R11: 0000000000000002 R12: ffff88003405fe70
[ 3142.656017] R13: ffff88005c801db8 R14: 0000000000000003 R15: ffff8800340604b0
[ 3142.656017] FS: 00007f7034667700(0000) GS:ffff88007dc00000(0000) knlGS:0000000000000000
[ 3142.656017] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 3142.656017] CR2: 00000000006b1fb0 CR3: 00000000778df000 CR4: 00000000000006f0
[ 3142.656017] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 3142.656017] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 3142.656017] Process xfs_io (pid: 27328, threadinfo ffff88005c800000, task ffff8800643a9ac0)
[ 3142.656017] Stack:
[ 3142.656017] 0000000000000000 0000000000000000 ffff8800340602b0 ffff880072efe000
[ 3142.656017] 0000000000000000 00000000000004c8 000000005c801dd8 ffff8800340604b0
[ 3142.656017] ffff88003ef39eb0 ffff88000dfb4420 0000000000000019 0000000000000308
[ 3142.656017] Call Trace:
[ 3142.656017] [<ffffffffa0070d28>] btrfs_log_inode_parent+0x198/0x480 [btrfs]
[ 3142.656017] [<ffffffff8115941c>] ? dget_parent+0x1c/0xd0
[ 3142.656017] [<ffffffffa0071059>] btrfs_log_dentry_safe+0x49/0x70 [btrfs]
[ 3142.656017] [<ffffffffa0049406>] btrfs_sync_file+0x146/0x1d0 [btrfs]
[ 3142.656017] [<ffffffff81143e80>] ? fget_raw+0x210/0x210
[ 3142.656017] [<ffffffff81171029>] do_fsync+0x59/0x80
[ 3142.656017] [<ffffffff81171340>] sys_fsync+0x10/0x20
[ 3142.656017] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 3142.656017] Code: 60 ff ff ff 45 89 f1 48 8b bd 58 ff ff ff 4c 89 e1 4c 89 fe 89 14 24 48 8b 95 50 ff ff ff e8 44 cf ff ff 85 c0 0f 84 04 fe ff ff <0f> 0b 48 89 df e8 f2 62 fa ff 48 8b bd 50 ff ff ff e8 e6 62 fa
[ 3142.656017] RIP [<ffffffffa006fe74>] btrfs_log_inode+0x554/0x600 [btrfs]
[ 3142.656017] RSP <ffff88005c801d58>
[ 3142.995743] ---[ end trace 81d2bc6c70d7ca63 ]---
2923 static int btrfs_log_inode(struct btrfs_trans_handle *trans,
2924 struct btrfs_root *root, struct inode *inode,
2925 int inode_only)
2926 {
...
3105 out_unlock:
3106 BUG_ON(err);
3107 mutex_unlock(&BTRFS_I(inode)->log_mutex);
3108
3109 btrfs_free_path(path);
3110 btrfs_free_path(dst_path);
3111 return err;
3112 }
from the value of RAX, -17 it's EEXIST
the code originates from:
commit 3ff4357398886f277c278fd039ba65abf5e3a113
Author: Liu Bo <liubo2009@cn.fujitsu.com>
Date: Wed Oct 12 20:01:24 2011 -0400
Btrfs: improve log with sub transaction
When logging an inode _A_, current btrfs will
a) clear all items belonged to _A_ in log,
b) copy all items belonged to _A_ from fs/file tree to log tree,
and this just wastes a lot of time, especially when logging big files.
So we want to use a smarter approach, i.e. "find and merge".
The amount of file extent items is the largest, so we focus on it.
Thanks to sub transaction, now we can find those file extent items which
are changed after last _transaction commit_ or last _log commit_, and
then merge them with the existed ones in log tree.
It will be great helpful on fsync performance, cause the common case is
"make changes on a _part_ of inode".
Chris Mason did some reworking of this code, any bugs left are his.
Signed-off-by: Liu Bo <liubo2009@cn.fujitsu.com>
Signed-off-by: Chris Mason <chris.mason@oracle.com>
---
david
^ permalink raw reply
* Re: [PATCH 14/21] Btrfs: save restripe parameters to disk
From: Arne Jansen @ 2011-11-01 10:29 UTC (permalink / raw)
To: Ilya Dryomov; +Cc: linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <1314129722-31601-15-git-send-email-idryomov@gmail.com>
On 23.08.2011 22:01, Ilya Dryomov wrote:
> Introduce a new btree objectid for storing restripe item. The reason is
> to be able to resume restriper after a crash with the same parameters.
> Restripe item has a very high objectid and goes into tree of tree roots.
>
> The key for the new item is as follows:
>
> [ BTRFS_RESTRIPE_OBJECTID ; 0 ; 0 ]
>
> Older kernels simply ignore it so it's safe to mount with an older
> kernel and then go back to the newer one.
>
> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
> ---
> fs/btrfs/ctree.h | 127 +++++++++++++++++++++++++++++++++++++++++++++++++++-
> fs/btrfs/volumes.c | 105 ++++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 228 insertions(+), 4 deletions(-)
>
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 65d7562..b524034 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
> @@ -85,6 +85,9 @@ struct btrfs_ordered_sum;
> /* holds checksums of all the data extents */
> #define BTRFS_CSUM_TREE_OBJECTID 7ULL
>
> +/* for storing restripe params in the root tree */
> +#define BTRFS_RESTRIPE_OBJECTID -4ULL
> +
> /* orhpan objectid for tracking unlinked/truncated files */
> #define BTRFS_ORPHAN_OBJECTID -5ULL
>
> @@ -649,6 +652,47 @@ struct btrfs_root_ref {
> __le16 name_len;
> } __attribute__ ((__packed__));
>
> +/*
> + * Restriper stuff
> + */
> +struct btrfs_disk_restripe_args {
> + /* profiles to touch, in-memory format */
> + __le64 profiles;
> +
> + /* usage filter */
> + __le64 usage;
> +
> + /* devid filter */
> + __le64 devid;
> +
> + /* devid subset filter [pstart..pend) */
> + __le64 pstart;
> + __le64 pend;
> +
> + /* btrfs virtual address space subset filter [vstart..vend) */
> + __le64 vstart;
> + __le64 vend;
> +
> + /* profile to convert to, in-memory format */
> + __le64 target;
> +
> + /* BTRFS_RESTRIPE_ARGS_* */
> + __le64 flags;
> +
> + __le64 unused[8];
> +} __attribute__ ((__packed__));
> +
> +struct btrfs_restripe_item {
> + /* BTRFS_RESTRIPE_* */
> + __le64 flags;
> +
> + struct btrfs_disk_restripe_args data;
> + struct btrfs_disk_restripe_args sys;
> + struct btrfs_disk_restripe_args meta;
> +
> + __le64 unused[4];
> +} __attribute__ ((__packed__));
what are those unused fields for? As I understand it, the restripe_item
is only temporary and gets removed after restripe finished, so I don't
see much point in leaving space for future expansions. You have the size
of the struct anyway, or can determine which fields to access through the
flags field.
> +
> #define BTRFS_FILE_EXTENT_INLINE 0
> #define BTRFS_FILE_EXTENT_REG 1
> #define BTRFS_FILE_EXTENT_PREALLOC 2
> @@ -727,7 +771,8 @@ struct btrfs_csum_item {
> BTRFS_BLOCK_GROUP_RAID10)
> /*
> * We need a bit for restriper to be able to tell when chunks of type
> - * SINGLE are available. It is used in avail_*_alloc_bits.
> + * SINGLE are available. It is used in avail_*_alloc_bits and restripe
> + * item fields.
> */
> #define BTRFS_AVAIL_ALLOC_BIT_SINGLE (1 << 7)
>
> @@ -2000,8 +2045,86 @@ static inline bool btrfs_root_readonly(struct btrfs_root *root)
> return root->root_item.flags & BTRFS_ROOT_SUBVOL_RDONLY;
> }
>
> -/* struct btrfs_super_block */
> +/* struct btrfs_restripe_item */
> +BTRFS_SETGET_FUNCS(restripe_flags, struct btrfs_restripe_item, flags, 64);
> +
> +static inline void btrfs_restripe_data(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + read_eb_member(eb, ri, struct btrfs_restripe_item, data, ra);
> +}
>
> +static inline void btrfs_set_restripe_data(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + write_eb_member(eb, ri, struct btrfs_restripe_item, data, ra);
> +}
> +
> +static inline void btrfs_restripe_meta(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + read_eb_member(eb, ri, struct btrfs_restripe_item, meta, ra);
> +}
> +
> +static inline void btrfs_set_restripe_meta(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + write_eb_member(eb, ri, struct btrfs_restripe_item, meta, ra);
> +}
> +
> +static inline void btrfs_restripe_sys(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + read_eb_member(eb, ri, struct btrfs_restripe_item, sys, ra);
> +}
> +
> +static inline void btrfs_set_restripe_sys(struct extent_buffer *eb,
> + struct btrfs_restripe_item *ri,
> + struct btrfs_disk_restripe_args *ra)
> +{
> + write_eb_member(eb, ri, struct btrfs_restripe_item, sys, ra);
> +}
> +
> +static inline void
> +btrfs_disk_restripe_args_to_cpu(struct btrfs_restripe_args *cpu,
> + struct btrfs_disk_restripe_args *disk)
> +{
> + memset(cpu, 0, sizeof(*cpu));
> +
> + cpu->profiles = le64_to_cpu(disk->profiles);
> + cpu->usage = le64_to_cpu(disk->usage);
> + cpu->devid = le64_to_cpu(disk->devid);
> + cpu->pstart = le64_to_cpu(disk->pstart);
> + cpu->pend = le64_to_cpu(disk->pend);
> + cpu->vstart = le64_to_cpu(disk->vstart);
> + cpu->vend = le64_to_cpu(disk->vend);
> + cpu->target = le64_to_cpu(disk->target);
> + cpu->flags = le64_to_cpu(disk->flags);
> +}
> +
> +static inline void
> +btrfs_cpu_restripe_args_to_disk(struct btrfs_disk_restripe_args *disk,
> + struct btrfs_restripe_args *cpu)
> +{
> + memset(disk, 0, sizeof(*disk));
> +
> + disk->profiles = cpu_to_le64(cpu->profiles);
> + disk->usage = cpu_to_le64(cpu->usage);
> + disk->devid = cpu_to_le64(cpu->devid);
> + disk->pstart = cpu_to_le64(cpu->pstart);
> + disk->pend = cpu_to_le64(cpu->pend);
> + disk->vstart = cpu_to_le64(cpu->vstart);
> + disk->vend = cpu_to_le64(cpu->vend);
> + disk->target = cpu_to_le64(cpu->target);
> + disk->flags = cpu_to_le64(cpu->flags);
> +}
> +
> +/* struct btrfs_super_block */
> BTRFS_SETGET_STACK_FUNCS(super_bytenr, struct btrfs_super_block, bytenr, 64);
> BTRFS_SETGET_STACK_FUNCS(super_flags, struct btrfs_super_block, flags, 64);
> BTRFS_SETGET_STACK_FUNCS(super_generation, struct btrfs_super_block,
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index eccd458..1057ad3 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -2150,6 +2150,97 @@ error:
> return ret;
> }
>
> +static int insert_restripe_item(struct btrfs_root *root,
> + struct restripe_control *rctl)
> +{
> + struct btrfs_trans_handle *trans;
> + struct btrfs_restripe_item *item;
> + struct btrfs_disk_restripe_args disk_rargs;
> + struct btrfs_path *path;
> + struct extent_buffer *leaf;
> + struct btrfs_key key;
> + int ret, err;
> +
> + path = btrfs_alloc_path();
> + if (!path)
> + return -ENOMEM;
> +
> + trans = btrfs_start_transaction(root, 0);
> + if (IS_ERR(trans)) {
> + btrfs_free_path(path);
> + return PTR_ERR(trans);
> + }
> +
> + key.objectid = BTRFS_RESTRIPE_OBJECTID;
> + key.type = 0;
> + key.offset = 0;
> +
> + ret = btrfs_insert_empty_item(trans, root, path, &key,
> + sizeof(*item));
> + if (ret)
> + goto out;
> +
> + leaf = path->nodes[0];
> + item = btrfs_item_ptr(leaf, path->slots[0], struct btrfs_restripe_item);
> +
> + memset_extent_buffer(leaf, 0, (unsigned long)item, sizeof(*item));
> +
> + btrfs_cpu_restripe_args_to_disk(&disk_rargs, &rctl->data);
> + btrfs_set_restripe_data(leaf, item, &disk_rargs);
> + btrfs_cpu_restripe_args_to_disk(&disk_rargs, &rctl->meta);
> + btrfs_set_restripe_meta(leaf, item, &disk_rargs);
> + btrfs_cpu_restripe_args_to_disk(&disk_rargs, &rctl->sys);
> + btrfs_set_restripe_sys(leaf, item, &disk_rargs);
> +
> + btrfs_set_restripe_flags(leaf, item, rctl->flags);
> +
> + btrfs_mark_buffer_dirty(leaf);
> +out:
> + btrfs_free_path(path);
> + err = btrfs_commit_transaction(trans, root);
> + if (err && !ret)
> + ret = err;
> + return ret;
> +}
> +
> +static int del_restripe_item(struct btrfs_root *root)
> +{
> + struct btrfs_trans_handle *trans;
> + struct btrfs_path *path;
> + struct btrfs_key key;
> + int ret, err;
> +
> + path = btrfs_alloc_path();
> + if (!path)
> + return -ENOMEM;
> +
> + trans = btrfs_start_transaction(root, 0);
> + if (IS_ERR(trans)) {
> + btrfs_free_path(path);
> + return PTR_ERR(trans);
> + }
> +
> + key.objectid = BTRFS_RESTRIPE_OBJECTID;
> + key.type = 0;
> + key.offset = 0;
> +
> + ret = btrfs_search_slot(trans, root, &key, path, -1, 1);
> + if (ret < 0)
> + goto out;
> + if (ret > 0) {
> + ret = -ENOENT;
> + goto out;
> + }
> +
> + ret = btrfs_del_item(trans, root, path);
> +out:
> + btrfs_free_path(path);
> + err = btrfs_commit_transaction(trans, root);
> + if (err && !ret)
> + ret = err;
> + return ret;
> +}
> +
> /*
> * Should be called with both restripe and volume mutexes held to
> * serialize other volume operations (add_dev/rm_dev/resize) wrt
> @@ -2485,6 +2576,7 @@ int btrfs_restripe(struct restripe_control *rctl)
> {
> struct btrfs_fs_info *fs_info = rctl->fs_info;
> u64 allowed;
> + int err;
> int ret;
>
> mutex_lock(&fs_info->volume_mutex);
> @@ -2572,16 +2664,25 @@ int btrfs_restripe(struct restripe_control *rctl)
> }
>
> do_restripe:
> + ret = insert_restripe_item(fs_info->tree_root, rctl);
> + if (ret && ret != -EEXIST)
> + goto out;
> + BUG_ON(ret == -EEXIST);
> +
> set_restripe_control(rctl);
> mutex_unlock(&fs_info->volume_mutex);
>
> - ret = __btrfs_restripe(fs_info->dev_root);
> + err = __btrfs_restripe(fs_info->dev_root);
>
> mutex_lock(&fs_info->volume_mutex);
> +
> unset_restripe_control(fs_info);
> + ret = del_restripe_item(fs_info->tree_root);
> + BUG_ON(ret);
> +
> mutex_unlock(&fs_info->volume_mutex);
>
> - return ret;
> + return err;
>
> out:
> mutex_unlock(&fs_info->volume_mutex);
^ permalink raw reply
* Warnings: destroy inode (was: new integration-danger branch pushed out (kernel))
From: David Sterba @ 2011-11-01 10:28 UTC (permalink / raw)
To: Chris Mason, linux-btrfs
In-Reply-To: <20111101024957.GH4468@shiny>
this time I'm not quite sure which xfstest triggered it, but there was
209 35s ... [11:20:19] [11:20:53] [failed, exit status 1] - output mismatch (see 209.out.bad)
--- 209.out 2011-02-11 11:42:31.000000000 +0100
+++ 209.out.bad 2011-11-01 11:20:53.000000000 +0100
@@ -1,2 +1,2 @@
QA output created by 209
-test ran for 30 seconds without error
+reader found old byte at pos 7380992
\ No newline at end of file
[ 3134.165501] ------------[ cut here ]------------
[ 3134.171488] WARNING: at fs/btrfs/inode.c:6789 btrfs_destroy_inode+0x315/0x350 [btrfs]()
[ 3134.180812] Hardware name: Santa Rosa platform
[ 3134.180814] Modules linked in: loop btrfs aoe
[ 3134.180820] Pid: 26264, comm: rm Tainted: G W 3.1.0-default+ #64
[ 3134.180822] Call Trace:
[ 3134.180829] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 3134.180832] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 3134.180845] [<ffffffffa0048165>] btrfs_destroy_inode+0x315/0x350 [btrfs]
[ 3134.180850] [<ffffffff8115db3c>] destroy_inode+0x3c/0x70
[ 3134.180853] [<ffffffff8115dc8e>] evict+0x11e/0x1c0
[ 3134.180855] [<ffffffff8115de3d>] iput+0x10d/0x210
[ 3134.180859] [<ffffffff8115240f>] do_unlinkat+0x16f/0x1d0
[ 3134.180863] [<ffffffff81a15319>] ? retint_swapgs+0x13/0x1b
[ 3134.180867] [<ffffffff81097bd5>] ? trace_hardirqs_on_caller+0x155/0x1d0
[ 3134.180872] [<ffffffff8135c9be>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 3134.180875] [<ffffffff811532b2>] sys_unlinkat+0x22/0x40
[ 3134.180879] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 3134.180882] ---[ end trace 81d2bc6c70d7ca5c ]---
[ 3134.180883] ------------[ cut here ]------------
[ 3134.180895] WARNING: at fs/btrfs/inode.c:6790 btrfs_destroy_inode+0x2ff/0x350 [btrfs]()
[ 3134.180897] Hardware name: Santa Rosa platform
[ 3134.180899] Modules linked in: loop btrfs aoe
[ 3134.180903] Pid: 26264, comm: rm Tainted: G W 3.1.0-default+ #64
[ 3134.180905] Call Trace:
[ 3134.180908] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 3134.180911] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 3134.180923] [<ffffffffa004814f>] btrfs_destroy_inode+0x2ff/0x350 [btrfs]
[ 3134.180927] [<ffffffff8115db3c>] destroy_inode+0x3c/0x70
[ 3134.180929] [<ffffffff8115dc8e>] evict+0x11e/0x1c0
[ 3134.180933] [<ffffffff8115de3d>] iput+0x10d/0x210
[ 3134.180936] [<ffffffff8115240f>] do_unlinkat+0x16f/0x1d0
[ 3134.180939] [<ffffffff81a15319>] ? retint_swapgs+0x13/0x1b
[ 3134.180942] [<ffffffff81097bd5>] ? trace_hardirqs_on_caller+0x155/0x1d0
[ 3134.180945] [<ffffffff8135c9be>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 3134.180948] [<ffffffff811532b2>] sys_unlinkat+0x22/0x40
[ 3134.180952] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 3134.180954] ---[ end trace 81d2bc6c70d7ca5d ]---
[ 3134.180956] ------------[ cut here ]------------
[ 3134.180967] WARNING: at fs/btrfs/inode.c:6792 btrfs_destroy_inode+0x2d3/0x350 [btrfs]()
[ 3134.181008] Hardware name: Santa Rosa platform
[ 3134.181010] Modules linked in: loop btrfs aoe
[ 3134.181014] Pid: 26264, comm: rm Tainted: G W 3.1.0-default+ #64
[ 3134.181016] Call Trace:
[ 3134.181019] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 3134.181023] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 3134.181035] [<ffffffffa0048123>] btrfs_destroy_inode+0x2d3/0x350 [btrfs]
[ 3134.181038] [<ffffffff8115db3c>] destroy_inode+0x3c/0x70
[ 3134.181041] [<ffffffff8115dc8e>] evict+0x11e/0x1c0
[ 3134.181043] [<ffffffff8115de3d>] iput+0x10d/0x210
[ 3134.181046] [<ffffffff8115240f>] do_unlinkat+0x16f/0x1d0
[ 3134.181049] [<ffffffff81a15319>] ? retint_swapgs+0x13/0x1b
[ 3134.181052] [<ffffffff81097bd5>] ? trace_hardirqs_on_caller+0x155/0x1d0
[ 3134.181055] [<ffffffff8135c9be>] ? trace_hardirqs_on_thunk+0x3a/0x3f
[ 3134.181059] [<ffffffff811532b2>] sys_unlinkat+0x22/0x40
[ 3134.181062] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 3134.181064] ---[ end trace 81d2bc6c70d7ca5e ]---
[ 3135.152109] ------------[ cut here ]------------
6782 void btrfs_destroy_inode(struct inode *inode)
6783 {
6784 struct btrfs_ordered_extent *ordered;
6785 struct btrfs_root *root = BTRFS_I(inode)->root;
6786
6787 WARN_ON(!list_empty(&inode->i_dentry));
6788 WARN_ON(inode->i_data.nrpages);
6789 WARN_ON(BTRFS_I(inode)->outstanding_extents);
6790 WARN_ON(BTRFS_I(inode)->reserved_extents);
6791 WARN_ON(BTRFS_I(inode)->delalloc_bytes);
6792 WARN_ON(BTRFS_I(inode)->csum_bytes);
david
^ permalink raw reply
* Re: [PATCH 10/21] Btrfs: usage filter
From: Arne Jansen @ 2011-11-01 10:18 UTC (permalink / raw)
To: Ilya Dryomov; +Cc: linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <1314129722-31601-11-git-send-email-idryomov@gmail.com>
On 23.08.2011 22:01, Ilya Dryomov wrote:
> Select chunks that are less than X percent full.
>
> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
> ---
> fs/btrfs/volumes.c | 33 +++++++++++++++++++++++++++++++++
> fs/btrfs/volumes.h | 1 +
> 2 files changed, 34 insertions(+), 0 deletions(-)
>
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index f045615..b49ecfa 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -2193,6 +2193,33 @@ static int chunk_profiles_filter(u64 chunk_profile,
> return 1;
> }
>
> +static u64 div_factor_fine(u64 num, int factor)
> +{
> + if (factor == 100)
> + return num;
You already have changed this to a range check that always returns
num when <0 or >= 100, but I'd find it more consistent to return 0
when factor < 0.
> + num *= factor;
> + do_div(num, 100);
> + return num;
> +}
> +
> +static int chunk_usage_filter(struct btrfs_fs_info *fs_info, u64 chunk_offset,
> + struct btrfs_restripe_args *rargs)
> +{
> + struct btrfs_block_group_cache *cache;
> + u64 chunk_used, user_thresh;
> + int ret = 1;
> +
> + cache = btrfs_lookup_block_group(fs_info, chunk_offset);
> + chunk_used = btrfs_block_group_used(&cache->item);
> +
> + user_thresh = div_factor_fine(cache->key.offset, rargs->usage);
> + if (chunk_used < user_thresh)
> + ret = 0;
> +
> + btrfs_put_block_group(cache);
> + return ret;
> +}
> +
> static int chunk_soft_convert_filter(u64 chunk_profile,
> struct btrfs_restripe_args *rargs)
> {
> @@ -2236,6 +2263,12 @@ static int should_restripe_chunk(struct btrfs_root *root,
> return 0;
> }
>
> + /* usage filter */
> + if ((rargs->flags & BTRFS_RESTRIPE_ARGS_USAGE) &&
> + chunk_usage_filter(rctl->fs_info, chunk_offset, rargs)) {
> + return 0;
> + }
> +
> /* soft profile changing mode */
> if ((rargs->flags & BTRFS_RESTRIPE_ARGS_SOFT) &&
> chunk_soft_convert_filter(chunk_type, rargs)) {
> diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
> index 9f96ad8..c6baf4b 100644
> --- a/fs/btrfs/volumes.h
> +++ b/fs/btrfs/volumes.h
> @@ -186,6 +186,7 @@ struct map_lookup {
> * Restripe filters
> */
> #define BTRFS_RESTRIPE_ARGS_PROFILES (1ULL << 0)
> +#define BTRFS_RESTRIPE_ARGS_USAGE (1ULL << 1)
>
> /*
> * Profile changing flags. When SOFT is set we won't relocate chunk if
^ permalink raw reply
* Warnings (was: Re: new integration-danger branch pushed out (kernel))
From: David Sterba @ 2011-11-01 10:11 UTC (permalink / raw)
To: Chris Mason, linux-btrfs; +Cc: josef
In-Reply-To: <20111101024957.GH4468@shiny>
Hi,
On Mon, Oct 31, 2011 at 10:49:57PM -0400, Chris Mason wrote:
> I've pushed out a new integration branch, but it is not for general use.
Thank you!
xfstests/083 and 089 cause the following warnings. I've seen them before when
testing josef/for-chris and josef/master branches (reported on irc).
[ 1149.108088] ------------[ cut here ]------------
[ 1149.114384] WARNING: at fs/btrfs/extent-tree.c:3932 btrfs_free_block_groups+0x304/0x320 [btrfs]()
[ 1149.124928] Hardware name: Santa Rosa platform
[ 1149.124931] Modules linked in: loop btrfs aoe
[ 1149.124938] Pid: 23991, comm: umount Not tainted 3.1.0-default+ #64
[ 1149.124941] Call Trace:
[ 1149.124950] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1149.124953] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1149.124973] [<ffffffffa002bba4>] btrfs_free_block_groups+0x304/0x320 [btrfs]
[ 1149.124985] [<ffffffffa0036ab9>] close_ctree+0x1e9/0x390 [btrfs]
[ 1149.124991] [<ffffffff8115e2df>] ? dispose_list+0x4f/0x60
[ 1149.124999] [<ffffffffa00108dd>] btrfs_put_super+0x1d/0x30 [btrfs]
[ 1149.125003] [<ffffffff81145202>] generic_shutdown_super+0x62/0xe0
[ 1149.125006] [<ffffffff81145316>] kill_anon_super+0x16/0x30
[ 1149.125009] [<ffffffff811461a2>] ? deactivate_super+0x42/0x70
[ 1149.125012] [<ffffffff811456dd>] deactivate_locked_super+0x3d/0xa0
[ 1149.125015] [<ffffffff811461aa>] deactivate_super+0x4a/0x70
[ 1149.125019] [<ffffffff81162972>] mntput_no_expire+0xa2/0xf0
[ 1149.125022] [<ffffffff811636df>] sys_umount+0x6f/0x390
[ 1149.125027] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 1149.125030] ---[ end trace 81d2bc6c70d7ca56 ]---
[ 1149.125032] ------------[ cut here ]------------
[ 1149.125041] WARNING: at fs/btrfs/extent-tree.c:3933 btrfs_free_block_groups+0x2ee/0x320 [btrfs]()
[ 1149.125044] Hardware name: Santa Rosa platform
[ 1149.125045] Modules linked in: loop btrfs aoe
[ 1149.125049] Pid: 23991, comm: umount Tainted: G W 3.1.0-default+ #64
[ 1149.125051] Call Trace:
[ 1149.125054] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1149.125057] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1149.125067] [<ffffffffa002bb8e>] btrfs_free_block_groups+0x2ee/0x320 [btrfs]
[ 1149.125078] [<ffffffffa0036ab9>] close_ctree+0x1e9/0x390 [btrfs]
[ 1149.125081] [<ffffffff8115e2df>] ? dispose_list+0x4f/0x60
[ 1149.125089] [<ffffffffa00108dd>] btrfs_put_super+0x1d/0x30 [btrfs]
[ 1149.125092] [<ffffffff81145202>] generic_shutdown_super+0x62/0xe0
[ 1149.125096] [<ffffffff81145316>] kill_anon_super+0x16/0x30
[ 1149.125099] [<ffffffff811461a2>] ? deactivate_super+0x42/0x70
[ 1149.125102] [<ffffffff811456dd>] deactivate_locked_super+0x3d/0xa0
[ 1149.125105] [<ffffffff811461aa>] deactivate_super+0x4a/0x70
[ 1149.125108] [<ffffffff81162972>] mntput_no_expire+0xa2/0xf0
[ 1149.125111] [<ffffffff811636df>] sys_umount+0x6f/0x390
[ 1149.125115] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 1149.125117] ---[ end trace 81d2bc6c70d7ca57 ]---
The code in question:
3927 static void release_global_block_rsv(struct btrfs_fs_info *fs_info)
3928 {
3929 block_rsv_release_bytes(&fs_info->global_block_rsv, NULL, (u64)-1);
3930 WARN_ON(fs_info->delalloc_block_rsv.size > 0);
3931 WARN_ON(fs_info->delalloc_block_rsv.reserved > 0);
3932 WARN_ON(fs_info->trans_block_rsv.size > 0);
3933 WARN_ON(fs_info->trans_block_rsv.reserved > 0);
3934 WARN_ON(fs_info->chunk_block_rsv.size > 0);
3935 WARN_ON(fs_info->chunk_block_rsv.reserved > 0);
3936 }
[ 1149.125119] ------------[ cut here ]------------
[ 1149.125128] WARNING: at fs/btrfs/extent-tree.c:7090 btrfs_free_block_groups+0x23b/0x320 [btrfs]()
[ 1149.125130] Hardware name: Santa Rosa platform
[ 1149.125131] Modules linked in: loop btrfs aoe
[ 1149.125135] Pid: 23991, comm: umount Tainted: G W 3.1.0-default+ #64
[ 1149.125137] Call Trace:
[ 1149.125140] [<ffffffff81057fdf>] warn_slowpath_common+0x7f/0xc0
[ 1149.125143] [<ffffffff8105803a>] warn_slowpath_null+0x1a/0x20
[ 1149.125153] [<ffffffffa002badb>] btrfs_free_block_groups+0x23b/0x320 [btrfs]
[ 1149.125164] [<ffffffffa0036ab9>] close_ctree+0x1e9/0x390 [btrfs]
[ 1149.125167] [<ffffffff8115e2df>] ? dispose_list+0x4f/0x60
[ 1149.125175] [<ffffffffa00108dd>] btrfs_put_super+0x1d/0x30 [btrfs]
[ 1149.125178] [<ffffffff81145202>] generic_shutdown_super+0x62/0xe0
[ 1149.125181] [<ffffffff81145316>] kill_anon_super+0x16/0x30
[ 1149.125184] [<ffffffff811461a2>] ? deactivate_super+0x42/0x70
[ 1149.125187] [<ffffffff811456dd>] deactivate_locked_super+0x3d/0xa0
[ 1149.125191] [<ffffffff811461aa>] deactivate_super+0x4a/0x70
[ 1149.125194] [<ffffffff81162972>] mntput_no_expire+0xa2/0xf0
[ 1149.125197] [<ffffffff811636df>] sys_umount+0x6f/0x390
[ 1149.125200] [<ffffffff81a1d782>] system_call_fastpath+0x16/0x1b
[ 1149.125202] ---[ end trace 81d2bc6c70d7ca58 ]---
[ 1149.125206] space_info 5 has 4931584 free, is full
[ 1149.125208] space_info total=48234496, used=43302912, pinned=0, reserved=0, may_use=294912, readonly=0
7028 int btrfs_free_block_groups(struct btrfs_fs_info *info)
...
7083 while(!list_empty(&info->space_info)) {
7084 space_info = list_entry(info->space_info.next,
7085 struct btrfs_space_info,
7086 list);
7087 if (space_info->bytes_pinned > 0 ||
7088 space_info->bytes_reserved > 0 ||
7089 space_info->bytes_may_use > 0) {
7090 WARN_ON(1);
7091 dump_space_info(space_info, 0, 0);
7092 }
7093 list_del(&space_info->list);
7094 kfree(space_info);
7095 }
7096 return 0;
7097 }
david
^ permalink raw reply
* Re: [PATCH 05/21] Btrfs: add basic restriper infrastructure
From: Arne Jansen @ 2011-11-01 10:08 UTC (permalink / raw)
To: Ilya Dryomov; +Cc: linux-btrfs, Chris Mason, Hugo Mills
In-Reply-To: <1314129722-31601-6-git-send-email-idryomov@gmail.com>
On 23.08.2011 22:01, Ilya Dryomov wrote:
> Add basic restriper infrastructure: ioctl to start restripe, all
> restripe ioctl data structures, add data structure for tracking
> restriper's state to fs_info. Duplicate balancing code for restriper,
> btrfs_balance() will be removed when restriper is implemented.
>
> Explicitly disallow any volume operations when restriper is running.
> (previously this restriction relied on volume_mutex being held during
> the execution of any volume operation)
>
> Signed-off-by: Ilya Dryomov <idryomov@gmail.com>
> ---
> fs/btrfs/ctree.h | 5 +
> fs/btrfs/disk-io.c | 4 +
> fs/btrfs/ioctl.c | 107 ++++++++++++++++++++++----
> fs/btrfs/ioctl.h | 37 +++++++++
> fs/btrfs/volumes.c | 219 ++++++++++++++++++++++++++++++++++++++++++++++++++--
> fs/btrfs/volumes.h | 18 ++++
> 6 files changed, 369 insertions(+), 21 deletions(-)
>
> diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> index 5b00eb8..65d7562 100644
> --- a/fs/btrfs/ctree.h
> +++ b/fs/btrfs/ctree.h
> @@ -895,6 +895,7 @@ struct btrfs_block_group_cache {
> };
>
> struct reloc_control;
> +struct restripe_control;
> struct btrfs_device;
> struct btrfs_fs_devices;
> struct btrfs_delayed_root;
> @@ -1116,6 +1117,10 @@ struct btrfs_fs_info {
> u64 avail_metadata_alloc_bits;
> u64 avail_system_alloc_bits;
>
> + spinlock_t restripe_lock;
> + struct mutex restripe_mutex;
> + struct restripe_control *restripe_ctl;
> +
Can you please add some comments on the usage of the locks and
how to protect the restripe_ctl pointer and the access to its
data structures?
> unsigned data_chunk_allocations;
> unsigned metadata_ratio;
>
> diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
> index 46d0412..fa2301b 100644
> --- a/fs/btrfs/disk-io.c
> +++ b/fs/btrfs/disk-io.c
> @@ -1700,6 +1700,10 @@ struct btrfs_root *open_ctree(struct super_block *sb,
> init_rwsem(&fs_info->scrub_super_lock);
> fs_info->scrub_workers_refcnt = 0;
>
> + spin_lock_init(&fs_info->restripe_lock);
> + mutex_init(&fs_info->restripe_mutex);
> + fs_info->restripe_ctl = NULL;
> +
> sb->s_blocksize = 4096;
> sb->s_blocksize_bits = blksize_bits(4096);
> sb->s_bdi = &fs_info->bdi;
> diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
> index 970977a..9dfc686 100644
> --- a/fs/btrfs/ioctl.c
> +++ b/fs/btrfs/ioctl.c
> @@ -1165,13 +1165,21 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> if (!capable(CAP_SYS_ADMIN))
> return -EPERM;
>
> + mutex_lock(&root->fs_info->volume_mutex);
> + if (root->fs_info->restripe_ctl) {
> + printk(KERN_INFO "btrfs: restripe in progress\n");
> + ret = -EINVAL;
> + goto out;
> + }
> +
> vol_args = memdup_user(arg, sizeof(*vol_args));
> - if (IS_ERR(vol_args))
> - return PTR_ERR(vol_args);
> + if (IS_ERR(vol_args)) {
> + ret = PTR_ERR(vol_args);
> + goto out;
> + }
>
> vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
>
> - mutex_lock(&root->fs_info->volume_mutex);
> sizestr = vol_args->name;
> devstr = strchr(sizestr, ':');
> if (devstr) {
> @@ -1188,7 +1196,7 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> printk(KERN_INFO "resizer unable to find device %llu\n",
> (unsigned long long)devid);
> ret = -EINVAL;
> - goto out_unlock;
> + goto out_free;
> }
> if (!strcmp(sizestr, "max"))
> new_size = device->bdev->bd_inode->i_size;
> @@ -1203,7 +1211,7 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> new_size = memparse(sizestr, NULL);
> if (new_size == 0) {
> ret = -EINVAL;
> - goto out_unlock;
> + goto out_free;
> }
> }
>
> @@ -1212,7 +1220,7 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> if (mod < 0) {
> if (new_size > old_size) {
> ret = -EINVAL;
> - goto out_unlock;
> + goto out_free;
> }
> new_size = old_size - new_size;
> } else if (mod > 0) {
> @@ -1221,11 +1229,11 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
>
> if (new_size < 256 * 1024 * 1024) {
> ret = -EINVAL;
> - goto out_unlock;
> + goto out_free;
> }
> if (new_size > device->bdev->bd_inode->i_size) {
> ret = -EFBIG;
> - goto out_unlock;
> + goto out_free;
> }
>
> do_div(new_size, root->sectorsize);
> @@ -1238,7 +1246,7 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> trans = btrfs_start_transaction(root, 0);
> if (IS_ERR(trans)) {
> ret = PTR_ERR(trans);
> - goto out_unlock;
> + goto out_free;
> }
> ret = btrfs_grow_device(trans, device, new_size);
> btrfs_commit_transaction(trans, root);
> @@ -1246,9 +1254,10 @@ static noinline int btrfs_ioctl_resize(struct btrfs_root *root,
> ret = btrfs_shrink_device(device, new_size);
> }
>
> -out_unlock:
> - mutex_unlock(&root->fs_info->volume_mutex);
> +out_free:
> kfree(vol_args);
> +out:
> + mutex_unlock(&root->fs_info->volume_mutex);
> return ret;
> }
>
> @@ -2014,14 +2023,25 @@ static long btrfs_ioctl_add_dev(struct btrfs_root *root, void __user *arg)
> if (!capable(CAP_SYS_ADMIN))
> return -EPERM;
>
> + mutex_lock(&root->fs_info->volume_mutex);
> + if (root->fs_info->restripe_ctl) {
> + printk(KERN_INFO "btrfs: restripe in progress\n");
> + ret = -EINVAL;
> + goto out;
> + }
> +
> vol_args = memdup_user(arg, sizeof(*vol_args));
> - if (IS_ERR(vol_args))
> - return PTR_ERR(vol_args);
> + if (IS_ERR(vol_args)) {
> + ret = PTR_ERR(vol_args);
> + goto out;
> + }
>
> vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
> ret = btrfs_init_new_device(root, vol_args->name);
>
> kfree(vol_args);
> +out:
> + mutex_unlock(&root->fs_info->volume_mutex);
> return ret;
> }
>
> @@ -2036,14 +2056,25 @@ static long btrfs_ioctl_rm_dev(struct btrfs_root *root, void __user *arg)
> if (root->fs_info->sb->s_flags & MS_RDONLY)
> return -EROFS;
>
> + mutex_lock(&root->fs_info->volume_mutex);
> + if (root->fs_info->restripe_ctl) {
> + printk(KERN_INFO "btrfs: restripe in progress\n");
> + ret = -EINVAL;
> + goto out;
> + }
> +
> vol_args = memdup_user(arg, sizeof(*vol_args));
> - if (IS_ERR(vol_args))
> - return PTR_ERR(vol_args);
> + if (IS_ERR(vol_args)) {
> + ret = PTR_ERR(vol_args);
> + goto out;
> + }
>
> vol_args->name[BTRFS_PATH_NAME_MAX] = '\0';
> ret = btrfs_rm_device(root, vol_args->name);
>
> kfree(vol_args);
> +out:
> + mutex_unlock(&root->fs_info->volume_mutex);
> return ret;
> }
>
> @@ -2833,6 +2864,50 @@ static long btrfs_ioctl_scrub_progress(struct btrfs_root *root,
> return ret;
> }
>
> +static long btrfs_ioctl_restripe(struct btrfs_root *root, void __user *arg)
> +{
> + struct btrfs_ioctl_restripe_args *rargs;
> + struct btrfs_fs_info *fs_info = root->fs_info;
> + struct restripe_control *rctl;
> + int ret;
> +
> + if (!capable(CAP_SYS_ADMIN))
> + return -EPERM;
> +
> + if (fs_info->sb->s_flags & MS_RDONLY)
> + return -EROFS;
> +
> + mutex_lock(&fs_info->restripe_mutex);
> +
> + rargs = memdup_user(arg, sizeof(*rargs));
> + if (IS_ERR(rargs)) {
> + ret = PTR_ERR(rargs);
> + goto out;
> + }
> +
> + rctl = kzalloc(sizeof(*rctl), GFP_NOFS);
> + if (!rctl) {
> + kfree(rargs);
> + ret = -ENOMEM;
> + goto out;
> + }
> +
> + rctl->fs_info = fs_info;
> + rctl->flags = rargs->flags;
> +
> + memcpy(&rctl->data, &rargs->data, sizeof(rctl->data));
> + memcpy(&rctl->meta, &rargs->meta, sizeof(rctl->meta));
> + memcpy(&rctl->sys, &rargs->sys, sizeof(rctl->sys));
> +
> + ret = btrfs_restripe(rctl);
> +
> + /* rctl freed in unset_restripe_control */
> + kfree(rargs);
> +out:
> + mutex_unlock(&fs_info->restripe_mutex);
> + return ret;
> +}
> +
> long btrfs_ioctl(struct file *file, unsigned int
> cmd, unsigned long arg)
> {
> @@ -2905,6 +2980,8 @@ long btrfs_ioctl(struct file *file, unsigned int
> return btrfs_ioctl_scrub_cancel(root, argp);
> case BTRFS_IOC_SCRUB_PROGRESS:
> return btrfs_ioctl_scrub_progress(root, argp);
> + case BTRFS_IOC_RESTRIPE:
> + return btrfs_ioctl_restripe(root, argp);
> }
>
> return -ENOTTY;
> diff --git a/fs/btrfs/ioctl.h b/fs/btrfs/ioctl.h
> index ad1ea78..798f1d4 100644
> --- a/fs/btrfs/ioctl.h
> +++ b/fs/btrfs/ioctl.h
> @@ -109,6 +109,41 @@ struct btrfs_ioctl_fs_info_args {
> __u64 reserved[124]; /* pad to 1k */
> };
>
> +struct btrfs_restripe_args {
> + __u64 profiles;
> + __u64 usage;
> + __u64 devid;
> + __u64 pstart;
> + __u64 pend;
> + __u64 vstart;
> + __u64 vend;
> +
> + __u64 target;
> +
> + __u64 flags;
> +
> + __u64 unused[8];
> +} __attribute__ ((__packed__));
> +
> +struct btrfs_restripe_progress {
> + __u64 expected;
> + __u64 considered;
> + __u64 completed;
> +};
> +
> +struct btrfs_ioctl_restripe_args {
> + __u64 flags;
> + __u64 state;
> +
> + struct btrfs_restripe_args data;
> + struct btrfs_restripe_args sys;
> + struct btrfs_restripe_args meta;
> +
> + struct btrfs_restripe_progress stat;
> +
> + __u64 unused[72]; /* pad to 1k */
> +};
> +
> #define BTRFS_INO_LOOKUP_PATH_MAX 4080
> struct btrfs_ioctl_ino_lookup_args {
> __u64 treeid;
> @@ -248,4 +283,6 @@ struct btrfs_ioctl_space_args {
> struct btrfs_ioctl_dev_info_args)
> #define BTRFS_IOC_FS_INFO _IOR(BTRFS_IOCTL_MAGIC, 31, \
> struct btrfs_ioctl_fs_info_args)
> +#define BTRFS_IOC_RESTRIPE _IOW(BTRFS_IOCTL_MAGIC, 32, \
> + struct btrfs_ioctl_restripe_args)
> #endif
> diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
> index af4bf56..0e4a276 100644
> --- a/fs/btrfs/volumes.c
> +++ b/fs/btrfs/volumes.c
> @@ -1262,7 +1262,6 @@ int btrfs_rm_device(struct btrfs_root *root, char *device_path)
> bool clear_super = false;
>
> mutex_lock(&uuid_mutex);
> - mutex_lock(&root->fs_info->volume_mutex);
>
> all_avail = root->fs_info->avail_data_alloc_bits |
> root->fs_info->avail_system_alloc_bits |
> @@ -1427,7 +1426,6 @@ error_close:
> if (bdev)
> blkdev_put(bdev, FMODE_READ | FMODE_EXCL);
> out:
> - mutex_unlock(&root->fs_info->volume_mutex);
> mutex_unlock(&uuid_mutex);
> return ret;
> error_undo:
> @@ -1604,7 +1602,6 @@ int btrfs_init_new_device(struct btrfs_root *root, char *device_path)
> }
>
> filemap_write_and_wait(bdev->bd_inode->i_mapping);
> - mutex_lock(&root->fs_info->volume_mutex);
>
> devices = &root->fs_info->fs_devices->devices;
> /*
> @@ -1728,8 +1725,7 @@ int btrfs_init_new_device(struct btrfs_root *root, char *device_path)
> ret = btrfs_relocate_sys_chunks(root);
> BUG_ON(ret);
> }
> -out:
> - mutex_unlock(&root->fs_info->volume_mutex);
> +
> return ret;
> error:
> blkdev_put(bdev, FMODE_EXCL);
> @@ -1737,7 +1733,7 @@ error:
> mutex_unlock(&uuid_mutex);
> up_write(&sb->s_umount);
> }
> - goto out;
> + return ret;
> }
>
> static noinline int btrfs_update_device(struct btrfs_trans_handle *trans,
> @@ -2155,6 +2151,217 @@ error:
> }
>
> /*
> + * Should be called with both restripe and volume mutexes held to
> + * serialize other volume operations (add_dev/rm_dev/resize) wrt
> + * restriper. Same goes for unset_restripe_control().
> + */
> +static void set_restripe_control(struct restripe_control *rctl)
> +{
> + struct btrfs_fs_info *fs_info = rctl->fs_info;
> +
> + spin_lock(&fs_info->restripe_lock);
> + fs_info->restripe_ctl = rctl;
> + spin_unlock(&fs_info->restripe_lock);
> +}
> +
> +static void unset_restripe_control(struct btrfs_fs_info *fs_info)
> +{
> + struct restripe_control *rctl = fs_info->restripe_ctl;
> +
> + spin_lock(&fs_info->restripe_lock);
> + fs_info->restripe_ctl = NULL;
> + spin_unlock(&fs_info->restripe_lock);
> +
> + kfree(rctl);
> +}
> +
> +static int __btrfs_restripe(struct btrfs_root *dev_root)
> +{
> + struct list_head *devices;
> + struct btrfs_device *device;
> + u64 old_size;
> + u64 size_to_free;
> + struct btrfs_root *chunk_root = dev_root->fs_info->chunk_root;
> + struct btrfs_path *path;
> + struct btrfs_key key;
> + struct btrfs_key found_key;
> + struct btrfs_trans_handle *trans;
> + int ret;
> + int enospc_errors = 0;
> +
> + /* step one make some room on all the devices */
> + devices = &dev_root->fs_info->fs_devices->devices;
> + list_for_each_entry(device, devices, dev_list) {
> + old_size = device->total_bytes;
> + size_to_free = div_factor(old_size, 1);
> + size_to_free = min(size_to_free, (u64)1 * 1024 * 1024);
> + if (!device->writeable ||
> + device->total_bytes - device->bytes_used > size_to_free)
> + continue;
> +
> + ret = btrfs_shrink_device(device, old_size - size_to_free);
> + if (ret == -ENOSPC)
> + break;
> + BUG_ON(ret);
> +
> + trans = btrfs_start_transaction(dev_root, 0);
> + BUG_ON(IS_ERR(trans));
> +
> + ret = btrfs_grow_device(trans, device, old_size);
> + BUG_ON(ret);
> +
> + btrfs_end_transaction(trans, dev_root);
> + }
> +
> + /* step two, relocate all the chunks */
> + path = btrfs_alloc_path();
> + if (!path) {
> + ret = -ENOMEM;
> + goto error;
> + }
> +
> + key.objectid = BTRFS_FIRST_CHUNK_TREE_OBJECTID;
> + key.offset = (u64)-1;
> + key.type = BTRFS_CHUNK_ITEM_KEY;
> +
> + while (1) {
> + ret = btrfs_search_slot(NULL, chunk_root, &key, path, 0, 0);
> + if (ret < 0)
> + goto error;
> +
> + /*
> + * this shouldn't happen, it means the last relocate
> + * failed
> + */
> + if (ret == 0)
> + BUG_ON(1); /* DIS - break ? */
> +
> + ret = btrfs_previous_item(chunk_root, path, 0,
> + BTRFS_CHUNK_ITEM_KEY);
> + if (ret)
> + BUG_ON(1); /* DIS - break ? */
> +
> + btrfs_item_key_to_cpu(path->nodes[0], &found_key,
> + path->slots[0]);
> + if (found_key.objectid != key.objectid)
> + break;
> +
> + /* chunk zero is special */
> + if (found_key.offset == 0)
> + break;
> +
> + btrfs_release_path(path);
> + ret = btrfs_relocate_chunk(chunk_root,
> + chunk_root->root_key.objectid,
> + found_key.objectid,
> + found_key.offset);
> + if (ret && ret != -ENOSPC)
> + goto error;
> + if (ret == -ENOSPC)
> + enospc_errors++;
> + key.offset = found_key.offset - 1;
> + }
> +
> +error:
> + btrfs_free_path(path);
> + if (enospc_errors) {
> + printk(KERN_INFO "btrfs: restripe finished with %d enospc "
> + "error(s)\n", enospc_errors);
> + ret = -ENOSPC;
> + }
> +
> + return ret;
> +}
> +
> +/*
> + * Should be called with restripe_mutex held
> + */
> +int btrfs_restripe(struct restripe_control *rctl)
> +{
> + struct btrfs_fs_info *fs_info = rctl->fs_info;
> + u64 allowed;
> + int ret;
> +
> + mutex_lock(&fs_info->volume_mutex);
> +
> + /*
> + * Profile changing sanity checks
> + */
> + allowed = BTRFS_AVAIL_ALLOC_BIT_SINGLE;
> + if (fs_info->fs_devices->num_devices == 1)
> + allowed |= BTRFS_BLOCK_GROUP_DUP;
> + else if (fs_info->fs_devices->num_devices < 4)
> + allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1);
> + else
> + allowed |= (BTRFS_BLOCK_GROUP_RAID0 | BTRFS_BLOCK_GROUP_RAID1 |
> + BTRFS_BLOCK_GROUP_RAID10);
> +
> + if (rctl->data.target & ~allowed) {
> + printk(KERN_ERR "btrfs: unable to start restripe with target "
> + "data profile %llu\n",
> + (unsigned long long)rctl->data.target);
> + ret = -EINVAL;
> + goto out;
> + }
> + if (rctl->sys.target & ~allowed) {
> + printk(KERN_ERR "btrfs: unable to start restripe with target "
> + "system profile %llu\n",
> + (unsigned long long)rctl->sys.target);
> + ret = -EINVAL;
> + goto out;
> + }
> + if (rctl->meta.target & ~allowed) {
> + printk(KERN_ERR "btrfs: unable to start restripe with target "
> + "metadata profile %llu\n",
> + (unsigned long long)rctl->meta.target);
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + if (rctl->data.target & BTRFS_BLOCK_GROUP_DUP) {
> + printk(KERN_ERR "btrfs: dup for data is not allowed\n");
> + ret = -EINVAL;
> + goto out;
> + }
It would be good to get these error messages somehow to the user,
or at least give the user a hint to look in dmesg.
> +
> + /* allow to reduce meta or sys integrity only if force set */
> + allowed = BTRFS_BLOCK_GROUP_DUP | BTRFS_BLOCK_GROUP_RAID1 |
> + BTRFS_BLOCK_GROUP_RAID10;
> + if (((rctl->sys.flags & BTRFS_RESTRIPE_ARGS_CONVERT) &&
> + (fs_info->avail_system_alloc_bits & allowed) &&
> + !(rctl->sys.target & allowed)) ||
> + ((rctl->meta.flags & BTRFS_RESTRIPE_ARGS_CONVERT) &&
> + (fs_info->avail_metadata_alloc_bits & allowed) &&
> + !(rctl->meta.target & allowed))) {
> + if (rctl->flags & BTRFS_RESTRIPE_FORCE) {
> + printk(KERN_INFO "btrfs: force reducing metadata "
> + "integrity\n");
> + } else {
> + printk(KERN_ERR "btrfs: can't reduce metadata "
> + "integrity\n");
> + ret = -EINVAL;
> + goto out;
> + }
> + }
> +
> + set_restripe_control(rctl);
> + mutex_unlock(&fs_info->volume_mutex);
> +
> + ret = __btrfs_restripe(fs_info->dev_root);
> +
> + mutex_lock(&fs_info->volume_mutex);
> + unset_restripe_control(fs_info);
> + mutex_unlock(&fs_info->volume_mutex);
> +
> + return ret;
> +
> +out:
> + mutex_unlock(&fs_info->volume_mutex);
> + kfree(rctl);
> + return ret;
> +}
> +
> +/*
> * shrinking a device means finding all of the device extents past
> * the new size, and then following the back refs to the chunks.
> * The chunk relocation code actually frees the device extent
> diff --git a/fs/btrfs/volumes.h b/fs/btrfs/volumes.h
> index 6d866db..8804c5c 100644
> --- a/fs/btrfs/volumes.h
> +++ b/fs/btrfs/volumes.h
> @@ -168,6 +168,23 @@ struct map_lookup {
> #define map_lookup_size(n) (sizeof(struct map_lookup) + \
> (sizeof(struct btrfs_bio_stripe) * (n)))
>
> +#define BTRFS_RESTRIPE_FORCE (1ULL << 3)
> +
> +/*
> + * Profile changing flags
> + */
> +#define BTRFS_RESTRIPE_ARGS_CONVERT (1ULL << 8)
> +
> +struct btrfs_restripe_args;
> +struct restripe_control {
> + struct btrfs_fs_info *fs_info;
> + u64 flags;
> +
> + struct btrfs_restripe_args data;
> + struct btrfs_restripe_args sys;
> + struct btrfs_restripe_args meta;
> +};
> +
> int btrfs_account_dev_extents_size(struct btrfs_device *device, u64 start,
> u64 end, u64 *length);
>
> @@ -211,6 +228,7 @@ struct btrfs_device *btrfs_find_device(struct btrfs_root *root, u64 devid,
> int btrfs_shrink_device(struct btrfs_device *device, u64 new_size);
> int btrfs_init_new_device(struct btrfs_root *root, char *path);
> int btrfs_balance(struct btrfs_root *dev_root);
> +int btrfs_restripe(struct restripe_control *rctl);
> int btrfs_chunk_readonly(struct btrfs_root *root, u64 chunk_offset);
> int find_free_dev_extent(struct btrfs_trans_handle *trans,
> struct btrfs_device *device, u64 num_bytes,
^ permalink raw reply
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox