* Integration branch pushed out to btrfs-unstable
@ 2011-05-23 19:54 Chris Mason
2011-05-24 9:01 ` Arne Jansen
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: Chris Mason @ 2011-05-23 19:54 UTC (permalink / raw)
To: linux-btrfs; +Cc: Li Zefan, Miao Xie, Arne Jansen
Hi everyone,
I've pushed out my current kernel git tree to a new branch called
integration-test. This is meant for integration testing only and should
not be run by anyone who doesn't love crashes.
I've pulled together a lot of important work from a lot of different
people. It includes:
The new inode number allocator
Delayed inode and directory item updates
Scrub, chunk allocator fixes
Races in device addition and removal
Josef's performance fixes
A large series of cleanups and fixes
Most of the integration work was just pushing the cleanups into the new
code bits. I still want to integrate Hugo Mills' balance progress
ioctls (really nice). But it was a lot of work to get this all working
together and I wanted to let everyone retest the result.
Miao, I didn't include your patch to delay inode item insertion because
I was worried about interactions with the caching thread of the new
inode number allocator. I did have to make a small change to make sure
the inode-map cache didn't go through the delayed item routines.
Li, one thing I haven't done yet is a lot of benchmarking on the
performance hit from waiting for the cache to fill on large existing
filesystems. Was there any particular reason you didn't setup the
caching to pre-cache the from root->highest_objectid to
BTRFS_LAST_FREE_OBJECTID?
diffstat for my current tree:
-chris
]# git diff --stat v2.6.39...HEAD
fs/btrfs/Makefile | 2 +-
fs/btrfs/acl.c | 2 +-
fs/btrfs/btrfs_inode.h | 14 +
fs/btrfs/compression.c | 47 +-
fs/btrfs/compression.h | 2 +-
fs/btrfs/ctree.c | 51 +-
fs/btrfs/ctree.h | 199 ++---
fs/btrfs/delayed-inode.c | 1695 ++++++++++++++++++++++++++++++++++++
fs/btrfs/delayed-inode.h | 141 +++
fs/btrfs/delayed-ref.c | 114 ---
fs/btrfs/delayed-ref.h | 6 -
fs/btrfs/dir-item.c | 39 +-
fs/btrfs/disk-io.c | 202 +++---
fs/btrfs/disk-io.h | 19 +-
fs/btrfs/export.c | 25 +-
fs/btrfs/extent-tree.c | 2014 ++++---------------------------------------
fs/btrfs/extent_io.c | 327 ++------
fs/btrfs/extent_io.h | 40 +-
fs/btrfs/extent_map.c | 8 +-
fs/btrfs/extent_map.h | 4 +-
fs/btrfs/file-item.c | 38 +-
fs/btrfs/file.c | 45 +-
fs/btrfs/free-space-cache.c | 993 +++++++++++++---------
fs/btrfs/free-space-cache.h | 48 +-
fs/btrfs/inode-item.c | 2 -
fs/btrfs/inode-map.c | 428 +++++++++-
fs/btrfs/inode-map.h | 13 +
fs/btrfs/inode.c | 688 ++++++---------
fs/btrfs/ioctl.c | 176 ++++-
fs/btrfs/ioctl.h | 76 ++
fs/btrfs/locking.c | 25 -
fs/btrfs/locking.h | 2 -
fs/btrfs/ref-cache.c | 164 ----
fs/btrfs/ref-cache.h | 24 -
fs/btrfs/relocation.c | 67 +-
fs/btrfs/root-tree.c | 61 +--
fs/btrfs/scrub.c | 1368 +++++++++++++++++++++++++++++
fs/btrfs/super.c | 44 +-
fs/btrfs/sysfs.c | 77 --
fs/btrfs/transaction.c | 196 ++---
fs/btrfs/transaction.h | 5 +-
fs/btrfs/tree-defrag.c | 2 +-
fs/btrfs/tree-log.c | 208 +++--
fs/btrfs/tree-log.h | 1 -
fs/btrfs/version.sh | 43 -
fs/btrfs/volumes.c | 657 ++++++---------
fs/btrfs/volumes.h | 27 +-
fs/btrfs/xattr.c | 12 +-
48 files changed, 5902 insertions(+), 4539 deletions(-)
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Integration branch pushed out to btrfs-unstable
2011-05-23 19:54 Integration branch pushed out to btrfs-unstable Chris Mason
@ 2011-05-24 9:01 ` Arne Jansen
2011-05-24 9:50 ` Li Zefan
2011-05-25 12:35 ` Arne Jansen
2 siblings, 0 replies; 6+ messages in thread
From: Arne Jansen @ 2011-05-24 9:01 UTC (permalink / raw)
To: Chris Mason; +Cc: linux-btrfs, Li Zefan, Miao Xie
On 23.05.2011 21:54, Chris Mason wrote:
> I've pushed out my current kernel git tree to a new branch called
> integration-test. This is meant for integration testing only and should
> not be run by anyone who doesn't love crashes.
>
> Scrub, chunk allocator fixes
please note that in order to test scrub functionality you need an
updated btrfs utility. You can pull it from:
git://git.kernel.org/pub/scm/linux/kernel/git/arne/btrfs-progs-unstable-arne.git
scrub
Thanks,
Arne
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Integration branch pushed out to btrfs-unstable
2011-05-23 19:54 Integration branch pushed out to btrfs-unstable Chris Mason
2011-05-24 9:01 ` Arne Jansen
@ 2011-05-24 9:50 ` Li Zefan
2011-05-25 12:35 ` Arne Jansen
2 siblings, 0 replies; 6+ messages in thread
From: Li Zefan @ 2011-05-24 9:50 UTC (permalink / raw)
To: Chris Mason; +Cc: linux-btrfs, Miao Xie, Arne Jansen
Chris Mason wrote:
> Hi everyone,
>
> I've pushed out my current kernel git tree to a new branch called
> integration-test. This is meant for integration testing only and should
> not be run by anyone who doesn't love crashes.
>
> I've pulled together a lot of important work from a lot of different
> people. It includes:
>
> The new inode number allocator
> Delayed inode and directory item updates
> Scrub, chunk allocator fixes
> Races in device addition and removal
> Josef's performance fixes
> A large series of cleanups and fixes
>
> Most of the integration work was just pushing the cleanups into the new
> code bits. I still want to integrate Hugo Mills' balance progress
> ioctls (really nice). But it was a lot of work to get this all working
> together and I wanted to let everyone retest the result.
>
> Miao, I didn't include your patch to delay inode item insertion because
> I was worried about interactions with the caching thread of the new
> inode number allocator. I did have to make a small change to make sure
> the inode-map cache didn't go through the delayed item routines.
>
> Li, one thing I haven't done yet is a lot of benchmarking on the
> performance hit from waiting for the cache to fill on large existing
> filesystems. Was there any particular reason you didn't setup the
> caching to pre-cache the from root->highest_objectid to
> BTRFS_LAST_FREE_OBJECTID?
>
I'll cook up a patch for this kind of pre-cache.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Integration branch pushed out to btrfs-unstable
2011-05-23 19:54 Integration branch pushed out to btrfs-unstable Chris Mason
2011-05-24 9:01 ` Arne Jansen
2011-05-24 9:50 ` Li Zefan
@ 2011-05-25 12:35 ` Arne Jansen
2011-05-25 18:44 ` Sergei Trofimovich
2 siblings, 1 reply; 6+ messages in thread
From: Arne Jansen @ 2011-05-25 12:35 UTC (permalink / raw)
To: Chris Mason; +Cc: linux-btrfs, Li Zefan, Miao Xie, slyfox
On 23.05.2011 21:54, Chris Mason wrote:
> Hi everyone,
>
> I've pushed out my current kernel git tree to a new branch called
> integration-test. This is meant for integration testing only and should
> not be run by anyone who doesn't love crashes.
>
> I've pulled together a lot of important work from a lot of different
> people. It includes:
>
> The new inode number allocator
> Delayed inode and directory item updates
> Scrub, chunk allocator fixes
> Races in device addition and removal
> Josef's performance fixes
> A large series of cleanups and fixes
>
> Most of the integration work was just pushing the cleanups into the new
> code bits. I still want to integrate Hugo Mills' balance progress
> ioctls (really nice). But it was a lot of work to get this all working
> together and I wanted to let everyone retest the result.
>
The commit
commit c4f675cd40d955d539180506c09515c90169b15b
Author: Sergei Trofimovich <slyfox@gentoo.org>
Date: Fri May 20 20:20:30 2011 +0000
btrfs: don't spin in shrink_delalloc if there is nothing to free
gives me very early ENOSPC in my tests on a small volume (7GB).
-Arne
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: Integration branch pushed out to btrfs-unstable
2011-05-25 12:35 ` Arne Jansen
@ 2011-05-25 18:44 ` Sergei Trofimovich
2011-05-25 20:41 ` Sergei Trofimovich
0 siblings, 1 reply; 6+ messages in thread
From: Sergei Trofimovich @ 2011-05-25 18:44 UTC (permalink / raw)
To: Arne Jansen; +Cc: Chris Mason, linux-btrfs, Li Zefan, Miao Xie
[-- Attachment #1: Type: text/plain, Size: 513 bytes --]
> The commit
>
> commit c4f675cd40d955d539180506c09515c90169b15b
> Author: Sergei Trofimovich <slyfox@gentoo.org>
> Date: Fri May 20 20:20:30 2011 +0000
>
> btrfs: don't spin in shrink_delalloc if there is nothing to free
>
> gives me very early ENOSPC in my tests on a small volume (7GB).
What king of test is that? I'd like to reproduce it.
Does it do a lot of parallel I/O?
May I ask you to try to reproduce the problem with this sole patch
on top of linus master?
--
Sergei
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Integration branch pushed out to btrfs-unstable
2011-05-25 18:44 ` Sergei Trofimovich
@ 2011-05-25 20:41 ` Sergei Trofimovich
0 siblings, 0 replies; 6+ messages in thread
From: Sergei Trofimovich @ 2011-05-25 20:41 UTC (permalink / raw)
To: Arne Jansen; +Cc: Chris Mason, linux-btrfs, Li Zefan, Miao Xie
[-- Attachment #1: Type: text/plain, Size: 2593 bytes --]
> > The commit
> >
> > commit c4f675cd40d955d539180506c09515c90169b15b
> > Author: Sergei Trofimovich <slyfox@gentoo.org>
> > Date: Fri May 20 20:20:30 2011 +0000
> >
> > btrfs: don't spin in shrink_delalloc if there is nothing to free
> >
> > gives me very early ENOSPC in my tests on a small volume (7GB).
>
> What king of test is that? I'd like to reproduce it.
> Does it do a lot of parallel I/O?
>
> May I ask you to try to reproduce the problem with this sole patch
> on top of linus master?
With Arne's params to fs_mark it triggers for 2.6.39+this patch.
I took 2GB loop device and the script [1].
$ ./make_it.sh
...
# /home/slyfox/dev/fs_mark/fs_mark -D 512 -t 16 -n 4096 -s 51200 -L7 -S0 -d /mnt/ut
# Version 3.3, 16 thread(s) starting at Wed May 25 23:35:06 2011
# Sync method: NO SYNC: Test does not issue sync() or fsync() calls.
# Directories: Time based hash between directories across 512 subdirectories with 180 seconds per subdirectory.
# File names: 40 bytes long, (16 initial bytes of time stamp with 24 random bytes at end of name)
# Files info: size 51200 bytes, written with an IO size of 16384 bytes per write
# App overhead is time in microseconds spent in the test not doing file writing related system calls.
FSUse% Count Size Files/sec App Overhead
fs_mark: write_file write failed: -1 No space left on device
fs_mark: write_file write failed: -1 No space left on device
fs_mark: write_file write failed: -1 No space left on device
...^C
$ .../btrfs fi df /mnt/ut/
Data: total=622.25MB, used=434.61MB
System: total=4.00MB, used=4.00KB
Metadata: total=417.50MB, used=6.69MB
$ sudo .../btrfs fi show /dev/loop0
Label: none uuid: ccb18c12-4cbe-458c-835a-30e767939121
Total devices 1 FS bytes used 441.31MB
devid 1 size 2.00GB used 1.02GB path /dev/loop0
[1] 'make_it.sh' script exposing bug:
export LANG=C
FS_MARK=$HOME/dev/fs_mark/fs_mark
MKFS_BTRFS=$HOME/dev/git/btrfs-progs-unstable/mkfs.btrfs
sz=2
the_file=${sz}G.img
mount_point=/mnt/ut
dd if=/dev/zero of="${the_file}" bs=1 count=1 seek=$((${sz}*1024*1024*1024-1))
$MKFS_BTRFS -d raid0 -m raid0 "${the_file}"
sudo mount -oloop "${the_file}" "${mount_point}"
sudo chown slyfox:users "${mount_point}"
chmod u+w "${mount_point}"
cd /tmp
$FS_MARK \
-D 512 -t 16 -n 4096 -s 51200 -L7 -S0 \
-d "${mount_point}"
sudo umount "${mount_point}"
--
Sergei
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-05-25 20:41 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-05-23 19:54 Integration branch pushed out to btrfs-unstable Chris Mason
2011-05-24 9:01 ` Arne Jansen
2011-05-24 9:50 ` Li Zefan
2011-05-25 12:35 ` Arne Jansen
2011-05-25 18:44 ` Sergei Trofimovich
2011-05-25 20:41 ` Sergei Trofimovich
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).