Poor creat/delete files performance

Poor creat/delete files performance

[Miao Xie]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

Hi,

We did some performance test and found the create/delete files performance
of btrfs is very poor.

The test is that we create 50000 files and measure the file-create time
first, and then delete these 50000 files and measure the file-delete time. 
(The attached file is the reproduce program)

The result is following:
(Unit: second)
  Create file performance
		BtrFS		Ext4
  Total times:	2.462625	1.449550
  Average:	0.000049	0.000029

  Delete file performance
		BtrFS		Ext4
  Total times:	3.312796	0.997946
  Average:	0.000066	0.000020

The results were measured on a x86_64 server with 4 cores and 2 SAS disks.
By debuging, we found the btrfs spent a lot of time on searching and
inserting/removing items in the ctree.

Is anyone looking at this issue?

Regards
Miao Xie

[-- Attachment #2: creat_unlink.c --]
[-- Type: text/x-csrc, Size: 2818 bytes --]

/* gcc creat_unlink.c -o creat_unlink */
#include <stdio.h>
#include <stdlib.h>
#include <sys/time.h>

#define UNUSED          __attribute__ ((unused))
#define PATH_SIZE       100
#define TST_DIR		"tst_dir"

int initialize(void)
{
	int ret;

	ret = mkdir(TST_DIR, 0775);
	if (ret) {
		perror("init fail - mkdir failed\n");
		goto err;
	}

	ret = chdir(TST_DIR);
	if (ret) {
		perror("init fail - chdir failed\n");
		rmdir(TST_DIR);
	}
err:
	return ret;
}

int cleanup(void)
{
	int ret;

	ret = chdir("..");
	if (ret) {
		perror("cleanup fail - chdir failed\n");
		goto err;
	}

	ret = rmdir(TST_DIR);
	if (ret)
		perror("cleanup fail - rmdir failed\n");

err:
	return ret;
}

int get_start_time(struct timeval *tv)
{
	if (gettimeofday(tv, NULL)) {
		perror("get start time failed.\n");
		return 1;
	}

	return 0;
}

void account_time(struct timeval *stv, struct timeval *etv, int files)
{
	if (gettimeofday(etv, NULL)) {
		perror("get end time failed.\n");
	} else if (files) {
		double ts;

		while (etv->tv_usec < stv->tv_usec) {
			etv->tv_sec--;
			etv->tv_usec += 1000000;
		}

		etv->tv_usec -= stv->tv_usec;
		etv->tv_sec -= stv->tv_sec;

		while (etv->tv_usec > 1000000) {
			etv->tv_usec -= 1000000;
			etv->tv_sec++;
		}

		ts = (double)etv->tv_usec / (double)1000000 + etv->tv_sec;
		printf("\tTotal files: %d\n", files);
		printf("\tTotal time: %lf\n", ts);

		ts /= files;
		printf("\tAverage time: %lf\n", ts);
	} else
		perror("Didn't create/delete any files");

}

int create_files(int nfiles)
{
	int i, fd;
	char fpath[PATH_SIZE];
	struct timeval stv, etv;

	printf("Create files:\n");
	if (get_start_time(&stv))
		return 0;

	for (i = 0; i < nfiles; i++) {
		sprintf(fpath, "%d", i);
		fd = creat(fpath, 0555);
		if (fd < 0) {
			perror("creat file failed.\n");
			break;
		}
		close(fd);
	}

	account_time(&stv, &etv, i);

	return i;
}

int unlink_files(int nfiles)
{
	int i, ret = 0;
	char fpath[PATH_SIZE];
	struct timeval stv, etv;

	printf("Delete files:\n");
	if (get_start_time(&stv))
		return 1;

	for (i = 0; i < nfiles; i++) {
		sprintf(fpath, "%d", i);
		ret = unlink(fpath);
		if (ret)
			break;
	}

	account_time(&stv, &etv, i);

	return ret;
}

int main(int argc, char **argv)
{
	int nfiles, n_done, ret;

	/* parse the options */
	if (argc != 2) {
		fprintf(stderr, "options is wrong.\n");
		fprintf(stderr, "%s [nfiles]\n", argv[0]);
		exit(1);
	}

	nfiles = atoi(argv[1]);
	if (nfiles <= 0) {
		fprintf(stderr, "option nfiles is wrong.");
		exit(1);
	}

	/* initialize - create the test directory and change the current dir */
	ret = initialize();
	if (ret)
		exit(1);

	n_done = create_files(nfiles);
	if (n_done != nfiles) {
		unlink_files(n_done);
		cleanup();
		exit(1);
	}

	ret = unlink_files(n_done);
	if (ret)
		exit(1);

	ret = cleanup();
	if (ret)
		exit(1);

	return 0;
}

RE: Poor creat/delete files performance

[Morten P.D. Stevens]

2010/8/18 Miao Xie <miaox@cn.fujitsu.com>:
> Hi,
>
> We did some performance test and found the create/delete files perfor=
mance
> of btrfs is very poor.
>
> The test is that we create 50000 files and measure the file-create ti=
me
> first, and then delete these 50000 files and measure the file-delete =
time.
> (The attached file is the reproduce program)
>
> The result is following:
> (Unit: second)
> =A0Create file performance
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BtrFS =A0 =A0 =A0 =A0 =A0 Ext4
> =A0Total times: =A02.462625 =A0 =A0 =A0 =A01.449550
> =A0Average: =A0 =A0 =A00.000049 =A0 =A0 =A0 =A00.000029
>
> =A0Delete file performance
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BtrFS =A0 =A0 =A0 =A0 =A0 Ext4
> =A0Total times: =A03.312796 =A0 =A0 =A0 =A00.997946
> =A0Average: =A0 =A0 =A00.000066 =A0 =A0 =A0 =A00.000020
>
> The results were measured on a x86_64 server with 4 cores and 2 SAS d=
isks.
> By debuging, we found the btrfs spent a lot of time on searching and
> inserting/removing items in the ctree.
>
> Is anyone looking at this issue?
>
> Regards
> Miao Xie

Hi,

i can confirm this issue with poor create and delete performance.

=46or example with unpacking and deleting the linux kernel:

Btrfs:

[root@fc13 btrfs]# time tar xfj linux-2.6.36-rc1.tar.bz2

real    0m18.794s
user    0m12.045s
sys     0m8.241s
[root@fc13 btrfs]# time rm -rf linux-2.6.36-rc1

real    0m2.156s
user    0m0.028s
sys     0m2.118s

ext4:

[root@fc13 ext4]# time tar xfj linux-2.6.36-rc1.tar.bz2

real    0m13.140s
user    0m9.074s
sys     0m6.327s
[root@fc13 ext4]# time rm -rf linux-2.6.36-rc1

real    0m0.781s
user    0m0.031s
sys     0m0.735s

In all of our benchmarks, ext4 is still faster than btrfs.

Best regards,

Morten

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Poor creat/delete files performance

[Leonidas Spyropoulos]

Have you tried umounting and mounting before the second test to
eliminate any caching?
Which kernel you use?

2010/8/18 Miao Xie <miaox@cn.fujitsu.com>:
> Hi,
>
> We did some performance test and found the create/delete files perfor=
mance
> of btrfs is very poor.
>
> The test is that we create 50000 files and measure the file-create ti=
me
> first, and then delete these 50000 files and measure the file-delete =
time.
> (The attached file is the reproduce program)
>
> The result is following:
> (Unit: second)
> =A0Create file performance
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BtrFS =A0 =A0 =A0 =A0 =A0 Ext4
> =A0Total times: =A02.462625 =A0 =A0 =A0 =A01.449550
> =A0Average: =A0 =A0 =A00.000049 =A0 =A0 =A0 =A00.000029
>
> =A0Delete file performance
> =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0BtrFS =A0 =A0 =A0 =A0 =A0 Ext4
> =A0Total times: =A03.312796 =A0 =A0 =A0 =A00.997946
> =A0Average: =A0 =A0 =A00.000066 =A0 =A0 =A0 =A00.000020
>
> The results were measured on a x86_64 server with 4 cores and 2 SAS d=
isks.
> By debuging, we found the btrfs spent a lot of time on searching and
> inserting/removing items in the ctree.
>
> Is anyone looking at this issue?
>
> Regards
> Miao Xie
>



--=20
Caution: breathing may be hazardous to your health.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Poor creat/delete files performance

[Miao Xie]

On Wed, 18 Aug 2010 11:49:16 +0100, Leonidas Spyropoulos wrote:
> Have you tried umounting and mounting before the second test to
> eliminate any caching?

Yes, I have done it.
The result is similar to the one I have reported.
(Unit: second)
   Create file performance
		BtrFS		Ext4
   Total times:	2.484392	1.505082
   Average:	0.000050	0.000030

   Delete file performance
		BtrFS		Ext4
   Total times:	3.369469	1.024886
   Average:	0.000067	0.000020


> Which kernel you use?

v2.6.35

Regards
Miao Xie

>
> 2010/8/18 Miao Xie<miaox@cn.fujitsu.com>:
>> Hi,
>>
>> We did some performance test and found the create/delete files performance
>> of btrfs is very poor.
>>
>> The test is that we create 50000 files and measure the file-create time
>> first, and then delete these 50000 files and measure the file-delete time.
>> (The attached file is the reproduce program)
>>
>> The result is following:
>> (Unit: second)
>>   Create file performance
>>                 BtrFS           Ext4
>>   Total times:  2.462625        1.449550
>>   Average:      0.000049        0.000029
>>
>>   Delete file performance
>>                 BtrFS           Ext4
>>   Total times:  3.312796        0.997946
>>   Average:      0.000066        0.000020
>>
>> The results were measured on a x86_64 server with 4 cores and 2 SAS disks.
>> By debuging, we found the btrfs spent a lot of time on searching and
>> inserting/removing items in the ctree.
>>
>> Is anyone looking at this issue?
>>
>> Regards
>> Miao Xie
>>
>
>
>

Re: Poor creat/delete files performance

[Chris Mason]

On Wed, Aug 18, 2010 at 06:12:46PM +0800, Miao Xie wrote:
> Hi,
> 
> We did some performance test and found the create/delete files performance
> of btrfs is very poor.
> 
> The test is that we create 50000 files and measure the file-create time
> first, and then delete these 50000 files and measure the file-delete time. 
> (The attached file is the reproduce program)
> 
> The result is following:
> (Unit: second)
>   Create file performance
> 		BtrFS		Ext4
>   Total times:	2.462625	1.449550
>   Average:	0.000049	0.000029
> 
>   Delete file performance
> 		BtrFS		Ext4
>   Total times:	3.312796	0.997946
>   Average:	0.000066	0.000020
> 
> The results were measured on a x86_64 server with 4 cores and 2 SAS disks.
> By debuging, we found the btrfs spent a lot of time on searching and
> inserting/removing items in the ctree.
> 
> Is anyone looking at this issue?

I'm looking at it now, which kernel were you on?  We do spend some CPU
time on the btree but it shouldn't be a big bottleneck compared to the
disk.

This should be very similar to the performance regression that Steve
Pratt hit.  Our deletion time has always been slower than ext4, but the
creation time should be the same or faster.

-chris

Re: Poor creat/delete files performance

[Chris Mason]

On Wed, Aug 18, 2010 at 12:49:05PM +0200, Morten P.D. Stevens wrote:
> Hi,
> 
> i can confirm this issue with poor create and delete performance.
> 
> For example with unpacking and deleting the linux kernel:
> 
> Btrfs:
> 
> [root@fc13 btrfs]# time tar xfj linux-2.6.36-rc1.tar.bz2
> 
> real    0m18.794s
> user    0m12.045s
> sys     0m8.241s
> [root@fc13 btrfs]# time rm -rf linux-2.6.36-rc1
> 
> real    0m2.156s
> user    0m0.028s
> sys     0m2.118s
> 
> ext4:
> 
> [root@fc13 ext4]# time tar xfj linux-2.6.36-rc1.tar.bz2
> 
> real    0m13.140s
> user    0m9.074s
> sys     0m6.327s
> [root@fc13 ext4]# time rm -rf linux-2.6.36-rc1
> 
> real    0m0.781s
> user    0m0.031s
> sys     0m0.735s
> 
> In all of our benchmarks, ext4 is still faster than btrfs.

For this kind of benchmark, I use my compilebench program.  We want to
time just the file creation and not how long it takes to uncompress or
read in the .tar.gz from the disk.

The results from 2.6.35 on a single sata drive are not good for ext4 at
all.  Btrfs is on par with where it always has been, but I'm
experimenting with other benchmarks to find the regression.  Basically
when you are benchmarking this kind of thing you want to make sure
you're benchmarking how quickly you can write to the disk, and not to
cache.

compilebench simulates this by creating files with the same size and
names as what you get from untarring kernel trees, and then it has a
phase where it creates files with the same size and names as the .o
files from compiling the kernel.

Btrfs:

# ./compilebench -i 30 --makej
creating /mnt/default
using working directory /mnt/default, 30 intial dirs 100 runs
native unpatched native-0 222MB in 1.76 seconds (126.35 MB/s)
native patched native-0 109MB in 0.51 seconds (215.04 MB/s)
native patched compiled native-0 691MB in 5.69 seconds (121.55 MB/s)
create dir kernel-0 222MB in 2.94 seconds (75.64 MB/s)
create dir kernel-1 222MB in 3.16 seconds (70.37 MB/s)
create dir kernel-2 222MB in 3.05 seconds (72.91 MB/s)
create dir kernel-3 222MB in 3.11 seconds (71.50 MB/s)
create dir kernel-4 222MB in 3.88 seconds (57.31 MB/s)
create dir kernel-5 222MB in 3.58 seconds (62.12 MB/s)
create dir kernel-6 222MB in 6.11 seconds (36.40 MB/s)
create dir kernel-7 222MB in 2.28 seconds (97.53 MB/s)
create dir kernel-8 222MB in 3.12 seconds (71.27 MB/s)
create dir kernel-9 222MB in 2.86 seconds (77.75 MB/s)
create dir kernel-10 222MB in 4.46 seconds (49.86 MB/s)
create dir kernel-11 222MB in 2.96 seconds (75.13 MB/s)
create dir kernel-12 222MB in 3.86 seconds (57.61 MB/s)
create dir kernel-13 222MB in 3.96 seconds (56.16 MB/s)
create dir kernel-14 222MB in 3.79 seconds (58.67 MB/s)
create dir kernel-15 222MB in 6.58 seconds (33.80 MB/s)
create dir kernel-16 222MB in 2.09 seconds (106.40 MB/s)
create dir kernel-17 222MB in 2.71 seconds (82.06 MB/s)
create dir kernel-18 222MB in 3.40 seconds (65.40 MB/s)
create dir kernel-19 222MB in 3.25 seconds (68.42 MB/s)
create dir kernel-20 222MB in 3.79 seconds (58.67 MB/s)
create dir kernel-21 222MB in 3.86 seconds (57.61 MB/s)
create dir kernel-22 222MB in 3.59 seconds (61.94 MB/s)
create dir kernel-23 222MB in 4.02 seconds (55.32 MB/s)
create dir kernel-24 222MB in 7.68 seconds (28.95 MB/s)
create dir kernel-25 222MB in 1.99 seconds (111.75 MB/s)
create dir kernel-26 222MB in 2.77 seconds (80.28 MB/s)
create dir kernel-27 222MB in 2.97 seconds (74.87 MB/s)
create dir kernel-28 222MB in 3.22 seconds (69.06 MB/s)
create dir kernel-29 222MB in 3.94 seconds (56.44 MB/s)
compile dir kernel-6 680MB in 3.78 seconds (180.06 MB/s)
compile dir kernel-29 680MB in 9.21 seconds (73.90 MB/s)
compile dir kernel-11 680MB in 9.38 seconds (72.56 MB/s)
compile dir kernel-22 680MB in 10.57 seconds (64.39 MB/s)
compile dir kernel-15 680MB in 7.68 seconds (88.62 MB/s)
compile dir kernel-13 680MB in 9.26 seconds (73.50 MB/s)
compile dir kernel-17 680MB in 9.44 seconds (72.10 MB/s)
compile dir kernel-3 680MB in 9.03 seconds (75.37 MB/s)
compile dir kernel-10 680MB in 9.30 seconds (73.19 MB/s)
compile dir kernel-12 680MB in 9.29 seconds (73.27 MB/s)
compile dir kernel-27 680MB in 10.84 seconds (62.79 MB/s)
compile dir kernel-14 680MB in 7.65 seconds (88.97 MB/s)
compile dir kernel-9 680MB in 9.55 seconds (71.27 MB/s)
compile dir kernel-4 680MB in 9.20 seconds (73.98 MB/s)
compile dir kernel-0 680MB in 12.06 seconds (56.44 MB/s)
compile dir kernel-23 680MB in 9.39 seconds (72.49 MB/s)
compile dir kernel-1 680MB in 9.06 seconds (75.13 MB/s)

Ext4:

./compilebench -i 30 --makej
creating /mnt/default
using working directory /mnt/default, 30 intial dirs 100 runs
native unpatched native-0 222MB in 1.11 seconds (200.34 MB/s)
native patched native-0 109MB in 0.33 seconds (332.33 MB/s)
native patched compiled native-0 691MB in 5.38 seconds (128.55 MB/s)
create dir kernel-0 222MB in 1.05 seconds (211.78 MB/s)
create dir kernel-1 222MB in 4.60 seconds (48.34 MB/s)
create dir kernel-2 222MB in 4.30 seconds (51.71 MB/s)
create dir kernel-3 222MB in 5.71 seconds (38.94 MB/s)
create dir kernel-4 222MB in 4.82 seconds (46.14 MB/s)
create dir kernel-5 222MB in 4.91 seconds (45.29 MB/s)
create dir kernel-6 222MB in 5.70 seconds (39.01 MB/s)
create dir kernel-7 222MB in 4.76 seconds (46.72 MB/s)
create dir kernel-8 222MB in 6.40 seconds (34.75 MB/s)
create dir kernel-9 222MB in 5.37 seconds (41.41 MB/s)
create dir kernel-10 222MB in 5.51 seconds (40.36 MB/s)
create dir kernel-11 222MB in 6.77 seconds (32.85 MB/s)
create dir kernel-12 222MB in 5.88 seconds (37.82 MB/s)
create dir kernel-13 222MB in 8.92 seconds (24.93 MB/s)
create dir kernel-14 222MB in 5.44 seconds (40.88 MB/s)
create dir kernel-15 222MB in 6.21 seconds (35.81 MB/s)
create dir kernel-16 222MB in 4.85 seconds (45.85 MB/s)
create dir kernel-17 222MB in 8.83 seconds (25.18 MB/s)
create dir kernel-18 222MB in 6.82 seconds (32.61 MB/s)
create dir kernel-19 222MB in 4.86 seconds (45.76 MB/s)
create dir kernel-20 222MB in 10.48 seconds (21.22 MB/s)
create dir kernel-21 222MB in 4.85 seconds (45.85 MB/s)
create dir kernel-22 222MB in 8.30 seconds (26.79 MB/s)
create dir kernel-23 222MB in 6.38 seconds (34.85 MB/s)
create dir kernel-24 222MB in 7.30 seconds (30.46 MB/s)
create dir kernel-25 222MB in 3.83 seconds (58.06 MB/s)
create dir kernel-26 222MB in 9.70 seconds (22.93 MB/s)
create dir kernel-27 222MB in 5.39 seconds (41.26 MB/s)
create dir kernel-28 222MB in 6.96 seconds (31.95 MB/s)
create dir kernel-29 222MB in 8.16 seconds (27.25 MB/s)
compile dir kernel-6 680MB in 7.47 seconds (91.12 MB/s)
compile dir kernel-29 680MB in 16.37 seconds (41.58 MB/s)
compile dir kernel-11 680MB in 19.57 seconds (34.78 MB/s)
compile dir kernel-22 680MB in 17.17 seconds (39.64 MB/s)
compile dir kernel-15 680MB in 17.40 seconds (39.12 MB/s)
compile dir kernel-13 680MB in 17.87 seconds (38.09 MB/s)
compile dir kernel-17 680MB in 18.14 seconds (37.52 MB/s)
compile dir kernel-3 680MB in 17.43 seconds (39.05 MB/s)
compile dir kernel-10 680MB in 15.18 seconds (44.84 MB/s)
compile dir kernel-12 680MB in 17.14 seconds (39.71 MB/s)
compile dir kernel-27 680MB in 18.71 seconds (36.38 MB/s)
compile dir kernel-14 680MB in 17.47 seconds (38.96 MB/s)
compile dir kernel-9 680MB in 19.76 seconds (34.45 MB/s)
compile dir kernel-4 680MB in 17.53 seconds (38.83 MB/s)
compile dir kernel-0 680MB in 17.20 seconds (39.57 MB/s)
compile dir kernel-23 680MB in 17.31 seconds (39.32 MB/s)
compile dir kernel-1 680MB in 18.19 seconds (37.42 MB/s)

RE: Poor creat/delete files performance

[Morten P.D. Stevens]

Hi Chris,

Your compilebench results are very interesting.

Here are my results with exactly the same benchmark:

Btrfs:

[root@fc13 compilebench-0.6]# ./compilebench -D /mnt/btrfs -i 30 --makej
using working directory /mnt/btrfs, 30 intial dirs 100 runs
native unpatched native-0 222MB in 3.14 seconds (70.82 MB/s)
native patched native-0 109MB in 0.69 seconds (158.94 MB/s)
native patched compiled native-0 691MB in 1.86 seconds (371.83 MB/s)
create dir kernel-0 222MB in 2.28 seconds (97.53 MB/s)
create dir kernel-1 222MB in 3.45 seconds (64.46 MB/s)
create dir kernel-2 222MB in 5.37 seconds (41.41 MB/s)
create dir kernel-3 222MB in 3.37 seconds (65.99 MB/s)
create dir kernel-4 222MB in 3.33 seconds (66.78 MB/s)
create dir kernel-5 222MB in 3.56 seconds (62.46 MB/s)
create dir kernel-6 222MB in 3.39 seconds (65.60 MB/s)
create dir kernel-7 222MB in 3.76 seconds (59.14 MB/s)
create dir kernel-8 222MB in 3.52 seconds (63.17 MB/s)
create dir kernel-9 222MB in 3.62 seconds (61.43 MB/s)
create dir kernel-10 222MB in 3.48 seconds (63.90 MB/s)
create dir kernel-11 222MB in 3.32 seconds (66.98 MB/s)
create dir kernel-12 222MB in 3.06 seconds (72.67 MB/s)
create dir kernel-13 222MB in 2.64 seconds (84.23 MB/s)
create dir kernel-14 222MB in 2.91 seconds (76.42 MB/s)
create dir kernel-15 222MB in 3.25 seconds (68.42 MB/s)
create dir kernel-16 222MB in 3.00 seconds (74.12 MB/s)
create dir kernel-17 222MB in 3.14 seconds (70.82 MB/s)
create dir kernel-18 222MB in 4.90 seconds (45.38 MB/s)
create dir kernel-19 222MB in 3.14 seconds (70.82 MB/s)
create dir kernel-20 222MB in 3.26 seconds (68.21 MB/s)
create dir kernel-21 222MB in 3.10 seconds (71.73 MB/s)
create dir kernel-22 222MB in 3.18 seconds (69.93 MB/s)
create dir kernel-23 222MB in 2.97 seconds (74.87 MB/s)
create dir kernel-24 222MB in 2.69 seconds (82.67 MB/s)
create dir kernel-25 222MB in 2.90 seconds (76.68 MB/s)
create dir kernel-26 222MB in 2.96 seconds (75.13 MB/s)
create dir kernel-27 222MB in 2.94 seconds (75.64 MB/s)
create dir kernel-28 222MB in 2.86 seconds (77.75 MB/s)
create dir kernel-29 222MB in 3.09 seconds (71.97 MB/s)
compile dir kernel-6 680MB in 1.35 seconds (504.17 MB/s)
compile dir kernel-29 680MB in 2.93 seconds (232.30 MB/s)
compile dir kernel-11 680MB in 3.24 seconds (210.07 MB/s)
compile dir kernel-22 680MB in 3.12 seconds (218.15 MB/s)
compile dir kernel-15 680MB in 3.25 seconds (209.43 MB/s)
compile dir kernel-13 680MB in 3.23 seconds (210.72 MB/s)
compile dir kernel-17 680MB in 3.19 seconds (213.36 MB/s)
compile dir kernel-3 680MB in 3.35 seconds (203.17 MB/s)
compile dir kernel-10 680MB in 3.25 seconds (209.43 MB/s)
compile dir kernel-12 680MB in 3.59 seconds (189.59 MB/s)
compile dir kernel-27 680MB in 4.05 seconds (168.06 MB/s)
compile dir kernel-14 680MB in 3.29 seconds (206.88 MB/s)
compile dir kernel-9 680MB in 3.22 seconds (211.38 MB/s)
compile dir kernel-4 680MB in 3.29 seconds (206.88 MB/s)
compile dir kernel-0 680MB in 3.25 seconds (209.43 MB/s)
compile dir kernel-23 680MB in 3.20 seconds (212.70 MB/s)
compile dir kernel-1 680MB in 3.23 seconds (210.72 MB/s)
compile dir kernel-28 680MB in 3.42 seconds (199.02 MB/s)
compile dir kernel-21 680MB in 3.33 seconds (204.39 MB/s)
compile dir kernel-8 680MB in 3.90 seconds (174.52 MB/s)
compile dir kernel-26 680MB in 4.07 seconds (167.23 MB/s)
compile dir kernel-25 680MB in 3.20 seconds (212.70 MB/s)
compile dir kernel-2 680MB in 3.22 seconds (211.38 MB/s)
compile dir kernel-16 680MB in 3.27 seconds (208.15 MB/s)
compile dir kernel-5 680MB in 3.30 seconds (206.25 MB/s)
compile dir kernel-18 680MB in 3.45 seconds (197.29 MB/s)
compile dir kernel-7 680MB in 3.66 seconds (185.97 MB/s)
compile dir kernel-20 680MB in 3.40 seconds (200.19 MB/s)
compile dir kernel-24 680MB in 3.51 seconds (193.91 MB/s)
compile dir kernel-19 680MB in 4.38 seconds (155.40 MB/s)
read dir kernel-11 in 36.13 24.99 MB/s
read dir kernel-19 in 36.00 25.08 MB/s
read dir kernel-8 in 37.51 24.07 MB/s
delete kernel-6 in 7.11 seconds
delete kernel-13 in 6.14 seconds
delete kernel-27 in 6.56 seconds
delete kernel-26 in 6.08 seconds
delete kernel-1 in 6.08 seconds
delete kernel-15 in 8.09 seconds
delete kernel-14 in 5.33 seconds
delete kernel-0 in 6.14 seconds
delete kernel-20 in 5.96 seconds
delete kernel-8 in 6.34 seconds
delete kernel-22 in 5.50 seconds
delete kernel-18 in 6.36 seconds
delete kernel-17 in 6.23 seconds
delete kernel-25 in 5.50 seconds
delete kernel-9 in 5.83 seconds
delete kernel-19 in 7.28 seconds
delete kernel-21 in 6.05 seconds
delete kernel-10 in 5.75 seconds
delete kernel-29 in 6.28 seconds
delete kernel-11 in 6.35 seconds
delete kernel-4 in 5.55 seconds
delete kernel-28 in 5.91 seconds
delete kernel-3 in 5.56 seconds
delete kernel-7 in 6.54 seconds
delete kernel-23 in 5.22 seconds
delete kernel-16 in 5.30 seconds
delete kernel-2 in 5.62 seconds
delete kernel-12 in 4.88 seconds
delete kernel-5 in 5.65 seconds
delete kernel-24 in 5.36 seconds

Ext4:

[root@fc13 compilebench-0.6]# ./compilebench -D /mnt/ext4 -i 30 --makej
using working directory /mnt/ext4, 30 intial dirs 100 runs
native unpatched native-0 222MB in 2.34 seconds (95.03 MB/s)
native patched native-0 109MB in 0.60 seconds (182.78 MB/s)
native patched compiled native-0 691MB in 2.00 seconds (345.80 MB/s)
create dir kernel-0 222MB in 2.16 seconds (102.95 MB/s)
create dir kernel-1 222MB in 2.37 seconds (93.83 MB/s)
create dir kernel-2 222MB in 3.11 seconds (71.50 MB/s)
create dir kernel-3 222MB in 3.71 seconds (59.94 MB/s)
create dir kernel-4 222MB in 3.55 seconds (62.64 MB/s)
create dir kernel-5 222MB in 3.61 seconds (61.60 MB/s)
create dir kernel-6 222MB in 3.34 seconds (66.58 MB/s)
create dir kernel-7 222MB in 3.72 seconds (59.78 MB/s)
create dir kernel-8 222MB in 2.66 seconds (83.60 MB/s)
create dir kernel-9 222MB in 2.74 seconds (81.16 MB/s)
create dir kernel-10 222MB in 2.67 seconds (83.29 MB/s)
create dir kernel-11 222MB in 2.85 seconds (78.03 MB/s)
create dir kernel-12 222MB in 2.21 seconds (100.62 MB/s)
create dir kernel-13 222MB in 2.39 seconds (93.04 MB/s)
create dir kernel-14 222MB in 2.18 seconds (102.01 MB/s)
create dir kernel-15 222MB in 2.27 seconds (97.96 MB/s)
create dir kernel-16 222MB in 2.45 seconds (90.76 MB/s)
create dir kernel-17 222MB in 2.35 seconds (94.63 MB/s)
create dir kernel-18 222MB in 2.37 seconds (93.83 MB/s)
create dir kernel-19 222MB in 4.00 seconds (55.59 MB/s)
create dir kernel-20 222MB in 2.38 seconds (93.43 MB/s)
create dir kernel-21 222MB in 2.39 seconds (93.04 MB/s)
create dir kernel-22 222MB in 2.31 seconds (96.27 MB/s)
create dir kernel-23 222MB in 2.58 seconds (86.19 MB/s)
create dir kernel-24 222MB in 2.56 seconds (86.86 MB/s)
create dir kernel-25 222MB in 2.42 seconds (91.89 MB/s)
create dir kernel-26 222MB in 2.51 seconds (88.60 MB/s)
create dir kernel-27 222MB in 2.45 seconds (90.76 MB/s)
create dir kernel-28 222MB in 2.53 seconds (87.89 MB/s)
create dir kernel-29 222MB in 2.46 seconds (90.40 MB/s)
compile dir kernel-6 680MB in 1.23 seconds (553.36 MB/s)
compile dir kernel-29 680MB in 4.10 seconds (166.01 MB/s)
compile dir kernel-11 680MB in 4.35 seconds (156.47 MB/s)
compile dir kernel-22 680MB in 4.42 seconds (153.99 MB/s)
compile dir kernel-15 680MB in 4.59 seconds (148.29 MB/s)
compile dir kernel-13 680MB in 4.31 seconds (157.92 MB/s)
compile dir kernel-17 680MB in 5.15 seconds (132.16 MB/s)
compile dir kernel-3 680MB in 4.40 seconds (154.69 MB/s)
compile dir kernel-10 680MB in 4.47 seconds (152.27 MB/s)
compile dir kernel-12 680MB in 4.51 seconds (150.92 MB/s)
compile dir kernel-27 680MB in 4.69 seconds (145.12 MB/s)
compile dir kernel-14 680MB in 4.75 seconds (143.29 MB/s)
compile dir kernel-9 680MB in 4.34 seconds (156.83 MB/s)
compile dir kernel-4 680MB in 4.49 seconds (151.59 MB/s)
compile dir kernel-0 680MB in 4.53 seconds (150.25 MB/s)
compile dir kernel-23 680MB in 4.62 seconds (147.32 MB/s)
compile dir kernel-1 680MB in 4.54 seconds (149.92 MB/s)
compile dir kernel-28 680MB in 4.32 seconds (157.55 MB/s)
compile dir kernel-21 680MB in 4.92 seconds (138.34 MB/s)
compile dir kernel-8 680MB in 4.63 seconds (147.01 MB/s)
compile dir kernel-26 680MB in 4.45 seconds (152.95 MB/s)
compile dir kernel-25 680MB in 4.66 seconds (146.06 MB/s)
compile dir kernel-2 680MB in 4.34 seconds (156.83 MB/s)
compile dir kernel-16 680MB in 4.66 seconds (146.06 MB/s)
compile dir kernel-5 680MB in 4.63 seconds (147.01 MB/s)
compile dir kernel-18 680MB in 4.41 seconds (154.34 MB/s)
compile dir kernel-7 680MB in 5.06 seconds (134.51 MB/s)
compile dir kernel-20 680MB in 4.85 seconds (140.34 MB/s)
compile dir kernel-24 680MB in 4.40 seconds (154.69 MB/s)
compile dir kernel-19 680MB in 4.48 seconds (151.93 MB/s)
read dir kernel-11 in 31.82 28.38 MB/s
read dir kernel-19 in 35.85 25.19 MB/s
read dir kernel-8 in 38.25 23.61 MB/s
delete kernel-6 in 1.46 seconds
delete kernel-13 in 1.43 seconds
delete kernel-27 in 1.49 seconds
delete kernel-26 in 1.59 seconds
delete kernel-1 in 1.50 seconds
delete kernel-15 in 1.52 seconds
delete kernel-14 in 1.64 seconds
delete kernel-0 in 1.50 seconds
delete kernel-20 in 1.47 seconds
delete kernel-8 in 1.60 seconds
delete kernel-22 in 1.54 seconds
delete kernel-18 in 1.50 seconds
delete kernel-17 in 1.46 seconds
delete kernel-25 in 1.88 seconds
delete kernel-9 in 1.49 seconds
delete kernel-19 in 1.51 seconds
delete kernel-21 in 1.63 seconds
delete kernel-10 in 1.47 seconds
delete kernel-29 in 1.50 seconds
delete kernel-11 in 1.83 seconds
delete kernel-4 in 1.51 seconds
delete kernel-28 in 1.51 seconds
delete kernel-3 in 1.89 seconds
delete kernel-7 in 1.54 seconds
delete kernel-23 in 1.46 seconds
delete kernel-16 in 2.88 seconds
delete kernel-2 in 1.48 seconds
delete kernel-12 in 1.51 seconds
delete kernel-5 in 1.57 seconds
delete kernel-24 in 1.52 seconds

Best regards,

Morten



On Wed, Aug 18, 2010 at 4:25 PM, Chris Mason <chris.mason@oracle.com> wrote:
> On Wed, Aug 18, 2010 at 12:49:05PM +0200, Morten P.D. Stevens wrote:
>> Hi,
>>
>> i can confirm this issue with poor create and delete performance.
>>
>> For example with unpacking and deleting the linux kernel:
>>
>> Btrfs:
>>
>> [root@fc13 btrfs]# time tar xfj linux-2.6.36-rc1.tar.bz2
>>
>> real    0m18.794s
>> user    0m12.045s
>> sys     0m8.241s
>> [root@fc13 btrfs]# time rm -rf linux-2.6.36-rc1
>>
>> real    0m2.156s
>> user    0m0.028s
>> sys     0m2.118s
>>
>> ext4:
>>
>> [root@fc13 ext4]# time tar xfj linux-2.6.36-rc1.tar.bz2
>>
>> real    0m13.140s
>> user    0m9.074s
>> sys     0m6.327s
>> [root@fc13 ext4]# time rm -rf linux-2.6.36-rc1
>>
>> real    0m0.781s
>> user    0m0.031s
>> sys     0m0.735s
>>
>> In all of our benchmarks, ext4 is still faster than btrfs.
>
> For this kind of benchmark, I use my compilebench program.  We want to
> time just the file creation and not how long it takes to uncompress or
> read in the .tar.gz from the disk.
>
> The results from 2.6.35 on a single sata drive are not good for ext4 at
> all.  Btrfs is on par with where it always has been, but I'm
> experimenting with other benchmarks to find the regression.  Basically
> when you are benchmarking this kind of thing you want to make sure
> you're benchmarking how quickly you can write to the disk, and not to
> cache.
>
> compilebench simulates this by creating files with the same size and
> names as what you get from untarring kernel trees, and then it has a
> phase where it creates files with the same size and names as the .o
> files from compiling the kernel.
>
> Btrfs:
>
> # ./compilebench -i 30 --makej
> creating /mnt/default
> using working directory /mnt/default, 30 intial dirs 100 runs
> native unpatched native-0 222MB in 1.76 seconds (126.35 MB/s)
> native patched native-0 109MB in 0.51 seconds (215.04 MB/s)
> native patched compiled native-0 691MB in 5.69 seconds (121.55 MB/s)
> create dir kernel-0 222MB in 2.94 seconds (75.64 MB/s)
> create dir kernel-1 222MB in 3.16 seconds (70.37 MB/s)
> create dir kernel-2 222MB in 3.05 seconds (72.91 MB/s)
> create dir kernel-3 222MB in 3.11 seconds (71.50 MB/s)
> create dir kernel-4 222MB in 3.88 seconds (57.31 MB/s)
> create dir kernel-5 222MB in 3.58 seconds (62.12 MB/s)
> create dir kernel-6 222MB in 6.11 seconds (36.40 MB/s)
> create dir kernel-7 222MB in 2.28 seconds (97.53 MB/s)
> create dir kernel-8 222MB in 3.12 seconds (71.27 MB/s)
> create dir kernel-9 222MB in 2.86 seconds (77.75 MB/s)
> create dir kernel-10 222MB in 4.46 seconds (49.86 MB/s)
> create dir kernel-11 222MB in 2.96 seconds (75.13 MB/s)
> create dir kernel-12 222MB in 3.86 seconds (57.61 MB/s)
> create dir kernel-13 222MB in 3.96 seconds (56.16 MB/s)
> create dir kernel-14 222MB in 3.79 seconds (58.67 MB/s)
> create dir kernel-15 222MB in 6.58 seconds (33.80 MB/s)
> create dir kernel-16 222MB in 2.09 seconds (106.40 MB/s)
> create dir kernel-17 222MB in 2.71 seconds (82.06 MB/s)
> create dir kernel-18 222MB in 3.40 seconds (65.40 MB/s)
> create dir kernel-19 222MB in 3.25 seconds (68.42 MB/s)
> create dir kernel-20 222MB in 3.79 seconds (58.67 MB/s)
> create dir kernel-21 222MB in 3.86 seconds (57.61 MB/s)
> create dir kernel-22 222MB in 3.59 seconds (61.94 MB/s)
> create dir kernel-23 222MB in 4.02 seconds (55.32 MB/s)
> create dir kernel-24 222MB in 7.68 seconds (28.95 MB/s)
> create dir kernel-25 222MB in 1.99 seconds (111.75 MB/s)
> create dir kernel-26 222MB in 2.77 seconds (80.28 MB/s)
> create dir kernel-27 222MB in 2.97 seconds (74.87 MB/s)
> create dir kernel-28 222MB in 3.22 seconds (69.06 MB/s)
> create dir kernel-29 222MB in 3.94 seconds (56.44 MB/s)
> compile dir kernel-6 680MB in 3.78 seconds (180.06 MB/s)
> compile dir kernel-29 680MB in 9.21 seconds (73.90 MB/s)
> compile dir kernel-11 680MB in 9.38 seconds (72.56 MB/s)
> compile dir kernel-22 680MB in 10.57 seconds (64.39 MB/s)
> compile dir kernel-15 680MB in 7.68 seconds (88.62 MB/s)
> compile dir kernel-13 680MB in 9.26 seconds (73.50 MB/s)
> compile dir kernel-17 680MB in 9.44 seconds (72.10 MB/s)
> compile dir kernel-3 680MB in 9.03 seconds (75.37 MB/s)
> compile dir kernel-10 680MB in 9.30 seconds (73.19 MB/s)
> compile dir kernel-12 680MB in 9.29 seconds (73.27 MB/s)
> compile dir kernel-27 680MB in 10.84 seconds (62.79 MB/s)
> compile dir kernel-14 680MB in 7.65 seconds (88.97 MB/s)
> compile dir kernel-9 680MB in 9.55 seconds (71.27 MB/s)
> compile dir kernel-4 680MB in 9.20 seconds (73.98 MB/s)
> compile dir kernel-0 680MB in 12.06 seconds (56.44 MB/s)
> compile dir kernel-23 680MB in 9.39 seconds (72.49 MB/s)
> compile dir kernel-1 680MB in 9.06 seconds (75.13 MB/s)
>
> Ext4:
>
> ./compilebench -i 30 --makej
> creating /mnt/default
> using working directory /mnt/default, 30 intial dirs 100 runs
> native unpatched native-0 222MB in 1.11 seconds (200.34 MB/s)
> native patched native-0 109MB in 0.33 seconds (332.33 MB/s)
> native patched compiled native-0 691MB in 5.38 seconds (128.55 MB/s)
> create dir kernel-0 222MB in 1.05 seconds (211.78 MB/s)
> create dir kernel-1 222MB in 4.60 seconds (48.34 MB/s)
> create dir kernel-2 222MB in 4.30 seconds (51.71 MB/s)
> create dir kernel-3 222MB in 5.71 seconds (38.94 MB/s)
> create dir kernel-4 222MB in 4.82 seconds (46.14 MB/s)
> create dir kernel-5 222MB in 4.91 seconds (45.29 MB/s)
> create dir kernel-6 222MB in 5.70 seconds (39.01 MB/s)
> create dir kernel-7 222MB in 4.76 seconds (46.72 MB/s)
> create dir kernel-8 222MB in 6.40 seconds (34.75 MB/s)
> create dir kernel-9 222MB in 5.37 seconds (41.41 MB/s)
> create dir kernel-10 222MB in 5.51 seconds (40.36 MB/s)
> create dir kernel-11 222MB in 6.77 seconds (32.85 MB/s)
> create dir kernel-12 222MB in 5.88 seconds (37.82 MB/s)
> create dir kernel-13 222MB in 8.92 seconds (24.93 MB/s)
> create dir kernel-14 222MB in 5.44 seconds (40.88 MB/s)
> create dir kernel-15 222MB in 6.21 seconds (35.81 MB/s)
> create dir kernel-16 222MB in 4.85 seconds (45.85 MB/s)
> create dir kernel-17 222MB in 8.83 seconds (25.18 MB/s)
> create dir kernel-18 222MB in 6.82 seconds (32.61 MB/s)
> create dir kernel-19 222MB in 4.86 seconds (45.76 MB/s)
> create dir kernel-20 222MB in 10.48 seconds (21.22 MB/s)
> create dir kernel-21 222MB in 4.85 seconds (45.85 MB/s)
> create dir kernel-22 222MB in 8.30 seconds (26.79 MB/s)
> create dir kernel-23 222MB in 6.38 seconds (34.85 MB/s)
> create dir kernel-24 222MB in 7.30 seconds (30.46 MB/s)
> create dir kernel-25 222MB in 3.83 seconds (58.06 MB/s)
> create dir kernel-26 222MB in 9.70 seconds (22.93 MB/s)
> create dir kernel-27 222MB in 5.39 seconds (41.26 MB/s)
> create dir kernel-28 222MB in 6.96 seconds (31.95 MB/s)
> create dir kernel-29 222MB in 8.16 seconds (27.25 MB/s)
> compile dir kernel-6 680MB in 7.47 seconds (91.12 MB/s)
> compile dir kernel-29 680MB in 16.37 seconds (41.58 MB/s)
> compile dir kernel-11 680MB in 19.57 seconds (34.78 MB/s)
> compile dir kernel-22 680MB in 17.17 seconds (39.64 MB/s)
> compile dir kernel-15 680MB in 17.40 seconds (39.12 MB/s)
> compile dir kernel-13 680MB in 17.87 seconds (38.09 MB/s)
> compile dir kernel-17 680MB in 18.14 seconds (37.52 MB/s)
> compile dir kernel-3 680MB in 17.43 seconds (39.05 MB/s)
> compile dir kernel-10 680MB in 15.18 seconds (44.84 MB/s)
> compile dir kernel-12 680MB in 17.14 seconds (39.71 MB/s)
> compile dir kernel-27 680MB in 18.71 seconds (36.38 MB/s)
> compile dir kernel-14 680MB in 17.47 seconds (38.96 MB/s)
> compile dir kernel-9 680MB in 19.76 seconds (34.45 MB/s)
> compile dir kernel-4 680MB in 17.53 seconds (38.83 MB/s)
> compile dir kernel-0 680MB in 17.20 seconds (39.57 MB/s)
> compile dir kernel-23 680MB in 17.31 seconds (39.32 MB/s)
> compile dir kernel-1 680MB in 18.19 seconds (37.42 MB/s)
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Poor creat/delete files performance

[Chris Mason]

On Wed, Aug 18, 2010 at 05:28:44PM +0200, Morten P.D. Stevens wrote:
> Hi Chris,
> 
> Your compilebench results are very interesting.
> 
> Here are my results with exactly the same benchmark:
> 
> Btrfs:
> 
> [root@fc13 compilebench-0.6]# ./compilebench -D /mnt/btrfs -i 30 --makej
> using working directory /mnt/btrfs, 30 intial dirs 100 runs
> native unpatched native-0 222MB in 3.14 seconds (70.82 MB/s)
> native patched native-0 109MB in 0.69 seconds (158.94 MB/s)
> native patched compiled native-0 691MB in 1.86 seconds (371.83 MB/s)
> create dir kernel-0 222MB in 2.28 seconds (97.53 MB/s)
> create dir kernel-1 222MB in 3.45 seconds (64.46 MB/s)

You'll probably find the btrfs create and delete speeds improve if you
mount with -o max_inline=0.  Please confirm which kernel you were
running?

Also btrfs hammers on slab/slub very hard, so a kernel with slab/slub
debugging on will tend to be slower on btrfs than the other filesystems.

[ btrfs results for the compile phase ]
> compile dir kernel-29 680MB in 2.93 seconds (232.30 MB/s)
> compile dir kernel-11 680MB in 3.24 seconds (210.07 MB/s)
> compile dir kernel-22 680MB in 3.12 seconds (218.15 MB/s)
> compile dir kernel-15 680MB in 3.25 seconds (209.43 MB/s)
> compile dir kernel-13 680MB in 3.23 seconds (210.72 MB/s)

> 
> Ext4:
> 
> [root@fc13 compilebench-0.6]# ./compilebench -D /mnt/ext4 -i 30 --makej
> using working directory /mnt/ext4, 30 intial dirs 100 runs
> native unpatched native-0 222MB in 2.34 seconds (95.03 MB/s)
> native patched native-0 109MB in 0.60 seconds (182.78 MB/s)
> native patched compiled native-0 691MB in 2.00 seconds (345.80 MB/s)
> create dir kernel-0 222MB in 2.16 seconds (102.95 MB/s)
> create dir kernel-1 222MB in 2.37 seconds (93.83 MB/s)
> create dir kernel-2 222MB in 3.11 seconds (71.50 MB/s)
> create dir kernel-3 222MB in 3.71 seconds (59.94 MB/s)
> create dir kernel-4 222MB in 3.55 seconds (62.64 MB/s)
> create dir kernel-5 222MB in 3.61 seconds (61.60 MB/s)
> create dir kernel-6 222MB in 3.34 seconds (66.58 MB/s)
> create dir kernel-7 222MB in 3.72 seconds (59.78 MB/s)
> create dir kernel-8 222MB in 2.66 seconds (83.60 MB/s)

Wow, much better than my drive.  What kind of storage is this?

> compile dir kernel-29 680MB in 4.10 seconds (166.01 MB/s)
> compile dir kernel-11 680MB in 4.35 seconds (156.47 MB/s)
> compile dir kernel-22 680MB in 4.42 seconds (153.99 MB/s)
> compile dir kernel-15 680MB in 4.59 seconds (148.29 MB/s)

Much lower than btrfs.  Strictly speaking this could be a layout
decision.  Ext4 could be trying to place the new .o files close to the
rest of the files in that subdirectory.  Btrfs tries harder to maximize
writeback speeds because COW causes fragmentation anyway.

-chris

RE: Poor creat/delete files performance

[Morten P.D. Stevens]

Hi Chris,

I'm using a standard fedora 13 kernel.
Linux fc13.corp.imt-systems.com 2.6.33.6-147.2.4.fc13.x86_64 #1 SMP Fri=
 Jul 23 17:14:44 UTC 2010 x86_64 x86_64 x86_64 GNU/Linux

If you like, I can rerun these benchmarks with 2.6.36-rc1.

My Storage:

2x Western Digital WD5000AAKS SATA Disks with hardware raid 0 on a LSI =
1068e controller.

Best regards,

Morten

On Wed, Aug 18, 2010 at 5:39 PM, Chris Mason <chris.mason@oracle.com> w=
rote:
> On Wed, Aug 18, 2010 at 05:28:44PM +0200, Morten P.D. Stevens wrote:
>> Hi Chris,
>>
>> Your compilebench results are very interesting.
>>
>> Here are my results with exactly the same benchmark:
>>
>> Btrfs:
>>
>> [root@fc13 compilebench-0.6]# ./compilebench -D /mnt/btrfs -i 30 --m=
akej
>> using working directory /mnt/btrfs, 30 intial dirs 100 runs
>> native unpatched native-0 222MB in 3.14 seconds (70.82 MB/s)
>> native patched native-0 109MB in 0.69 seconds (158.94 MB/s)
>> native patched compiled native-0 691MB in 1.86 seconds (371.83 MB/s)
>> create dir kernel-0 222MB in 2.28 seconds (97.53 MB/s)
>> create dir kernel-1 222MB in 3.45 seconds (64.46 MB/s)
>
> You'll probably find the btrfs create and delete speeds improve if yo=
u
> mount with -o max_inline=3D0. =A0Please confirm which kernel you were
> running?
>
> Also btrfs hammers on slab/slub very hard, so a kernel with slab/slub
> debugging on will tend to be slower on btrfs than the other filesyste=
ms.
>
> [ btrfs results for the compile phase ]
>> compile dir kernel-29 680MB in 2.93 seconds (232.30 MB/s)
>> compile dir kernel-11 680MB in 3.24 seconds (210.07 MB/s)
>> compile dir kernel-22 680MB in 3.12 seconds (218.15 MB/s)
>> compile dir kernel-15 680MB in 3.25 seconds (209.43 MB/s)
>> compile dir kernel-13 680MB in 3.23 seconds (210.72 MB/s)
>
>>
>> Ext4:
>>
>> [root@fc13 compilebench-0.6]# ./compilebench -D /mnt/ext4 -i 30 --ma=
kej
>> using working directory /mnt/ext4, 30 intial dirs 100 runs
>> native unpatched native-0 222MB in 2.34 seconds (95.03 MB/s)
>> native patched native-0 109MB in 0.60 seconds (182.78 MB/s)
>> native patched compiled native-0 691MB in 2.00 seconds (345.80 MB/s)
>> create dir kernel-0 222MB in 2.16 seconds (102.95 MB/s)
>> create dir kernel-1 222MB in 2.37 seconds (93.83 MB/s)
>> create dir kernel-2 222MB in 3.11 seconds (71.50 MB/s)
>> create dir kernel-3 222MB in 3.71 seconds (59.94 MB/s)
>> create dir kernel-4 222MB in 3.55 seconds (62.64 MB/s)
>> create dir kernel-5 222MB in 3.61 seconds (61.60 MB/s)
>> create dir kernel-6 222MB in 3.34 seconds (66.58 MB/s)
>> create dir kernel-7 222MB in 3.72 seconds (59.78 MB/s)
>> create dir kernel-8 222MB in 2.66 seconds (83.60 MB/s)
>
> Wow, much better than my drive. =A0What kind of storage is this?
>
>> compile dir kernel-29 680MB in 4.10 seconds (166.01 MB/s)
>> compile dir kernel-11 680MB in 4.35 seconds (156.47 MB/s)
>> compile dir kernel-22 680MB in 4.42 seconds (153.99 MB/s)
>> compile dir kernel-15 680MB in 4.59 seconds (148.29 MB/s)
>
> Much lower than btrfs. =A0Strictly speaking this could be a layout
> decision. =A0Ext4 could be trying to place the new .o files close to =
the
> rest of the files in that subdirectory. =A0Btrfs tries harder to maxi=
mize
> writeback speeds because COW causes fragmentation anyway.
>
> -chris
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs=
" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at =A0http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" =
in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Poor creat/delete files performance

[Miao Xie]

On Wed, 18 Aug 2010 08:09:41 -0400, Chris Mason wrote:
>> We did some performance test and found the create/delete files performance
>> of btrfs is very poor.
>>
>> The test is that we create 50000 files and measure the file-create time
>> first, and then delete these 50000 files and measure the file-delete time.
>> (The attached file is the reproduce program)
>>
>> The result is following:
>> (Unit: second)
>>    Create file performance
>> 		BtrFS		Ext4
>>    Total times:	2.462625	1.449550
>>    Average:	0.000049	0.000029
>>
>>    Delete file performance
>> 		BtrFS		Ext4
>>    Total times:	3.312796	0.997946
>>    Average:	0.000066	0.000020
>>
>> The results were measured on a x86_64 server with 4 cores and 2 SAS disks.
>> By debuging, we found the btrfs spent a lot of time on searching and
>> inserting/removing items in the ctree.
>>
>> Is anyone looking at this issue?
>
> I'm looking at it now, which kernel were you on?  We do spend some CPU
> time on the btree but it shouldn't be a big bottleneck compared to the
> disk.

I tested it on v2.6.35 kernel.

Regards
Miao Xie

>
> This should be very similar to the performance regression that Steve
> Pratt hit.  Our deletion time has always been slower than ext4, but the
> creation time should be the same or faster.
>
> -chris
>
>
>

Re: Poor creat/delete files performance

[Chris Mason]

On Thu, Aug 19, 2010 at 08:35:18AM +0800, Miao Xie wrote:
> On Wed, 18 Aug 2010 08:09:41 -0400, Chris Mason wrote:
> >>We did some performance test and found the create/delete files performance
> >>of btrfs is very poor.
> >>
> >>The test is that we create 50000 files and measure the file-create time
> >>first, and then delete these 50000 files and measure the file-delete time.
> >>(The attached file is the reproduce program)
> >>
> >>The result is following:
> >>(Unit: second)
> >>   Create file performance
> >>		BtrFS		Ext4
> >>   Total times:	2.462625	1.449550
> >>   Average:	0.000049	0.000029
> >>
> >>   Delete file performance
> >>		BtrFS		Ext4
> >>   Total times:	3.312796	0.997946
> >>   Average:	0.000066	0.000020
> >>
> >>The results were measured on a x86_64 server with 4 cores and 2 SAS disks.
> >>By debuging, we found the btrfs spent a lot of time on searching and
> >>inserting/removing items in the ctree.
> >>
> >>Is anyone looking at this issue?
> >
> >I'm looking at it now, which kernel were you on?  We do spend some CPU
> >time on the btree but it shouldn't be a big bottleneck compared to the
> >disk.
> 
> I tested it on v2.6.35 kernel.

Sorry, I misread your first email slightly, I didn't realize the files
from the benchmark program were empty.

Since the files are empty, and we aren't doing enough files to trigger
IO, it is really benchmarking the cost of the btree insertions/removals
in comparison with ext4.  I do expect this to be higher because btrfs is
indexing the directories twice (once by name and once by sequence number
for faster backups).

On my machine:

Btrfs defaults:

Create files:
	Total files: 50000
	Total time: 0.916680
	Average time: 0.000018
Delete files:
	Total files: 50000
	Total time: 1.329892
	Average time: 0.000027

Ext4:

creat_unlink 50000
Create files:
	Total files: 50000
	Total time: 0.718190
	Average time: 0.000014
Delete files:
	Total files: 50000
	Total time: 0.308815
	Average time: 0.000006

We're definitely slower than ext4, but as Ric's benchmarks show things
tend to tilt in our favor once IO is actually done.

There are two big things that would help fix this performance gap:
Switching the extent buffer rbtree into a radix tree (esp a lockless
radix tree), and delaying insertion of the inode so that we can do more
in btree operations in bulk.

The radix tree is a much easier and more contained project.

-chris

Re: Poor creat/delete files performance

[Miao Xie]

On Wed, 18 Aug 2010 20:57:43 -0400, Chris Mason wrote:
> Since the files are empty, and we aren't doing enough files to trigger
> IO, it is really benchmarking the cost of the btree insertions/removals
> in comparison with ext4.  I do expect this to be higher because btrfs is
> indexing the directories twice (once by name and once by sequence number
> for faster backups).
>
> On my machine:
>
> Btrfs defaults:
>
> Create files:
> 	Total files: 50000
> 	Total time: 0.916680
> 	Average time: 0.000018
> Delete files:
> 	Total files: 50000
> 	Total time: 1.329892
> 	Average time: 0.000027
>
> Ext4:
>
> creat_unlink 50000
> Create files:
> 	Total files: 50000
> 	Total time: 0.718190
> 	Average time: 0.000014
> Delete files:
> 	Total files: 50000
> 	Total time: 0.308815
> 	Average time: 0.000006
>
> We're definitely slower than ext4, but as Ric's benchmarks show things
> tend to tilt in our favor once IO is actually done.
>
> There are two big things that would help fix this performance gap:
> Switching the extent buffer rbtree into a radix tree (esp a lockless
> radix tree), and delaying insertion of the inode so that we can do more
> in btree operations in bulk.
>
> The radix tree is a much easier and more contained project.

It is good idea, We will try to do it.

Thanks
Miao Xie

Re: Poor creat/delete files performance

[Miao Xie]

On Wed, 18 Aug 2010 20:57:43 -0400, Chris Mason wrote:
> Since the files are empty, and we aren't doing enough files to trigger
> IO, it is really benchmarking the cost of the btree insertions/removals
> in comparison with ext4.  I do expect this to be higher because btrfs is
> indexing the directories twice (once by name and once by sequence number
> for faster backups).
>
> On my machine:
>
> Btrfs defaults:
>
> Create files:
> 	Total files: 50000
> 	Total time: 0.916680
> 	Average time: 0.000018
> Delete files:
> 	Total files: 50000
> 	Total time: 1.329892
> 	Average time: 0.000027
>
> Ext4:
>
> creat_unlink 50000
> Create files:
> 	Total files: 50000
> 	Total time: 0.718190
> 	Average time: 0.000014
> Delete files:
> 	Total files: 50000
> 	Total time: 0.308815
> 	Average time: 0.000006
>
> We're definitely slower than ext4, but as Ric's benchmarks show things
> tend to tilt in our favor once IO is actually done.
>
> There are two big things that would help fix this performance gap:
> Switching the extent buffer rbtree into a radix tree (esp a lockless
> radix tree), and delaying insertion of the inode so that we can do more
> in btree operations in bulk.
>
> The radix tree is a much easier and more contained project.

The type of the radix tree's key is "unsigned long", but the type of the
extent buffer's key is "u64". That is we can't use the radix tree instead of
rbtree on the 32-bits boxs. So we can't switching the extent buffer rbtree
into a radix tree.

Thanks
Miao Xie

Re: Poor creat/delete files performance

[Chris Mason]

On Thu, Aug 26, 2010 at 06:07:55PM +0800, Miao Xie wrote:
> On Wed, 18 Aug 2010 20:57:43 -0400, Chris Mason wrote:
> >Since the files are empty, and we aren't doing enough files to trigger
> >IO, it is really benchmarking the cost of the btree insertions/removals
> >in comparison with ext4.  I do expect this to be higher because btrfs is
> >indexing the directories twice (once by name and once by sequence number
> >for faster backups).
> >
> >On my machine:
> >
> >Btrfs defaults:
> >
> >Create files:
> >	Total files: 50000
> >	Total time: 0.916680
> >	Average time: 0.000018
> >Delete files:
> >	Total files: 50000
> >	Total time: 1.329892
> >	Average time: 0.000027
> >
> >Ext4:
> >
> >creat_unlink 50000
> >Create files:
> >	Total files: 50000
> >	Total time: 0.718190
> >	Average time: 0.000014
> >Delete files:
> >	Total files: 50000
> >	Total time: 0.308815
> >	Average time: 0.000006
> >
> >We're definitely slower than ext4, but as Ric's benchmarks show things
> >tend to tilt in our favor once IO is actually done.
> >
> >There are two big things that would help fix this performance gap:
> >Switching the extent buffer rbtree into a radix tree (esp a lockless
> >radix tree), and delaying insertion of the inode so that we can do more
> >in btree operations in bulk.
> >
> >The radix tree is a much easier and more contained project.
> 
> The type of the radix tree's key is "unsigned long", but the type of the
> extent buffer's key is "u64". That is we can't use the radix tree instead of
> rbtree on the 32-bits boxs. So we can't switching the extent buffer rbtree
> into a radix tree.

Right, but the key is just the byte number offset from 0.  The extent
buffers are backed by pages, and the pages are allocated off the
metadata inode's address space, which is backed by a radix tree.

You can try using the (bytes offset >> PAGE_CACHE_SHIFT).  The problem
you might hit is the radix tree is tuned pretty hard now for the page
cache.

Another option is to attach the extent buffers to page->private, and use
the page cache's radix tree (remove the rbtree completely).  For
blocksize > pagesize, we could only put the first page in each block
into the page cache, and just tie the rest of the off the extent buffer.

But, if you get the 4K metadata block size part working, I can cram in
the larger block sizes pretty easily now.

-chris