* The value displayed by 'ls -s' command is strange.
@ 2010-12-07 7:59 Tsutomu Itoh
2010-12-07 9:25 ` Li Zefan
2010-12-07 18:44 ` Chris Mason
0 siblings, 2 replies; 11+ messages in thread
From: Tsutomu Itoh @ 2010-12-07 7:59 UTC (permalink / raw)
To: Linux Btrfs
Hi,
I think that the disk allocation size of each file becomes a monotone increase
when the file is made.
But, it sometimes return to 0. Is it correct?
The result of the test at 2.6.37-rc4 is shown below.
(see inode no. 291)
# df -T /test14
Filesystem Type 1K-blocks Used Available Use% Mounted on
/dev/sdd14 btrfs 4162560 8736 3709440 1% /test14
# dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100
# dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200
# dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct
# dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400
# ls -lis /test14/dir
total 406528
288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603
289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603
-> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603
290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603
# sleep 3
# ls -lis /test14/dir
total 406528
288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603
289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603
-> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603
290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603
# sleep 3
# ls -lis /test14/dir
total 307200
288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603
289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603
-> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603
290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603
# sleep 3
# ls -lis /test14/dir
total 409600
288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603
289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603
-> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603
290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603
# sync
# ls -lis /test14/dir
total 1024000
288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603
289 204800 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603
-> 291 409600 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603
290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603
The trace result of btrfs_getattr() is shown below.
Dec 7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
Dec 7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872
Dec 7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
Dec 7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0
Dec 7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0
Regards,
Itoh
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 7:59 The value displayed by 'ls -s' command is strange Tsutomu Itoh @ 2010-12-07 9:25 ` Li Zefan 2010-12-07 23:53 ` Tsutomu Itoh 2010-12-07 18:44 ` Chris Mason 1 sibling, 1 reply; 11+ messages in thread From: Li Zefan @ 2010-12-07 9:25 UTC (permalink / raw) To: Tsutomu Itoh; +Cc: Linux Btrfs Tsutomu Itoh wrote: > Hi, > > I think that the disk allocation size of each file becomes a monotone increase > when the file is made. > But, it sometimes return to 0. Is it correct? > The # of blocks is: stat->blocks = (inode_get_bytes(inode) + BTRFS_I(inode)->delalloc_bytes) >> 9; So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may see 0 value. > > The result of the test at 2.6.37-rc4 is shown below. > (see inode no. 291) > > # df -T /test14 > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/sdd14 btrfs 4162560 8736 3709440 1% /test14 > # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100 > # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200 > # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct > # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400 > # ls -lis /test14/dir > total 406528 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 406528 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 307200 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 409600 > 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sync > # ls -lis /test14/dir > total 1024000 > 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 204800 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > > The trace result of btrfs_getattr() is shown below. > > Dec 7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 > Dec 7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 > Dec 7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 > Dec 7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 > Dec 7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0 > > > Regards, > Itoh > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 9:25 ` Li Zefan @ 2010-12-07 23:53 ` Tsutomu Itoh 2010-12-09 10:42 ` Miao Xie 0 siblings, 1 reply; 11+ messages in thread From: Tsutomu Itoh @ 2010-12-07 23:53 UTC (permalink / raw) To: Li Zefan; +Cc: Linux Btrfs (2010/12/07 18:25), Li Zefan wrote: > Tsutomu Itoh wrote: >> Hi, >> >> I think that the disk allocation size of each file becomes a monotone increase >> when the file is made. >> But, it sometimes return to 0. Is it correct? >> > > The # of blocks is: > > stat->blocks = (inode_get_bytes(inode) + > BTRFS_I(inode)->delalloc_bytes) >> 9; > > So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may > see 0 value. Yes, I also think so. But, I think that such a state is too long for only the update timing... > >> >> The result of the test at 2.6.37-rc4 is shown below. >> (see inode no. 291) >> >> # df -T /test14 >> Filesystem Type 1K-blocks Used Available Use% Mounted on >> /dev/sdd14 btrfs 4162560 8736 3709440 1% /test14 >> # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100 >> # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200 >> # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct >> # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400 >> # ls -lis /test14/dir >> total 406528 >> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >> -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >> # sleep 3 >> # ls -lis /test14/dir >> total 406528 >> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >> -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >> # sleep 3 >> # ls -lis /test14/dir >> total 307200 >> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >> -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >> # sleep 3 >> # ls -lis /test14/dir >> total 409600 >> 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >> -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >> # sync >> # ls -lis /test14/dir >> total 1024000 >> 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >> 289 204800 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >> -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >> >> The trace result of btrfs_getattr() is shown below. >> >> Dec 7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 >> Dec 7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 >> Dec 7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 >> Dec 7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 >> Dec 7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0 >> >> >> Regards, >> Itoh >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 23:53 ` Tsutomu Itoh @ 2010-12-09 10:42 ` Miao Xie 0 siblings, 0 replies; 11+ messages in thread From: Miao Xie @ 2010-12-09 10:42 UTC (permalink / raw) To: Tsutomu Itoh; +Cc: Li Zefan, Linux Btrfs On wed, 08 Dec 2010 08:53:55 +0900, Tsutomu Itoh wrote: >>> I think that the disk allocation size of each file becomes a monotone increase >>> when the file is made. >>> But, it sometimes return to 0. Is it correct? >>> >> >> The # of blocks is: >> >> stat->blocks = (inode_get_bytes(inode) + >> BTRFS_I(inode)->delalloc_bytes)>> 9; >> >> So I think after sub(delalloc_bytes) and before inode_add_bytes(), you may >> see 0 value. > > Yes, I also think so. > But, I think that such a state is too long for only the update timing... Several months ago, some one posted a patch to get the allocated size of the compressed file, http://marc.info/?l=linux-btrfs&m=128109745012238&w=2 this patch may help you to implement what you need. Regards Miao >> >>> >>> The result of the test at 2.6.37-rc4 is shown below. >>> (see inode no. 291) >>> >>> # df -T /test14 >>> Filesystem Type 1K-blocks Used Available Use% Mounted on >>> /dev/sdd14 btrfs 4162560 8736 3709440 1% /test14 >>> # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100 >>> # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200 >>> # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct >>> # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400 >>> # ls -lis /test14/dir >>> total 406528 >>> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >>> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >>> -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >>> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >>> # sleep 3 >>> # ls -lis /test14/dir >>> total 406528 >>> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >>> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >>> -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >>> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >>> # sleep 3 >>> # ls -lis /test14/dir >>> total 307200 >>> 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >>> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >>> -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >>> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >>> # sleep 3 >>> # ls -lis /test14/dir >>> total 409600 >>> 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >>> 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >>> -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >>> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >>> # sync >>> # ls -lis /test14/dir >>> total 1024000 >>> 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 >>> 289 204800 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 >>> -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 >>> 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 >>> >>> The trace result of btrfs_getattr() is shown below. >>> >>> Dec 7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 >>> Dec 7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 >>> Dec 7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 >>> Dec 7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 >>> Dec 7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0 >>> >>> >>> Regards, >>> Itoh >>> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majordomo@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> >> > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 7:59 The value displayed by 'ls -s' command is strange Tsutomu Itoh 2010-12-07 9:25 ` Li Zefan @ 2010-12-07 18:44 ` Chris Mason 2010-12-07 19:16 ` Mike Fedyk 1 sibling, 1 reply; 11+ messages in thread From: Chris Mason @ 2010-12-07 18:44 UTC (permalink / raw) To: Tsutomu Itoh; +Cc: Linux Btrfs Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500: > Hi, > > I think that the disk allocation size of each file becomes a monotone increase > when the file is made. > But, it sometimes return to 0. Is it correct? Well, there's a window during the processing of delayed allocation where we don't have the bytes recorded as delalloc and we don't have the bytes recorded in the inode yet. That's why they are showing up as zero. We don't call inode_add_bytes() until after we insert the extent, but we drop the delalloc byte count on the file before the IO is done. Fixing it will be a little tricky because all the extent accounting assumes the inode_add_bytes happens at extent insertion time. -chris > > > The result of the test at 2.6.37-rc4 is shown below. > (see inode no. 291) > > # df -T /test14 > Filesystem Type 1K-blocks Used Available Use% Mounted on > /dev/sdd14 btrfs 4162560 8736 3709440 1% /test14 > # dd if=/dev/zero of=/test14/dir/as001.26603 bs=1M count=100 > # dd if=/dev/zero of=/test14/dir/as002.26603 bs=1M count=200 > # dd if=/dev/zero of=/test14/dir/sy001.26603 bs=1M count=300 oflag=direct > # dd if=/dev/zero of=/test14/dir/as003.26603 bs=1M count=400 > # ls -lis /test14/dir > total 406528 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 406528 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 99328 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 307200 > 288 0 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sleep 3 > # ls -lis /test14/dir > total 409600 > 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 0 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 0 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > # sync > # ls -lis /test14/dir > total 1024000 > 288 102400 -rw-r--r-- 1 root root 104857600 Dec 7 15:07 as001.26603 > 289 204800 -rw-r--r-- 1 root root 209715200 Dec 7 15:07 as002.26603 > -> 291 409600 -rw-r--r-- 1 root root 419430400 Dec 7 15:08 as003.26603 > 290 307200 -rw-r--r-- 1 root root 314572800 Dec 7 15:08 sy001.26603 > > The trace result of btrfs_getattr() is shown below. > > Dec 7 15:08:03 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 > Dec 7 15:08:06 luna kernel: ino:291 blocks:198656 i_blocks:0 i_bytes:0 delalloc_bytes:101711872 > Dec 7 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 > Dec 7 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 i_bytes:0 delalloc_bytes:0 > Dec 7 15:08:18 luna kernel: ino:291 blocks:819200 i_blocks:819200 i_bytes:0 delalloc_bytes:0 > > > Regards, > Itoh > ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 18:44 ` Chris Mason @ 2010-12-07 19:16 ` Mike Fedyk 2010-12-07 19:29 ` Chris Mason 0 siblings, 1 reply; 11+ messages in thread From: Mike Fedyk @ 2010-12-07 19:16 UTC (permalink / raw) To: Chris Mason; +Cc: Tsutomu Itoh, Linux Btrfs On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.com> w= rote: > Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500: >> Hi, >> >> I think that the disk allocation size of each file becomes a monoton= e increase >> when the file is made. >> But, it sometimes return to 0. =C2=A0Is it correct? > > Well, there's a window during the processing of delayed allocation wh= ere > we don't have the bytes recorded as delalloc and we don't have the by= tes > recorded in the inode yet. =C2=A0That's why they are showing up as ze= ro. > > We don't call inode_add_bytes() until after we insert the extent, but= we > drop the delalloc byte count on the file before the IO is done. > > Fixing it will be a little tricky because all the extent accounting > assumes the inode_add_bytes happens at extent insertion time. > How does opening the inode with O_APPEND during this window know where to write the bytes? If it's a pointer/cursor to the EOF then that size could be used during the window. Is that right? >> >> >> The result of the test at 2.6.37-rc4 is shown below. >> (see inode no. 291) >> >> =C2=A0 =C2=A0 # df -T /test14 >> =C2=A0 =C2=A0 Filesystem =C2=A0 =C2=A0Type =C2=A0 1K-blocks =C2=A0 =C2= =A0 =C2=A0Used Available Use% Mounted on >> =C2=A0 =C2=A0 /dev/sdd14 =C2=A0 btrfs =C2=A0 =C2=A0 4162560 =C2=A0 =C2= =A0 =C2=A08736 =C2=A0 3709440 =C2=A0 1% /test14 >> =C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as001.26603 bs=3D= 1M count=3D100 >> =C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as002.26603 bs=3D= 1M count=3D200 >> =C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/sy001.26603 bs=3D= 1M count=3D300 oflag=3Ddirect >> =C2=A0 =C2=A0 # dd if=3D/dev/zero of=3D/test14/dir/as003.26603 bs=3D= 1M count=3D400 >> =C2=A0 =C2=A0 # ls -lis /test14/dir >> =C2=A0 =C2=A0 total 406528 >> =C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485= 7600 Dec =C2=A07 15:07 as001.26603 >> =C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971= 5200 Dec =C2=A07 15:07 as002.26603 >> =C2=A0-> 291 =C2=A099328 -rw-r--r-- 1 root root 419430400 Dec =C2=A0= 7 15:08 as003.26603 >> =C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0= 7 15:08 sy001.26603 >> =C2=A0 =C2=A0 # sleep 3 >> =C2=A0 =C2=A0 # ls -lis /test14/dir >> =C2=A0 =C2=A0 total 406528 >> =C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485= 7600 Dec =C2=A07 15:07 as001.26603 >> =C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971= 5200 Dec =C2=A07 15:07 as002.26603 >> =C2=A0-> 291 =C2=A099328 -rw-r--r-- 1 root root 419430400 Dec =C2=A0= 7 15:08 as003.26603 >> =C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0= 7 15:08 sy001.26603 >> =C2=A0 =C2=A0 # sleep 3 >> =C2=A0 =C2=A0 # ls -lis /test14/dir >> =C2=A0 =C2=A0 total 307200 >> =C2=A0 =C2=A0 288 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 10485= 7600 Dec =C2=A07 15:07 as001.26603 >> =C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971= 5200 Dec =C2=A07 15:07 as002.26603 >> =C2=A0-> 291 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 419430400 = Dec =C2=A07 15:08 as003.26603 >> =C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0= 7 15:08 sy001.26603 >> =C2=A0 =C2=A0 # sleep 3 >> =C2=A0 =C2=A0 # ls -lis /test14/dir >> =C2=A0 =C2=A0 total 409600 >> =C2=A0 =C2=A0 288 102400 -rw-r--r-- 1 root root 104857600 Dec =C2=A0= 7 15:07 as001.26603 >> =C2=A0 =C2=A0 289 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 20971= 5200 Dec =C2=A07 15:07 as002.26603 >> =C2=A0-> 291 =C2=A0 =C2=A0 =C2=A00 -rw-r--r-- 1 root root 419430400 = Dec =C2=A07 15:08 as003.26603 >> =C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0= 7 15:08 sy001.26603 >> =C2=A0 =C2=A0 # sync >> =C2=A0 =C2=A0 # ls -lis /test14/dir >> =C2=A0 =C2=A0 total 1024000 >> =C2=A0 =C2=A0 288 102400 -rw-r--r-- 1 root root 104857600 Dec =C2=A0= 7 15:07 as001.26603 >> =C2=A0 =C2=A0 289 204800 -rw-r--r-- 1 root root 209715200 Dec =C2=A0= 7 15:07 as002.26603 >> =C2=A0-> 291 409600 -rw-r--r-- 1 root root 419430400 Dec =C2=A07 15:= 08 as003.26603 >> =C2=A0 =C2=A0 290 307200 -rw-r--r-- 1 root root 314572800 Dec =C2=A0= 7 15:08 sy001.26603 >> >> The trace result of btrfs_getattr() is shown below. >> >> =C2=A0Dec =C2=A07 15:08:03 luna kernel: ino:291 blocks:198656 i_bloc= ks:0 i_bytes:0 delalloc_bytes:101711872 >> =C2=A0Dec =C2=A07 15:08:06 luna kernel: ino:291 blocks:198656 i_bloc= ks:0 i_bytes:0 delalloc_bytes:101711872 >> =C2=A0Dec =C2=A07 15:08:09 luna kernel: ino:291 blocks:0 i_blocks:0 = i_bytes:0 delalloc_bytes:0 >> =C2=A0Dec =C2=A07 15:08:12 luna kernel: ino:291 blocks:0 i_blocks:0 = i_bytes:0 delalloc_bytes:0 >> =C2=A0Dec =C2=A07 15:08:18 luna kernel: ino:291 blocks:819200 i_bloc= ks:819200 i_bytes:0 delalloc_bytes:0 >> >> >> Regards, >> Itoh >> > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs= " in > the body of a message to majordomo@vger.kernel.org > More majordomo info at =C2=A0http://vger.kernel.org/majordomo-info.ht= ml > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 19:16 ` Mike Fedyk @ 2010-12-07 19:29 ` Chris Mason 2010-12-07 20:07 ` Mike Fedyk 0 siblings, 1 reply; 11+ messages in thread From: Chris Mason @ 2010-12-07 19:29 UTC (permalink / raw) To: Mike Fedyk; +Cc: Tsutomu Itoh, Linux Btrfs Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500: > On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.com>= wrote: > > Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500: > >> Hi, > >> > >> I think that the disk allocation size of each file becomes a monot= one increase > >> when the file is made. > >> But, it sometimes return to 0. =C2=A0Is it correct? > > > > Well, there's a window during the processing of delayed allocation = where > > we don't have the bytes recorded as delalloc and we don't have the = bytes > > recorded in the inode yet. =C2=A0That's why they are showing up as = zero. > > > > We don't call inode_add_bytes() until after we insert the extent, b= ut we > > drop the delalloc byte count on the file before the IO is done. > > > > Fixing it will be a little tricky because all the extent accounting > > assumes the inode_add_bytes happens at extent insertion time. > > >=20 > How does opening the inode with O_APPEND during this window know wher= e > to write the bytes? If it's a pointer/cursor to the EOF then that > size could be used during the window. Is that right? This counter records the number of blocks allocated to the file, and reading it with ls -l or stat is somewhat racey by nature. Most of the time its fine, btrfs just has a really big window where the results fro= m ls -l seem wrong. But, the counter really means nothing to the btrfs internals. When we do file operations we go based on the extent pointers we find in the tree and i_size (i_size is strictly maintained). The incorrect results are confusing but they don't hurt the metadata itself. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 19:29 ` Chris Mason @ 2010-12-07 20:07 ` Mike Fedyk 2010-12-07 20:15 ` Chris Mason 0 siblings, 1 reply; 11+ messages in thread From: Mike Fedyk @ 2010-12-07 20:07 UTC (permalink / raw) To: Chris Mason; +Cc: Tsutomu Itoh, Linux Btrfs On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <chris.mason@oracle.com> w= rote: > Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500: >> On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.com= > wrote: >> > Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500: >> >> Hi, >> >> >> >> I think that the disk allocation size of each file becomes a mono= tone increase >> >> when the file is made. >> >> But, it sometimes return to 0. =C2=A0Is it correct? >> > >> > Well, there's a window during the processing of delayed allocation= where >> > we don't have the bytes recorded as delalloc and we don't have the= bytes >> > recorded in the inode yet. =C2=A0That's why they are showing up as= zero. >> > >> > We don't call inode_add_bytes() until after we insert the extent, = but we >> > drop the delalloc byte count on the file before the IO is done. >> > >> > Fixing it will be a little tricky because all the extent accountin= g >> > assumes the inode_add_bytes happens at extent insertion time. >> > >> >> How does opening the inode with O_APPEND during this window know whe= re >> to write the bytes? =C2=A0If it's a pointer/cursor to the EOF then t= hat >> size could be used during the window. =C2=A0Is that right? > > This counter records the number of blocks allocated to the file, and > reading it with ls -l or stat is somewhat racey by nature. =C2=A0Most= of the > time its fine, btrfs just has a really big window where the results f= rom > ls -l seem wrong. > I see. Is it using per-cpu vars or something similar? > But, the counter really means nothing to the btrfs internals. =C2=A0W= hen we > do file operations we go based on the extent pointers we find in the > tree and i_size (i_size is strictly maintained). > Would it be too heavy of an operation to have stat walk the btrfs tree to get its data? > The incorrect results are confusing but they don't hurt the metadata > itself. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 20:07 ` Mike Fedyk @ 2010-12-07 20:15 ` Chris Mason 2010-12-07 22:06 ` Mike Fedyk 2010-12-08 0:15 ` Tsutomu Itoh 0 siblings, 2 replies; 11+ messages in thread From: Chris Mason @ 2010-12-07 20:15 UTC (permalink / raw) To: Mike Fedyk; +Cc: Tsutomu Itoh, Linux Btrfs Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500: > On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <chris.mason@oracle.com>= wrote: > > Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500: > >> On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.c= om> wrote: > >> > Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -050= 0: > >> >> Hi, > >> >> > >> >> I think that the disk allocation size of each file becomes a mo= notone increase > >> >> when the file is made. > >> >> But, it sometimes return to 0. =C2=A0Is it correct? > >> > > >> > Well, there's a window during the processing of delayed allocati= on where > >> > we don't have the bytes recorded as delalloc and we don't have t= he bytes > >> > recorded in the inode yet. =C2=A0That's why they are showing up = as zero. > >> > > >> > We don't call inode_add_bytes() until after we insert the extent= , but we > >> > drop the delalloc byte count on the file before the IO is done. > >> > > >> > Fixing it will be a little tricky because all the extent account= ing > >> > assumes the inode_add_bytes happens at extent insertion time. > >> > > >> > >> How does opening the inode with O_APPEND during this window know w= here > >> to write the bytes? =C2=A0If it's a pointer/cursor to the EOF then= that > >> size could be used during the window. =C2=A0Is that right? > > > > This counter records the number of blocks allocated to the file, an= d > > reading it with ls -l or stat is somewhat racey by nature. =C2=A0Mo= st of the > > time its fine, btrfs just has a really big window where the results= from > > ls -l seem wrong. > > >=20 > I see. Is it using per-cpu vars or something similar? Our stat function returns the block count in the inode plus the number of bytes we have accounted as delayed allocation. As we do writes to the file, the delayed allocation count goes up and then eventually we decide we need to do some IO. Before we do the IO, we have to decide where on the disk to write the extents. Once that is decided, we decrement the count of delayed allocation bytes. This is when stat starts returning the wrong answer. Then we do the IO, and when the IO is done we actually insert the file extents into the file metadata. This is when stat starts returning the right answer again. The whole setup sounds strange, but this is how btrfs implements the semantics from data=3Dordered. We don't update the file to point to the new blocks until after the IO is done, so we never have to wait on the data IO before we can do a transaction commit. It avoids all kinds of latencies with fsync and other problems. One easy solution is to just add another counter in the in-memory inode for the number of bytes in flight that aren't accounted for in other places. But I'd rather not make the inode any bigger, so I'll have to think if we can solve this another way. >=20 > > But, the counter really means nothing to the btrfs internals. =C2=A0= When we > > do file operations we go based on the extent pointers we find in th= e > > tree and i_size (i_size is strictly maintained). > > >=20 > Would it be too heavy of an operation to have stat walk the btrfs tre= e > to get its data? >=20 I'm afraid so, stat is fairly performance critical. -chris -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 20:15 ` Chris Mason @ 2010-12-07 22:06 ` Mike Fedyk 2010-12-08 0:15 ` Tsutomu Itoh 1 sibling, 0 replies; 11+ messages in thread From: Mike Fedyk @ 2010-12-07 22:06 UTC (permalink / raw) To: Chris Mason; +Cc: Tsutomu Itoh, Linux Btrfs On Tue, Dec 7, 2010 at 12:15 PM, Chris Mason <chris.mason@oracle.com> w= rote: > Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500: >> On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <chris.mason@oracle.com= > wrote: >> > Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500: >> >> On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.= com> wrote: >> >> > Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -05= 00: >> >> >> Hi, >> >> >> >> >> >> I think that the disk allocation size of each file becomes a m= onotone increase >> >> >> when the file is made. >> >> >> But, it sometimes return to 0. =C2=A0Is it correct? >> >> > >> >> > Well, there's a window during the processing of delayed allocat= ion where >> >> > we don't have the bytes recorded as delalloc and we don't have = the bytes >> >> > recorded in the inode yet. =C2=A0That's why they are showing up= as zero. >> >> > >> >> > We don't call inode_add_bytes() until after we insert the exten= t, but we >> >> > drop the delalloc byte count on the file before the IO is done. >> >> > >> >> > Fixing it will be a little tricky because all the extent accoun= ting >> >> > assumes the inode_add_bytes happens at extent insertion time. >> >> > >> >> >> >> How does opening the inode with O_APPEND during this window know = where >> >> to write the bytes? =C2=A0If it's a pointer/cursor to the EOF the= n that >> >> size could be used during the window. =C2=A0Is that right? >> > >> > This counter records the number of blocks allocated to the file, a= nd >> > reading it with ls -l or stat is somewhat racey by nature. =C2=A0M= ost of the >> > time its fine, btrfs just has a really big window where the result= s from >> > ls -l seem wrong. >> > >> >> I see. =C2=A0Is it using per-cpu vars or something similar? > Ok, so to make sure I fully understand I'm going to make some psuedo code based on your description. > Our stat function returns the block count in the inode plus the numbe= r > of bytes we have accounted as delayed allocation. > stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes > As we do writes to the file, the delayed allocation count goes up and > then eventually we decide we need to do some IO. > > Before we do the IO, we have to decide where on the disk to write the > extents. inode_a2 =3D inode_a1 inode_a1 and inode_a2 are the same inode, but inode_a2 has a different list of extents and is not written yet (in the case of appending, most of the extents will be the same in the two extent lists, but inode_a2 will have more extents for the newly appended data) > Once that is decided, we decrement the count of delayed > allocation bytes. > > This is when stat starts returning the wrong answer. > inode_a2.bytes +=3D inode_a1_delayed_allocation_bytes inode_a1_delayed_allocation_bytes -=3D inode_a1_delayed_allocation_byte= s stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes Is it possible to have stat read from inode_a2 during this window? So it would be instead: stat =3D inode_a2.bytes > Then we do the IO, and when the IO is done we actually insert the fil= e > extents into the file metadata. =C2=A0This is when stat starts return= ing the > right answer again. > /* implicit when write completes */ inode_a1 =3D inode_a2 kfree(inode_a2) stat =3D inode_a1.bytes + inode_a1_delayed_allocation_bytes > The whole setup sounds strange, but this is how btrfs implements the > semantics from data=3Dordered. =C2=A0We don't update the file to poin= t to > the new blocks until after the IO is done, so we never have to wait o= n > the data IO before we can do a transaction commit. =C2=A0It avoids al= l kinds > of latencies with fsync and other problems. > > One easy solution is to just add another counter in the in-memory ino= de > for the number of bytes in flight that aren't accounted for in other > places. =C2=A0But I'd rather not make the inode any bigger, so I'll h= ave to > think if we can solve this another way. > -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" = in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html ^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: The value displayed by 'ls -s' command is strange. 2010-12-07 20:15 ` Chris Mason 2010-12-07 22:06 ` Mike Fedyk @ 2010-12-08 0:15 ` Tsutomu Itoh 1 sibling, 0 replies; 11+ messages in thread From: Tsutomu Itoh @ 2010-12-08 0:15 UTC (permalink / raw) To: Chris Mason; +Cc: Mike Fedyk, Linux Btrfs (2010/12/08 5:15), Chris Mason wrote: > Excerpts from Mike Fedyk's message of 2010-12-07 15:07:08 -0500: >> On Tue, Dec 7, 2010 at 11:29 AM, Chris Mason <chris.mason@oracle.com> wrote: >>> Excerpts from Mike Fedyk's message of 2010-12-07 14:16:55 -0500: >>>> On Tue, Dec 7, 2010 at 10:44 AM, Chris Mason <chris.mason@oracle.com> wrote: >>>>> Excerpts from Tsutomu Itoh's message of 2010-12-07 02:59:52 -0500: >>>>>> Hi, >>>>>> >>>>>> I think that the disk allocation size of each file becomes a monotone increase >>>>>> when the file is made. >>>>>> But, it sometimes return to 0. Is it correct? >>>>> >>>>> Well, there's a window during the processing of delayed allocation where >>>>> we don't have the bytes recorded as delalloc and we don't have the bytes >>>>> recorded in the inode yet. That's why they are showing up as zero. >>>>> >>>>> We don't call inode_add_bytes() until after we insert the extent, but we >>>>> drop the delalloc byte count on the file before the IO is done. >>>>> >>>>> Fixing it will be a little tricky because all the extent accounting >>>>> assumes the inode_add_bytes happens at extent insertion time. >>>>> >>>> >>>> How does opening the inode with O_APPEND during this window know where >>>> to write the bytes? If it's a pointer/cursor to the EOF then that >>>> size could be used during the window. Is that right? >>> >>> This counter records the number of blocks allocated to the file, and >>> reading it with ls -l or stat is somewhat racey by nature. Most of the >>> time its fine, btrfs just has a really big window where the results from >>> ls -l seem wrong. >>> >> >> I see. Is it using per-cpu vars or something similar? > > Our stat function returns the block count in the inode plus the number > of bytes we have accounted as delayed allocation. > > As we do writes to the file, the delayed allocation count goes up and > then eventually we decide we need to do some IO. > > Before we do the IO, we have to decide where on the disk to write the > extents. Once that is decided, we decrement the count of delayed > allocation bytes. > > This is when stat starts returning the wrong answer. > > Then we do the IO, and when the IO is done we actually insert the file > extents into the file metadata. This is when stat starts returning the > right answer again. I understood. However, I worry that the user is confused because the wrong condition is too long. > > The whole setup sounds strange, but this is how btrfs implements the > semantics from data=ordered. We don't update the file to point to > the new blocks until after the IO is done, so we never have to wait on > the data IO before we can do a transaction commit. It avoids all kinds > of latencies with fsync and other problems. > > One easy solution is to just add another counter in the in-memory inode > for the number of bytes in flight that aren't accounted for in other > places. But I'd rather not make the inode any bigger, so I'll have to > think if we can solve this another way. > >> >>> But, the counter really means nothing to the btrfs internals. When we >>> do file operations we go based on the extent pointers we find in the >>> tree and i_size (i_size is strictly maintained). >>> >> >> Would it be too heavy of an operation to have stat walk the btrfs tree >> to get its data? >> > > I'm afraid so, stat is fairly performance critical. > > -chris ^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2010-12-09 10:42 UTC | newest] Thread overview: 11+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2010-12-07 7:59 The value displayed by 'ls -s' command is strange Tsutomu Itoh 2010-12-07 9:25 ` Li Zefan 2010-12-07 23:53 ` Tsutomu Itoh 2010-12-09 10:42 ` Miao Xie 2010-12-07 18:44 ` Chris Mason 2010-12-07 19:16 ` Mike Fedyk 2010-12-07 19:29 ` Chris Mason 2010-12-07 20:07 ` Mike Fedyk 2010-12-07 20:15 ` Chris Mason 2010-12-07 22:06 ` Mike Fedyk 2010-12-08 0:15 ` Tsutomu Itoh
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).