Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Edward Shishkin]

[-- Attachment #1: Type: text/plain, Size: 14306 bytes --]

Mr. Tao wrote:
> Hi Edward,
>
>   

Hello.

> I tried to post this through gmane at reiserfs list but somehow I couldn't manage. Perhaps you might find it usefull.
>
>   

Yes, this is an interesting oops that has never occurred before.
The situation is clear: page is not uptodate and disk cluster is
"unprepped".

In accordance with Reiser4 design, unprepped disk clusters are allowed to
be written to disk, however the transactions remain active, as unprepped
dick cluster should be replaced with compressed data. If such replacement
is failed for some reasons (for example, system crash), then the changes
should be rolled back when replaying journal. So, normally, the kernel
shouldn't meet unprepped disk clusters in usual working sessions (when
servicing system calls).

However in this case we can see that journal was not replayed (properly),
so I would ask you to recall (if possible) in what circumstances did this
problem happen.

The attached patch will allow you to avoid unnecessary oopses in future.

Also we have encountered a problem that fsck can not handle unprepped
disk clusters properly. There is nothing criminal in having them on
disk, but
the correct solution is to cut them as "dead" metadata. So I'll try to
fix fsck to
handle them properly. Now live with kernel warnings ("edward-1570") for a
while.. Ok?

Thanks,
Edward.

> This is what i got on fres new install running zen 2.6.26-rc8-zen1.0
>
> System is gentoo amd64 on Pentium 4 with 2.5 GB RAM.
>
> FS is accessible and system boots off it, but when fscking it, fsck.reiser4 exits prematurely with signal 11. This happens on 2 (root and portage) partitions of three I have running R4. It's no different if I fsck it from SystemRescueCD.
>
> [  674.044908] ------------[ cut here ]------------
> [  674.044908] kernel BUG at fs/reiser4/plugin/item/ctail.c:672!
> [  674.044908] invalid opcode: 0000 [1] PREEMPT SMP
> [  674.044908] CPU 0
> [  674.044908] Modules linked in: libafs(P) lm85 hwmon_vid snd_pcm_oss snd_mixer_oss fuse parport_pc parport snd_hda_intel snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc fglrx(P)
> [  674.044908] Pid: 8848, comm: svn Tainted: P          2.6.26-rc8-zen1.0-makalinux #2
> [  674.044908] RIP: 0010:[<ffffffff80331437>]  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [  674.044908] RSP: 0018:ffff8100641ef9e8  EFLAGS: 00010246
> [  674.044908] RAX: 00000000000003b0 RBX: 0000000000000000 RCX: 0000000000000000
> [  674.044908] RDX: 00000000000003b0 RSI: 00000000000003b0 RDI: 0000000000000000
> [  674.044908] RBP: 00000000000003b0 R08: 0000000000000020 R09: ffff810063cfd5f0
> [  674.044908] R10: ffff8100641efae8 R11: ffffffff80329807 R12: ffffe20001a4a598
> [  674.044908] R13: 0000000000000002 R14: ffff8100641efae8 R15: ffff810063cfd5f0
> [  674.044908] FS:  00007f37d91ba740(0000) GS:ffffffff80816000(0000) knlGS:0000000000000000
> [  674.044908] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [  674.044908] CR2: 00000030df60bd61 CR3: 00000000641e7000 CR4: 00000000000006e0
> [  674.044908] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [  674.044908] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [  674.044908] Process svn (pid: 8848, threadinfo ffff8100641ee000, task ffff8100738385a0)
> [  674.044908] Stack:  ffffffff8089c380 0000000080325578 ffff810063cfd5f0 ffff810073544500
> [  674.044908]  0000000100000001 ffffe20001a4a598 ffff8100641efae8 0000000000000000
> [  674.044908]  0000000000000000 0000000000000000 0000000000000000 ffffffff803316d0
> [  674.044908] Call Trace:
> [  674.044908]  [<ffffffff803316d0>] ? ctail_readpages_filler+0x19a/0x1f2
> [  674.044908]  [<ffffffff80272cc9>] ? lru_cache_add+0x66/0x84
> [  674.044908]  [<ffffffff80331536>] ? ctail_readpages_filler+0x0/0x1f2
> [  674.044908]  [<ffffffff802720ea>] ? read_cache_pages+0x70/0x93
> [  674.044908]  [<ffffffff80331124>] ? readpages_ctail+0x33b/0x343
> [  674.044908]  [<ffffffff80324b14>] ? readpages_cryptcompress+0x3b/0x64
> [  674.044908]  [<ffffffff80271f0e>] ? __do_page_cache_readahead+0x142/0x1cf
> [  674.044908]  [<ffffffff802722af>] ? ondemand_readahead+0x1a2/0x1b4
> [  674.044908]  [<ffffffff8026ca83>] ? generic_file_aio_read+0x1d0/0x4d1
> [  674.044908]  [<ffffffff8029247a>] ? do_sync_read+0xce/0x113
> [  674.044908]  [<ffffffff80601c4d>] ? _spin_unlock_irqrestore+0x12/0x2d
> [  674.044908]  [<ffffffff80243144>] ? autoremove_wake_function+0x0/0x2e
> [  674.044908]  [<ffffffff80290fd7>] ? __dentry_open+0x198/0x281
> [  674.044908]  [<ffffffff80601cd5>] ? _spin_unlock+0x10/0x2a
> [  674.044908]  [<ffffffff80308a57>] ? reiser4_grab+0x97/0xa1
> [  674.044908]  [<ffffffff80324756>] ? read_cryptcompress+0x6d/0x96
> [  674.044908]  [<ffffffff80323085>] ? reiser4_read_careful+0xc9/0x121
> [  674.044908]  [<ffffffff80292dc5>] ? vfs_read+0xaa/0x132
> [  674.044908]  [<ffffffff80292f09>] ? sys_read+0x45/0x6e
> [  674.044908]  [<ffffffff8020b25b>] ? system_call_after_swapgs+0x7b/0x80
> [  674.044908]
> [  674.044908]
> [  674.044908] Code: 00 00 45 8b ae 88 00 00 00 41 83 fd 01 0f 84 84 00 00 00 41 83 fd 02 74 12 41 8d 45 fd 45 31 ed 83 f8 01 0f 87 de 00 00 00 eb 04 <0f> 0b eb fe bf 01 00 00 00 e8 34 32 2d 00 48 bf 00 00 00 00 00
> [  674.044908] RIP  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [  674.044908]  RSP <ffff8100641ef9e8>
> [  674.045313] ---[ end trace 6d41c2445da53cc4 ]---
> [  674.045778] reiser4[svn(8848)]: release_unix_file (fs/reiser4/plugin/file/file.c:2365)[vs-44]:
> [  674.045781] WARNING: out of memory?
> [  739.980542] evolution[8886]: segfault at 8 ip 7f37f5b9f46e sp 42620fa0 error 4 in libcamelimap.so[7f37f5b8c000+1f000]
> [  829.478811] evolution[8974]: segfault at 8 ip 7ff8c08e346e sp 42599fa0 error 4 in libcamelimap.so[7ff8c08d0000+1f000]
> [  936.087105] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 1076.022618] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 1389.091485] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 2091.023477] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 2983.204399] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 3110.586708] ------------[ cut here ]------------
> [ 3110.586708] kernel BUG at fs/reiser4/plugin/item/ctail.c:672!
> [ 3110.586708] invalid opcode: 0000 [2] PREEMPT SMP
> [ 3110.586708] CPU 0
> [ 3110.586708] Modules linked in: libafs(P) lm85 hwmon_vid snd_pcm_oss snd_mixer_oss fuse parport_pc parport snd_hda_intel snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc fglrx(P)
> [ 3110.586708] Pid: 10078, comm: paludis Tainted: P      D   2.6.26-rc8-zen1.0-makalinux #2
> [ 3110.586708] RIP: 0010:[<ffffffff80331437>]  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [ 3110.586708] RSP: 0018:ffff810057d619e8  EFLAGS: 00010246
> [ 3110.586708] RAX: 0000000000000265 RBX: 0000000000000000 RCX: 0000000000000000
> [ 3110.586708] RDX: 0000000000000265 RSI: 0000000000000265 RDI: 0000000000000000
> [ 3110.586708] RBP: 0000000000000265 R08: 0000000000000018 R09: ffff810069e6d970
> [ 3110.586708] R10: ffff810057d61ae8 R11: ffffffff80329807 R12: ffffe20001363ea8
> [ 3110.586708] R13: 0000000000000002 R14: ffff810057d61ae8 R15: ffff810069e6d970
> [ 3110.586708] FS:  00007f1ec34df700(0000) GS:ffffffff80816000(0000) knlGS:0000000000000000
> [ 3110.586708] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3110.586708] CR2: 0000000001bd4328 CR3: 0000000057d7a000 CR4: 00000000000006e0
> [ 3110.586708] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 3110.586708] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 3110.586708] Process paludis (pid: 10078, threadinfo ffff810057d60000, task ffff81007383a1c0)
> [ 3110.586708] Stack:  ffff810057d61c88 0000000080325578 ffff810069e6d970 ffff8100570a0880
> [ 3110.586708]  0000000100000001 ffffe20001363ea8 ffff810057d61ae8 0000000000000000
> [ 3110.586708]  0000000000000000 0000000000000000 0000000000000000 ffffffff803316d0
> [ 3110.586708] Call Trace:
> [ 3110.586708]  [<ffffffff803316d0>] ? ctail_readpages_filler+0x19a/0x1f2
> [ 3110.586708]  [<ffffffff80272cc9>] ? lru_cache_add+0x66/0x84
> [ 3110.586708]  [<ffffffff80331536>] ? ctail_readpages_filler+0x0/0x1f2
> [ 3110.586708]  [<ffffffff802720ea>] ? read_cache_pages+0x70/0x93
> [ 3110.586708]  [<ffffffff80331124>] ? readpages_ctail+0x33b/0x343
> [ 3110.586708]  [<ffffffff80324b14>] ? readpages_cryptcompress+0x3b/0x64
> [ 3110.586708]  [<ffffffff80271f0e>] ? __do_page_cache_readahead+0x142/0x1cf
> [ 3110.586708]  [<ffffffff802722af>] ? ondemand_readahead+0x1a2/0x1b4
> [ 3110.586708]  [<ffffffff8026ca83>] ? generic_file_aio_read+0x1d0/0x4d1
> [ 3110.586708]  [<ffffffff8029247a>] ? do_sync_read+0xce/0x113
> [ 3110.586708]  [<ffffffff8027adf0>] ? handle_mm_fault+0x6cf/0x6ec
> [ 3110.586708]  [<ffffffff80243144>] ? autoremove_wake_function+0x0/0x2e
> [ 3110.586708]  [<ffffffff80604254>] ? do_page_fault+0x505/0x8b6
> [ 3110.586708]  [<ffffffff80601cd5>] ? _spin_unlock+0x10/0x2a
> [ 3110.586708]  [<ffffffff80308a57>] ? reiser4_grab+0x97/0xa1
> [ 3110.586708]  [<ffffffff80324756>] ? read_cryptcompress+0x6d/0x96
> [ 3110.586708]  [<ffffffff80323085>] ? reiser4_read_careful+0xc9/0x121
> [ 3110.586708]  [<ffffffff80292dc5>] ? vfs_read+0xaa/0x132
> [ 3110.586708]  [<ffffffff80292f09>] ? sys_read+0x45/0x6e
> [ 3110.586708]  [<ffffffff8020b25b>] ? system_call_after_swapgs+0x7b/0x80
> [ 3110.586708]
> [ 3110.586708]
> [ 3110.586708] Code: 00 00 45 8b ae 88 00 00 00 41 83 fd 01 0f 84 84 00 00 00 41 83 fd 02 74 12 41 8d 45 fd 45 31 ed 83 f8 01 0f 87 de 00 00 00 eb 04 <0f> 0b eb fe bf 01 00 00 00 e8 34 32 2d 00 48 bf 00 00 00 00 00
> [ 3110.586708] RIP  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [ 3110.586708]  RSP <ffff810057d619e8>
> [ 3110.586708] ---[ end trace 6d41c2445da53cc4 ]---
> [ 3127.017668] afs: byte-range lock/unlock ignored; make sure no one else is running this program.
> [ 3238.628906] gvfs-fuse-daemo[6459]: segfault at 3000000000 ip 30dde0b237 sp 7fff3e975c98 error 6 in libpthread-2.8.so[30dde00000+16000]
> [ 3247.321819] mtrr: type mismatch for d8000000,8000000 old: write-back new: write-combining
> [ 3247.321831] [fglrx:firegl_addmap] *ERROR* mtrr allocation failed (-22)
> [ 3247.324921] [fglrx] Reserved FB block: Shared offset:0, size:40000
> [ 3247.324929] [fglrx] Reserved FB block: Unshared offset:7fbf000, size:40000
> [ 3247.324937] [fglrx] Reserved FB block: Unshared offset:7fff000, size:1000
> [ 3384.566804] ------------[ cut here ]------------
> [ 3384.566934] kernel BUG at fs/reiser4/plugin/item/ctail.c:672!
> [ 3384.567059] invalid opcode: 0000 [3] PREEMPT SMP
> [ 3384.567268] CPU 0
> [ 3384.567285] Modules linked in: libafs(P) lm85 hwmon_vid snd_pcm_oss snd_mixer_oss fuse parport_pc parport snd_hda_intel snd_pcm snd_timer snd soundcore i2c_i801 snd_page_alloc fglrx(P)
> [ 3384.567285] Pid: 10666, comm: reconcilio Tainted: P      D   2.6.26-rc8-zen1.0-makalinux #2
> [ 3384.567285] RIP: 0010:[<ffffffff80331437>]  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [ 3384.567285] RSP: 0018:ffff81007241d9e8  EFLAGS: 00010246
> [ 3384.567285] RAX: 0000000000000005 RBX: 0000000000000000 RCX: 0000000000000000
> [ 3384.567285] RDX: 0000000000001000 RSI: 0000000000005ddc RDI: 0000000000000005
> [ 3384.567285] RBP: 0000000000001000 R08: 0000000000000018 R09: ffff8100669e3570
> [ 3384.567285] R10: ffff81007241dae8 R11: ffffffff80329807 R12: ffffe20001e7f798
> [ 3384.567285] R13: 0000000000000002 R14: ffff81007241dae8 R15: ffff8100669e3570
> [ 3384.567285] FS:  00007fc18ba33700(0000) GS:ffffffff80816000(0000) knlGS:0000000000000000
> [ 3384.567285] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 3384.567285] CR2: 00007fc18ad39058 CR3: 00000000614b1000 CR4: 00000000000006e0
> [ 3384.567285] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 3384.567285] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> [ 3384.567285] Process reconcilio (pid: 10666, threadinfo ffff81007241c000, task ffff81007383e540)
> [ 3384.567285] Stack:  0000000000000000 0000000080325578 ffff8100669e3570 ffff810073544c00
> [ 3384.567285]  0000000100000006 ffffe20001e7f798 ffff81007241dae8 0000000000000000
> [ 3384.567285]  0000000000000000 0000000000000000 0000000000000000 ffffffff803316d0
> [ 3384.567285] Call Trace:
> [ 3384.567285]  [<ffffffff803316d0>] ? ctail_readpages_filler+0x19a/0x1f2
> [ 3384.567285]  [<ffffffff80272cc9>] ? lru_cache_add+0x66/0x84
> [ 3384.567285]  [<ffffffff80331536>] ? ctail_readpages_filler+0x0/0x1f2
> [ 3384.567285]  [<ffffffff802720ea>] ? read_cache_pages+0x70/0x93
> [ 3384.567285]  [<ffffffff80331124>] ? readpages_ctail+0x33b/0x343
> [ 3384.567285]  [<ffffffff80324b14>] ? readpages_cryptcompress+0x3b/0x64
> [ 3384.567285]  [<ffffffff80271f0e>] ? __do_page_cache_readahead+0x142/0x1cf
> [ 3384.567285]  [<ffffffff802722af>] ? ondemand_readahead+0x1a2/0x1b4
> [ 3384.567285]  [<ffffffff8026ca83>] ? generic_file_aio_read+0x1d0/0x4d1
> [ 3384.567285]  [<ffffffff8029247a>] ? do_sync_read+0xce/0x113
> [ 3384.567285]  [<ffffffff80601c4d>] ? _spin_unlock_irqrestore+0x12/0x2d
> [ 3384.567285]  [<ffffffff80243144>] ? autoremove_wake_function+0x0/0x2e
> [ 3384.567285]  [<ffffffff80290fd7>] ? __dentry_open+0x198/0x281
> [ 3384.567285]  [<ffffffff80601cd5>] ? _spin_unlock+0x10/0x2a
> [ 3384.567285]  [<ffffffff80308a57>] ? reiser4_grab+0x97/0xa1
> [ 3384.567285]  [<ffffffff80324756>] ? read_cryptcompress+0x6d/0x96
> [ 3384.567285]  [<ffffffff80323085>] ? reiser4_read_careful+0xc9/0x121
> [ 3384.567285]  [<ffffffff80292dc5>] ? vfs_read+0xaa/0x132
> [ 3384.567285]  [<ffffffff80292f09>] ? sys_read+0x45/0x6e
> [ 3384.567285]  [<ffffffff8020b25b>] ? system_call_after_swapgs+0x7b/0x80
> [ 3384.567285]
> [ 3384.567285]
> [ 3384.567285] Code: 00 00 45 8b ae 88 00 00 00 41 83 fd 01 0f 84 84 00 00 00 41 83 fd 02 74 12 41 8d 45 fd 45 31 ed 83 f8 01 0f 87 de 00 00 00 eb 04 <0f> 0b eb fe bf 01 00 00 00 e8 34 32 2d 00 48 bf 00 00 00 00 00
> [ 3384.567285] RIP  [<ffffffff80331437>] do_readpage_ctail+0x30b/0x40a
> [ 3384.567285]  RSP <ffff81007241d9e8>
> [ 3384.576749] ---[ end trace 6d41c2445da53cc4 ]---
>
>   


[-- Attachment #2: reiser4-use-warning-instead-of-bugon.patch --]
[-- Type: text/x-patch, Size: 788 bytes --]

do_readpage_ctail():
Fixup in handling the case when page is not uptodate and
disk cluster is unprepped:
. fill the page with zeroes;
. replace BUG_ON(1) with warning and suggestion to run fsck.

Signed-off-by: Edward Shishkin <edward.shishkin@gmail.com>
---
 linux-2.6.25-mm1/fs/reiser4/plugin/item/ctail.c |    4 +++-
 1 files changed, 3 insertions(+), 1 deletion(-)


--- linux-2.6.25-mm1/fs/reiser4/plugin/item/ctail.c.orig
+++ linux-2.6.25-mm1/fs/reiser4/plugin/item/ctail.c
@@ -669,7 +669,9 @@
 
 	switch (clust->dstat) {
 	case UNPR_DISK_CLUSTER:
-		BUG_ON(1);
+		warning("edward-1570",
+                        "Inode %llu : Invalid disk cluster (%lu). Fsck?",
+			(unsigned long long)get_inode_oid(inode), clust->index);
 	case TRNC_DISK_CLUSTER:
 		/*
 		 * Race with truncate!

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Edward Shishkin]

Mr. Tao wrote:
> I just wanted to add sample of fsck failure:
>
> [103a3:2e4368616e6765:3ce4f] (ccreg40), node [721], item [10]: item of the wrong
> cluster size (-2147483648) found, Should be (65536).
> zsh: segmentation fault  fsck.reiser4 /dev/sda8
>
>   

Note, that data of logical cluster #249423 of the file (inode 66467) are
lost.
Maybe It makes sense to replace this file (it can be found, for example,
with "find -inum 66467") with a correct one..

Edward.

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Dushan Tcholich]

Would it be possible if there would be a new version of fsck.reiser4
to fix those false positives about wrong size when checking
cryptocompress partitions?
There are some users that reported those and it's not comforting when
your fs spews thousands of errors, even when someone tells you they're
false.

Oh and what about making cryptocompress default format on mkfs.reiser4 ?

Have a nice day
Dushan

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Edward Shishkin]

Dushan Tcholich wrote:
> Would it be possible if there would be a new version of fsck.reiser4
> to fix those false positives about wrong size when checking
> cryptocompress partitions?
>   

Yes,  per-file warnings about wrong i_bytes should be suppressed.
Fsck just should report at the end of work in default mode, that N wrong
i_bytes were detected and suggest to fix it with --fix option.
Anyone care to try to make a patch?

> There are some users that reported those and it's not comforting when
> your fs spews thousands of errors, even when someone tells you they're
> false.
>
> Oh and what about making cryptocompress default format on mkfs.reiser4 ?
>   

mm.. not sure. First, fsck should be fixed to remove unprepped items.
Mr. Tao, do you still have partitions that make fsck segfault?

Thanks,
Edward.

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Dushan Tcholich]

On Mon, Aug 4, 2008 at 4:47 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Dushan Tcholich wrote:
>> Would it be possible if there would be a new version of fsck.reiser4
>> to fix those false positives about wrong size when checking
>> cryptocompress partitions?
>>
>
> Yes,  per-file warnings about wrong i_bytes should be suppressed.
> Fsck just should report at the end of work in default mode, that N wrong
> i_bytes were detected and suggest to fix it with --fix option.
> Anyone care to try to make a patch?
>
But what happens with systems with ccreg and fsck on boot? They would
stop and wait for user interaction.
Or you meant just to print a message about it with recomended solution
and continue.

I'd like to make a patch but dunno how :)

>> There are some users that reported those and it's not comforting when
>> your fs spews thousands of errors, even when someone tells you they're
>> false.
>>
>> Oh and what about making cryptocompress default format on mkfs.reiser4 ?
>>
>
> mm.. not sure. First, fsck should be fixed to remove unprepped items.
Ofcourse.

> Mr. Tao, do you still have partitions that make fsck segfault?
>
> Thanks,
> Edward.
>
Dushan

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Edward Shishkin]

Dushan Tcholich wrote:
> On Mon, Aug 4, 2008 at 4:47 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>   
>> Dushan Tcholich wrote:
>>     
>>> Would it be possible if there would be a new version of fsck.reiser4
>>> to fix those false positives about wrong size when checking
>>> cryptocompress partitions?
>>>
>>>       
>> Yes,  per-file warnings about wrong i_bytes should be suppressed.
>> Fsck just should report at the end of work in default mode, that N wrong
>> i_bytes were detected and suggest to fix it with --fix option.
>> Anyone care to try to make a patch?
>>
>>     
> But what happens with systems with ccreg and fsck on boot? They would
> stop and wait for user interaction.
>   

Why?
Do you have any problems with boot?

> Or you meant just to print a message about it with recomended solution
> and continue.
>   

Everything should be the same except per-file warnings.

> I'd like to make a patch but dunno how :)
>
>   
>>> There are some users that reported those and it's not comforting when
>>> your fs spews thousands of errors, even when someone tells you they're
>>> false.
>>>
>>> Oh and what about making cryptocompress default format on mkfs.reiser4 ?
>>>
>>>       
>> mm.. not sure. First, fsck should be fixed to remove unprepped items.
>>     
> Ofcourse.
>
>   
>> Mr. Tao, do you still have partitions that make fsck segfault?
>>
>> Thanks,
>> Edward.
>>
>>     
> Dushan
>
>   

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Dushan Tcholich]

On Mon, Aug 4, 2008 at 6:50 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Dushan Tcholich wrote:
>> On Mon, Aug 4, 2008 at 4:47 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>
>>> Dushan Tcholich wrote:
>>>
>>>> Would it be possible if there would be a new version of fsck.reiser4
>>>> to fix those false positives about wrong size when checking
>>>> cryptocompress partitions?
>>>>
>>>>
>>> Yes,  per-file warnings about wrong i_bytes should be suppressed.
>>> Fsck just should report at the end of work in default mode, that N wrong
>>> i_bytes were detected and suggest to fix it with --fix option.
>>> Anyone care to try to make a patch?
>>>
>>>
>> But what happens with systems with ccreg and fsck on boot? They would
>> stop and wait for user interaction.
>>
>
> Why?
> Do you have any problems with boot?

I don't have problems.
I just thought that checking for these false positives lasted 10
minutes on my 6GB / so wouldn't this be a little too long if we don't
put some status of some sorts?

>> Or you meant just to print a message about it with recomended solution
>> and continue.
>>
>
> Everything should be the same except per-file warnings.
>
Maybe a better explanation would be better instead just: "600000 small
errors found" :)

>> I'd like to make a patch but dunno how :)
>>
>>
>>>> There are some users that reported those and it's not comforting when
>>>> your fs spews thousands of errors, even when someone tells you they're
>>>> false.
>>>>
>>>> Oh and what about making cryptocompress default format on mkfs.reiser4 ?
>>>>
>>>>
>>> mm.. not sure. First, fsck should be fixed to remove unprepped items.
>>>
>> Ofcourse.
>>
>>
>>> Mr. Tao, do you still have partitions that make fsck segfault?
>>>
>>> Thanks,
>>> Edward.
>>>
>>>
>> Dushan
>>
>>
>
>

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Volker Armin Hemmann]

On Montag, 4. August 2008, Dushan Tcholich wrote:
> On Mon, Aug 4, 2008 at 6:50 PM, Edward Shishkin
>
> <edward.shishkin@gmail.com> wrote:
> > Dushan Tcholich wrote:
> >> On Mon, Aug 4, 2008 at 4:47 PM, Edward Shishkin
> >>
> >> <edward.shishkin@gmail.com> wrote:
> >>> Dushan Tcholich wrote:
> >>>> Would it be possible if there would be a new version of fsck.reiser4
> >>>> to fix those false positives about wrong size when checking
> >>>> cryptocompress partitions?
> >>>
> >>> Yes,  per-file warnings about wrong i_bytes should be suppressed.
> >>> Fsck just should report at the end of work in default mode, that N
> >>> wrong i_bytes were detected and suggest to fix it with --fix option.
> >>> Anyone care to try to make a patch?
> >>
> >> But what happens with systems with ccreg and fsck on boot? They would
> >> stop and wait for user interaction.
> >
> > Why?
> > Do you have any problems with boot?
>
> I don't have problems.
> I just thought that checking for these false positives lasted 10
> minutes on my 6GB / so wouldn't this be a little too long if we don't
> put some status of some sorts?
>
> >> Or you meant just to print a message about it with recomended solution
> >> and continue.
> >
> > Everything should be the same except per-file warnings.
>
> Maybe a better explanation would be better instead just: "600000 small
> errors found" :)
>

that would completly freak out people. How about '60000 false file sizes - 
this is normal with compression'?

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Mr. Tao]

> >>> Dushan Tcholich wrote:
> >>
> >>
> >>> Mr. Tao, do you still have partitions that make fsck segfault?
> >>>
No, sorry, (un)fortunatelly not any more. These errors occured when fscking portage and ccache partitions, which I decided to put in separate disk partitions, so I used mkfs.reiser4 to "correct" them. (In the past I had them as uncompressible loop files on root reiser4 CC partition, but this _always_ (and I've tried a lot of times with different kernels) led to fatal corruption of root fs --> *loop files on R4 is no go* solution).  I also had them as part of root FS, but this approach was also futile and led to corruption of / FS. Seems like (at least at thouse times) R4 couldn't handle very well heavy load, as there are moments with gentoo, where both ccache and portage are under stress.

Re: kernel BUG at fs/reiser4/plugin/item/ctail.c

[Edward Shishkin]

Volker Armin Hemmann wrote:
> On Montag, 4. August 2008, Dushan Tcholich wrote:
>   
>> On Mon, Aug 4, 2008 at 6:50 PM, Edward Shishkin
>>
>> <edward.shishkin@gmail.com> wrote:
>>     
>>> Dushan Tcholich wrote:
>>>       
>>>> On Mon, Aug 4, 2008 at 4:47 PM, Edward Shishkin
>>>>
>>>> <edward.shishkin@gmail.com> wrote:
>>>>         
>>>>> Dushan Tcholich wrote:
>>>>>           
>>>>>> Would it be possible if there would be a new version of fsck.reiser4
>>>>>> to fix those false positives about wrong size when checking
>>>>>> cryptocompress partitions?
>>>>>>             
>>>>> Yes,  per-file warnings about wrong i_bytes should be suppressed.
>>>>> Fsck just should report at the end of work in default mode, that N
>>>>> wrong i_bytes were detected and suggest to fix it with --fix option.
>>>>> Anyone care to try to make a patch?
>>>>>           
>>>> But what happens with systems with ccreg and fsck on boot? They would
>>>> stop and wait for user interaction.
>>>>         
>>> Why?
>>> Do you have any problems with boot?
>>>       
>> I don't have problems.
>> I just thought that checking for these false positives lasted 10
>> minutes on my 6GB / so wouldn't this be a little too long if we don't
>> put some status of some sorts?
>>
>>     
>>>> Or you meant just to print a message about it with recomended solution
>>>> and continue.
>>>>         
>>> Everything should be the same except per-file warnings.
>>>       
>> Maybe a better explanation would be better instead just: "600000 small
>> errors found" :)
>>
>>     
>
>   

There is no big problem here:

Kernel:
-----------
For unix-file plugin:
(1) set i_size and real size which coincide with each other;
(2) don't protect data by checksums (for performance reasons).

For cryptcompress plugin:
(1) set  i_size which usually doesn't coincide with real size;
(2) don' t set real size for performance issues.
(3) set data checksums for every logical cluster;

Fsck:
----------
I. For unix-file plugin:
(1) check if i_size == real size, if no then
     (a) report error and
     (b) handle as corruption

II. For cryptcompress plugin:
(1) check checksums; If wrong, then handle as corruption;
(2) check if i_size == real size, if no, then
     (a) report error;
     (b) set correct real size;

It is clear, that (II.2.a) is not needed. Just because (II.1): checksums is
the best means to detect corruptions.

> that would completly freak out people. How about '60000 false file sizes - 
> this is normal with compression'?
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>   

[reiser4] Point

[Ryan Hope]

Is this todo really done?

 >> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
 >> replace wake_up() with wake_up_process()

There are still a few more wake_up()'s in the code, the following patch 
takes care of 2 of them.

diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
index a164f5a..355548d 100644
--- a/fs/reiser4/entd.c
+++ b/fs/reiser4/entd.c
@@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
  #endif
  	spin_unlock(&ent->guard);
  	if (wake_up_ent)
-		wake_up(&ent->wait);
+		wake_up_process(ent->tsk);
  }

  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
@@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct 
writeback_control *wbc)
  	ent->nr_todo_reqs++;
  	list_add_tail(&rq.link, &ent->todo_list);
  	if (ent->nr_todo_reqs == 1)
-		wake_up(&ent->wait);
+		wake_up_process(ent->tsk);

  	spin_unlock(&ent->guard);

Re: [reiser4] Point

[Edward Shishkin]

Ryan Hope wrote:
> Is this todo really done?

nop

>
> >> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
> >> replace wake_up() with wake_up_process()
>
> There are still a few more wake_up()'s in the code, the following
> patch takes care of 2 of them.

 both are not obvious for me, review is needed..

Edward.

>
> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
> index a164f5a..355548d 100644
> --- a/fs/reiser4/entd.c
> +++ b/fs/reiser4/entd.c
> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>  #endif
>      spin_unlock(&ent->guard);
>      if (wake_up_ent)
> -        wake_up(&ent->wait);
> +        wake_up_process(ent->tsk);
>  }
>
>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
> writeback_control *wbc)
>      ent->nr_todo_reqs++;
>      list_add_tail(&rq.link, &ent->todo_list);
>      if (ent->nr_todo_reqs == 1)
> -        wake_up(&ent->wait);
> +        wake_up_process(ent->tsk);
>
>      spin_unlock(&ent->guard);
>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe
> reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [reiser4] Point

[Ryan Hope]

This works because the struct "ent" is of type "entd_context" which
has members "wait_queue_head_t wait" and "struct task_struct *tsk".
The function wake_up() takes the parameter wait_queue_head_t the
function wake_up_process() takes task_struct... I figured this would
be a legit switch. I have seen no adverse affects yet.

-Ryan

On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Ryan Hope wrote:
>> Is this todo really done?
>
> nop
>
>>
>> >> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>> >> replace wake_up() with wake_up_process()
>>
>> There are still a few more wake_up()'s in the code, the following
>> patch takes care of 2 of them.
>
>  both are not obvious for me, review is needed..
>
> Edward.
>
>>
>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>> index a164f5a..355548d 100644
>> --- a/fs/reiser4/entd.c
>> +++ b/fs/reiser4/entd.c
>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>  #endif
>>      spin_unlock(&ent->guard);
>>      if (wake_up_ent)
>> -        wake_up(&ent->wait);
>> +        wake_up_process(ent->tsk);
>>  }
>>
>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>> writeback_control *wbc)
>>      ent->nr_todo_reqs++;
>>      list_add_tail(&rq.link, &ent->todo_list);
>>      if (ent->nr_todo_reqs == 1)
>> -        wake_up(&ent->wait);
>> +        wake_up_process(ent->tsk);
>>
>>      spin_unlock(&ent->guard);
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> reiserfs-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
>

Re: [reiser4] Point

[Edward Shishkin]

Ryan Hope wrote:
> This works because the struct "ent" is of type "entd_context"

This logic train goes to a wrong direction ;)

>  which
> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
> The function wake_up() takes the parameter wait_queue_head_t the
> function wake_up_process() takes task_struct... I figured this would
> be a legit switch. I have seen no adverse affects yet.
>   

Did you stress test it?

Thanks,
Edward.

> -Ryan
>
> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>   
>> Ryan Hope wrote:
>>     
>>> Is this todo really done?
>>>       
>> nop
>>
>>     
>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>> replace wake_up() with wake_up_process()
>>>>>           
>>> There are still a few more wake_up()'s in the code, the following
>>> patch takes care of 2 of them.
>>>       
>>  both are not obvious for me, review is needed..
>>
>> Edward.
>>
>>     
>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>> index a164f5a..355548d 100644
>>> --- a/fs/reiser4/entd.c
>>> +++ b/fs/reiser4/entd.c
>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>  #endif
>>>      spin_unlock(&ent->guard);
>>>      if (wake_up_ent)
>>> -        wake_up(&ent->wait);
>>> +        wake_up_process(ent->tsk);
>>>  }
>>>
>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>> writeback_control *wbc)
>>>      ent->nr_todo_reqs++;
>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>      if (ent->nr_todo_reqs == 1)
>>> -        wake_up(&ent->wait);
>>> +        wake_up_process(ent->tsk);
>>>
>>>      spin_unlock(&ent->guard);
>>>
>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> reiserfs-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>       
>>     
>
>   

Re: [reiser4] Point

[Matthew]

Hi Edward,

I somehow did kind of a stress-test to it (all of those patches):
usual file operations were comparable (compared to the unpatched
reiser4-tree), but (f)sync-"performance" was somehow abysmal:

whereas it would take 2-5 minutes for the default reiser4 to sync data
out to disk after a kernel-compilation (issueing 'time sync'), it
would take >40+ minutes for the patched version to do so, also the hdd
would be clattering all the time during that

in following cases it would take significant longer:
- emergency syncing [with magic sysrq key]
- emergency remount (sometimes it wouldn't even finish)
- parallel tasks (doing more things at a time / launching several apps at once)
- hdparm -t /dev/sda (seemingly cache-flushing, heavy clattering
before & after the output)
- sync
- shutdown -h now, reboot

here's the page of the thread where I posted all of the testing-experience:
http://forums.gentoo.org/viewtopic-t-696253-start-375.html

could it be that Ryan's problems with ext3/ext4 are caused by probable
inherent problems / bugs with them than rather being a bug in reiser4
itself ?

I don't have any partition with ext* filesystems, only reiserfs +
reiser4 and unfortunately  also can't reproduce this behavior ...

the reiser4-devel/reiser4 testing-tree (for zen) can be found on the
zen-sources git-server:
http://zen-sources.org/cgi-bin/gitweb.cgi?p=kernel.git;a=summary
kudos to Ryan, Miguel, Dominic and Corbin :)

keep up the work, guys ! :)

Thanks

Mat

On Tue, Aug 12, 2008 at 8:58 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Ryan Hope wrote:
>> This works because the struct "ent" is of type "entd_context"
>
> This logic train goes to a wrong direction ;)
>
>>  which
>> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
>> The function wake_up() takes the parameter wait_queue_head_t the
>> function wake_up_process() takes task_struct... I figured this would
>> be a legit switch. I have seen no adverse affects yet.
>>
>
> Did you stress test it?
>
> Thanks,
> Edward.
>
>> -Ryan
>>
>> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>
>>> Ryan Hope wrote:
>>>
>>>> Is this todo really done?
>>>>
>>> nop
>>>
>>>
>>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>>> replace wake_up() with wake_up_process()
>>>>>>
>>>> There are still a few more wake_up()'s in the code, the following
>>>> patch takes care of 2 of them.
>>>>
>>>  both are not obvious for me, review is needed..
>>>
>>> Edward.
>>>
>>>
>>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>>> index a164f5a..355548d 100644
>>>> --- a/fs/reiser4/entd.c
>>>> +++ b/fs/reiser4/entd.c
>>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>>  #endif
>>>>      spin_unlock(&ent->guard);
>>>>      if (wake_up_ent)
>>>> -        wake_up(&ent->wait);
>>>> +        wake_up_process(ent->tsk);
>>>>  }
>>>>
>>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>>> writeback_control *wbc)
>>>>      ent->nr_todo_reqs++;
>>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>>      if (ent->nr_todo_reqs == 1)
>>>> -        wake_up(&ent->wait);
>>>> +        wake_up_process(ent->tsk);
>>>>
>>>>      spin_unlock(&ent->guard);
>>>>
>>>>
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe
>>>> reiserfs-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>
>>
>>
>
> --
> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: [reiser4] Point

[Edward Shishkin]

Matthew wrote:
> Hi Edward,
>
> I somehow did kind of a stress-test to it (all of those patches):
> usual file operations were comparable (compared to the unpatched
> reiser4-tree), but (f)sync-"performance" was somehow abysmal:
>
> whereas it would take 2-5 minutes for the default reiser4 to sync data
> out to disk after a kernel-compilation (issueing 'time sync'), it
> would take >40+ minutes for the patched version to do so, also the hdd
> would be clattering all the time during that
>
> in following cases it would take significant longer:
> - emergency syncing [with magic sysrq key]
> - emergency remount (sometimes it wouldn't even finish)
> - parallel tasks (doing more things at a time / launching several apps at once)
> - hdparm -t /dev/sda (seemingly cache-flushing, heavy clattering
> before & after the output)
> - sync
> - shutdown -h now, reboot
>
> here's the page of the thread where I posted all of the testing-experience:
> http://forums.gentoo.org/viewtopic-t-696253-start-375.html
>
> could it be that Ryan's problems with ext3/ext4 are caused by probable
> inherent problems / bugs with them than rather being a bug in reiser4
> itself ?
>
> I don't have any partition with ext* filesystems, only reiserfs +
> reiser4 and unfortunately  also can't reproduce this behavior ...
>
>   

Guys, patches 2/3, 3/3 of the latest series are wrong and all
manipulations with this stuff (testing, getting stacktraces, etc.)
is just useless wasting your time.

The patch 1/3 (use-wake_up_process-instead-of-wake_up)
seems to be ok, but it must be stress tested.
If there are any problems specific to this patch, then let me know.

Thanks,
Edward.

> the reiser4-devel/reiser4 testing-tree (for zen) can be found on the
> zen-sources git-server:
> http://zen-sources.org/cgi-bin/gitweb.cgi?p=kernel.git;a=summary
> kudos to Ryan, Miguel, Dominic and Corbin :)
>
> keep up the work, guys ! :)
>
> Thanks
>
> Mat
>
> On Tue, Aug 12, 2008 at 8:58 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>   
>> Ryan Hope wrote:
>>     
>>> This works because the struct "ent" is of type "entd_context"
>>>       
>> This logic train goes to a wrong direction ;)
>>
>>     
>>>  which
>>> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
>>> The function wake_up() takes the parameter wait_queue_head_t the
>>> function wake_up_process() takes task_struct... I figured this would
>>> be a legit switch. I have seen no adverse affects yet.
>>>
>>>       
>> Did you stress test it?
>>
>> Thanks,
>> Edward.
>>
>>     
>>> -Ryan
>>>
>>> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>
>>>       
>>>> Ryan Hope wrote:
>>>>
>>>>         
>>>>> Is this todo really done?
>>>>>
>>>>>           
>>>> nop
>>>>
>>>>
>>>>         
>>>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>>>> replace wake_up() with wake_up_process()
>>>>>>>
>>>>>>>               
>>>>> There are still a few more wake_up()'s in the code, the following
>>>>> patch takes care of 2 of them.
>>>>>
>>>>>           
>>>>  both are not obvious for me, review is needed..
>>>>
>>>> Edward.
>>>>
>>>>
>>>>         
>>>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>>>> index a164f5a..355548d 100644
>>>>> --- a/fs/reiser4/entd.c
>>>>> +++ b/fs/reiser4/entd.c
>>>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>>>  #endif
>>>>>      spin_unlock(&ent->guard);
>>>>>      if (wake_up_ent)
>>>>> -        wake_up(&ent->wait);
>>>>> +        wake_up_process(ent->tsk);
>>>>>  }
>>>>>
>>>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>>>> writeback_control *wbc)
>>>>>      ent->nr_todo_reqs++;
>>>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>>>      if (ent->nr_todo_reqs == 1)
>>>>> -        wake_up(&ent->wait);
>>>>> +        wake_up_process(ent->tsk);
>>>>>
>>>>>      spin_unlock(&ent->guard);
>>>>>
>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>> reiserfs-devel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>>
>>>>>           
>>>       
>> --
>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>     
>
>   

Re: [reiser4] Point

[Matthew]

Hi Edward,

ok, I'm testing it (patch 1) on my production system since yesterday,
no adverse effects until now,

could you please explain what you exactly mean with "stress test"?
which loads does it include ?

I'm using this for the following tasks right now:
- syncing the portage-tree (lots of small files, at least once a day;
a separate reiser4-partition for /usr/portage)
- emerging applications (reiser4 on the system-partition; emerge &
compilations put relative heavy load on it)
- creating stage4-tarballs (creating a tarball out of the running system)
- heavy compiling kernels (with -j 25)
- usual daily "work" (booting into the OS & launching / working with apps)
- updatedb
- *

if you need more tasks to do let me know and I'll try to include them

Thanks

Mat

On Tue, Aug 12, 2008 at 10:47 PM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Matthew wrote:
>> Hi Edward,
>>
>> I somehow did kind of a stress-test to it (all of those patches):
>> usual file operations were comparable (compared to the unpatched
>> reiser4-tree), but (f)sync-"performance" was somehow abysmal:
>>
>> whereas it would take 2-5 minutes for the default reiser4 to sync data
>> out to disk after a kernel-compilation (issueing 'time sync'), it
>> would take >40+ minutes for the patched version to do so, also the hdd
>> would be clattering all the time during that
>>
>> in following cases it would take significant longer:
>> - emergency syncing [with magic sysrq key]
>> - emergency remount (sometimes it wouldn't even finish)
>> - parallel tasks (doing more things at a time / launching several apps at once)
>> - hdparm -t /dev/sda (seemingly cache-flushing, heavy clattering
>> before & after the output)
>> - sync
>> - shutdown -h now, reboot
>>
>> here's the page of the thread where I posted all of the testing-experience:
>> http://forums.gentoo.org/viewtopic-t-696253-start-375.html
>>
>> could it be that Ryan's problems with ext3/ext4 are caused by probable
>> inherent problems / bugs with them than rather being a bug in reiser4
>> itself ?
>>
>> I don't have any partition with ext* filesystems, only reiserfs +
>> reiser4 and unfortunately  also can't reproduce this behavior ...
>>
>>
>
> Guys, patches 2/3, 3/3 of the latest series are wrong and all
> manipulations with this stuff (testing, getting stacktraces, etc.)
> is just useless wasting your time.
>
> The patch 1/3 (use-wake_up_process-instead-of-wake_up)
> seems to be ok, but it must be stress tested.
> If there are any problems specific to this patch, then let me know.
>
> Thanks,
> Edward.
>
>> the reiser4-devel/reiser4 testing-tree (for zen) can be found on the
>> zen-sources git-server:
>> http://zen-sources.org/cgi-bin/gitweb.cgi?p=kernel.git;a=summary
>> kudos to Ryan, Miguel, Dominic and Corbin :)
>>
>> keep up the work, guys ! :)
>>
>> Thanks
>>
>> Mat
>>
>> On Tue, Aug 12, 2008 at 8:58 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>
>>> Ryan Hope wrote:
>>>
>>>> This works because the struct "ent" is of type "entd_context"
>>>>
>>> This logic train goes to a wrong direction ;)
>>>
>>>
>>>>  which
>>>> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
>>>> The function wake_up() takes the parameter wait_queue_head_t the
>>>> function wake_up_process() takes task_struct... I figured this would
>>>> be a legit switch. I have seen no adverse affects yet.
>>>>
>>>>
>>> Did you stress test it?
>>>
>>> Thanks,
>>> Edward.
>>>
>>>
>>>> -Ryan
>>>>
>>>> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>>
>>>>> Ryan Hope wrote:
>>>>>
>>>>>
>>>>>> Is this todo really done?
>>>>>>
>>>>>>
>>>>> nop
>>>>>
>>>>>
>>>>>
>>>>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>>>>> replace wake_up() with wake_up_process()
>>>>>>>>
>>>>>>>>
>>>>>> There are still a few more wake_up()'s in the code, the following
>>>>>> patch takes care of 2 of them.
>>>>>>
>>>>>>
>>>>>  both are not obvious for me, review is needed..
>>>>>
>>>>> Edward.
>>>>>
>>>>>
>>>>>
>>>>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>>>>> index a164f5a..355548d 100644
>>>>>> --- a/fs/reiser4/entd.c
>>>>>> +++ b/fs/reiser4/entd.c
>>>>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>>>>  #endif
>>>>>>      spin_unlock(&ent->guard);
>>>>>>      if (wake_up_ent)
>>>>>> -        wake_up(&ent->wait);
>>>>>> +        wake_up_process(ent->tsk);
>>>>>>  }
>>>>>>
>>>>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>>>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>>>>> writeback_control *wbc)
>>>>>>      ent->nr_todo_reqs++;
>>>>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>>>>      if (ent->nr_todo_reqs == 1)
>>>>>> -        wake_up(&ent->wait);
>>>>>> +        wake_up_process(ent->tsk);
>>>>>>
>>>>>>      spin_unlock(&ent->guard);
>>>>>>
>>>>>>
>>>>>> --
>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>> reiserfs-devel" in
>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>
>>>>>>
>>>>>>
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>>
>>
>>
>
>

Re: [reiser4] Point

[Edward Shishkin]

Matthew wrote:
> Hi Edward,
>
> ok, I'm testing it (patch 1) on my production system since yesterday,
> no adverse effects until now,
>
> could you please explain what you exactly mean with "stress test"?
> which loads does it include ?
>
>   

It means regression testing with any set of applications
that involves as much file api as possible and causes
intensive IO activity.

> I'm using this for the following tasks right now:
> - syncing the portage-tree (lots of small files, at least once a day;
> a separate reiser4-partition for /usr/portage)
> - emerging applications (reiser4 on the system-partition; emerge &
> compilations put relative heavy load on it)
> - creating stage4-tarballs (creating a tarball out of the running system)
> - heavy compiling kernels (with -j 25)
> - usual daily "work" (booting into the OS & launching / working with apps)
> - updatedb
> - *
>
>   

Ok, its enough.
Thanks to everyone!

> if you need more tasks to do let me know and I'll try to include them
>
> Thanks
>
> Mat
>
> On Tue, Aug 12, 2008 at 10:47 PM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>   
>> Matthew wrote:
>>     
>>> Hi Edward,
>>>
>>> I somehow did kind of a stress-test to it (all of those patches):
>>> usual file operations were comparable (compared to the unpatched
>>> reiser4-tree), but (f)sync-"performance" was somehow abysmal:
>>>
>>> whereas it would take 2-5 minutes for the default reiser4 to sync data
>>> out to disk after a kernel-compilation (issueing 'time sync'), it
>>> would take >40+ minutes for the patched version to do so, also the hdd
>>> would be clattering all the time during that
>>>
>>> in following cases it would take significant longer:
>>> - emergency syncing [with magic sysrq key]
>>> - emergency remount (sometimes it wouldn't even finish)
>>> - parallel tasks (doing more things at a time / launching several apps at once)
>>> - hdparm -t /dev/sda (seemingly cache-flushing, heavy clattering
>>> before & after the output)
>>> - sync
>>> - shutdown -h now, reboot
>>>
>>> here's the page of the thread where I posted all of the testing-experience:
>>> http://forums.gentoo.org/viewtopic-t-696253-start-375.html
>>>
>>> could it be that Ryan's problems with ext3/ext4 are caused by probable
>>> inherent problems / bugs with them than rather being a bug in reiser4
>>> itself ?
>>>
>>> I don't have any partition with ext* filesystems, only reiserfs +
>>> reiser4 and unfortunately  also can't reproduce this behavior ...
>>>
>>>
>>>       
>> Guys, patches 2/3, 3/3 of the latest series are wrong and all
>> manipulations with this stuff (testing, getting stacktraces, etc.)
>> is just useless wasting your time.
>>
>> The patch 1/3 (use-wake_up_process-instead-of-wake_up)
>> seems to be ok, but it must be stress tested.
>> If there are any problems specific to this patch, then let me know.
>>
>> Thanks,
>> Edward.
>>
>>     
>>> the reiser4-devel/reiser4 testing-tree (for zen) can be found on the
>>> zen-sources git-server:
>>> http://zen-sources.org/cgi-bin/gitweb.cgi?p=kernel.git;a=summary
>>> kudos to Ryan, Miguel, Dominic and Corbin :)
>>>
>>> keep up the work, guys ! :)
>>>
>>> Thanks
>>>
>>> Mat
>>>
>>> On Tue, Aug 12, 2008 at 8:58 PM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>
>>>       
>>>> Ryan Hope wrote:
>>>>
>>>>         
>>>>> This works because the struct "ent" is of type "entd_context"
>>>>>
>>>>>           
>>>> This logic train goes to a wrong direction ;)
>>>>
>>>>
>>>>         
>>>>>  which
>>>>> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
>>>>> The function wake_up() takes the parameter wait_queue_head_t the
>>>>> function wake_up_process() takes task_struct... I figured this would
>>>>> be a legit switch. I have seen no adverse affects yet.
>>>>>
>>>>>
>>>>>           
>>>> Did you stress test it?
>>>>
>>>> Thanks,
>>>> Edward.
>>>>
>>>>
>>>>         
>>>>> -Ryan
>>>>>
>>>>> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>
>>>>>
>>>>>           
>>>>>> Ryan Hope wrote:
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> Is this todo really done?
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>> nop
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>>>>>> replace wake_up() with wake_up_process()
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>                   
>>>>>>> There are still a few more wake_up()'s in the code, the following
>>>>>>> patch takes care of 2 of them.
>>>>>>>
>>>>>>>
>>>>>>>               
>>>>>>  both are not obvious for me, review is needed..
>>>>>>
>>>>>> Edward.
>>>>>>
>>>>>>
>>>>>>
>>>>>>             
>>>>>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>>>>>> index a164f5a..355548d 100644
>>>>>>> --- a/fs/reiser4/entd.c
>>>>>>> +++ b/fs/reiser4/entd.c
>>>>>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>>>>>  #endif
>>>>>>>      spin_unlock(&ent->guard);
>>>>>>>      if (wake_up_ent)
>>>>>>> -        wake_up(&ent->wait);
>>>>>>> +        wake_up_process(ent->tsk);
>>>>>>>  }
>>>>>>>
>>>>>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>>>>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>>>>>> writeback_control *wbc)
>>>>>>>      ent->nr_todo_reqs++;
>>>>>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>>>>>      if (ent->nr_todo_reqs == 1)
>>>>>>> -        wake_up(&ent->wait);
>>>>>>> +        wake_up_process(ent->tsk);
>>>>>>>
>>>>>>>      spin_unlock(&ent->guard);
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>> reiserfs-devel" in
>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>               
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>
>>>>
>>>>         
>>>       
>>     
>
>   

Re: [reiser4] Point

[Ryan Hope]

With just that first patch this is how I was stress-testing it....

My rootfs is ext3 and my /usr/portage partition is reiser4...

I did this: 'cd /usr/portage; while (true); do bonnie; done'
Then on another console I ran 'emerge glibc'
On another console I ran 'emerge --sync'
On another console I compiled my kernel

I did not do any benchmarks, just looked in dmesg for anything abnormal.

-Ryan

On Wed, Aug 13, 2008 at 5:57 AM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> Matthew wrote:
>> Hi Edward,
>>
>> ok, I'm testing it (patch 1) on my production system since yesterday,
>> no adverse effects until now,
>>
>> could you please explain what you exactly mean with "stress test"?
>> which loads does it include ?
>>
>>
>
> It means regression testing with any set of applications
> that involves as much file api as possible and causes
> intensive IO activity.
>
>> I'm using this for the following tasks right now:
>> - syncing the portage-tree (lots of small files, at least once a day;
>> a separate reiser4-partition for /usr/portage)
>> - emerging applications (reiser4 on the system-partition; emerge &
>> compilations put relative heavy load on it)
>> - creating stage4-tarballs (creating a tarball out of the running system)
>> - heavy compiling kernels (with -j 25)
>> - usual daily "work" (booting into the OS & launching / working with apps)
>> - updatedb
>> - *
>>
>>
>
> Ok, its enough.
> Thanks to everyone!
>
>> if you need more tasks to do let me know and I'll try to include them
>>
>> Thanks
>>
>> Mat
>>
>> On Tue, Aug 12, 2008 at 10:47 PM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>
>>> Matthew wrote:
>>>
>>>> Hi Edward,
>>>>
>>>> I somehow did kind of a stress-test to it (all of those patches):
>>>> usual file operations were comparable (compared to the unpatched
>>>> reiser4-tree), but (f)sync-"performance" was somehow abysmal:
>>>>
>>>> whereas it would take 2-5 minutes for the default reiser4 to sync data
>>>> out to disk after a kernel-compilation (issueing 'time sync'), it
>>>> would take >40+ minutes for the patched version to do so, also the hdd
>>>> would be clattering all the time during that
>>>>
>>>> in following cases it would take significant longer:
>>>> - emergency syncing [with magic sysrq key]
>>>> - emergency remount (sometimes it wouldn't even finish)
>>>> - parallel tasks (doing more things at a time / launching several apps at once)
>>>> - hdparm -t /dev/sda (seemingly cache-flushing, heavy clattering
>>>> before & after the output)
>>>> - sync
>>>> - shutdown -h now, reboot
>>>>
>>>> here's the page of the thread where I posted all of the testing-experience:
>>>> http://forums.gentoo.org/viewtopic-t-696253-start-375.html
>>>>
>>>> could it be that Ryan's problems with ext3/ext4 are caused by probable
>>>> inherent problems / bugs with them than rather being a bug in reiser4
>>>> itself ?
>>>>
>>>> I don't have any partition with ext* filesystems, only reiserfs +
>>>> reiser4 and unfortunately  also can't reproduce this behavior ...
>>>>
>>>>
>>>>
>>> Guys, patches 2/3, 3/3 of the latest series are wrong and all
>>> manipulations with this stuff (testing, getting stacktraces, etc.)
>>> is just useless wasting your time.
>>>
>>> The patch 1/3 (use-wake_up_process-instead-of-wake_up)
>>> seems to be ok, but it must be stress tested.
>>> If there are any problems specific to this patch, then let me know.
>>>
>>> Thanks,
>>> Edward.
>>>
>>>
>>>> the reiser4-devel/reiser4 testing-tree (for zen) can be found on the
>>>> zen-sources git-server:
>>>> http://zen-sources.org/cgi-bin/gitweb.cgi?p=kernel.git;a=summary
>>>> kudos to Ryan, Miguel, Dominic and Corbin :)
>>>>
>>>> keep up the work, guys ! :)
>>>>
>>>> Thanks
>>>>
>>>> Mat
>>>>
>>>> On Tue, Aug 12, 2008 at 8:58 PM, Edward Shishkin
>>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>>
>>>>> Ryan Hope wrote:
>>>>>
>>>>>
>>>>>> This works because the struct "ent" is of type "entd_context"
>>>>>>
>>>>>>
>>>>> This logic train goes to a wrong direction ;)
>>>>>
>>>>>
>>>>>
>>>>>>  which
>>>>>> has members "wait_queue_head_t wait" and "struct task_struct *tsk".
>>>>>> The function wake_up() takes the parameter wait_queue_head_t the
>>>>>> function wake_up_process() takes task_struct... I figured this would
>>>>>> be a legit switch. I have seen no adverse affects yet.
>>>>>>
>>>>>>
>>>>>>
>>>>> Did you stress test it?
>>>>>
>>>>> Thanks,
>>>>> Edward.
>>>>>
>>>>>
>>>>>
>>>>>> -Ryan
>>>>>>
>>>>>> On Fri, Aug 8, 2008 at 4:47 PM, Edward Shishkin
>>>>>> <edward.shishkin@gmail.com> wrote:
>>>>>>
>>>>>>
>>>>>>
>>>>>>> Ryan Hope wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> Is this todo really done?
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>> nop
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>>>> As in point 6. in todo list (pub.namesys.com/Reiser4/ToDo ) try to
>>>>>>>>>> replace wake_up() with wake_up_process()
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>> There are still a few more wake_up()'s in the code, the following
>>>>>>>> patch takes care of 2 of them.
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>  both are not obvious for me, review is needed..
>>>>>>>
>>>>>>> Edward.
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>> diff --git a/fs/reiser4/entd.c b/fs/reiser4/entd.c
>>>>>>>> index a164f5a..355548d 100644
>>>>>>>> --- a/fs/reiser4/entd.c
>>>>>>>> +++ b/fs/reiser4/entd.c
>>>>>>>> @@ -218,7 +218,7 @@ void reiser4_leave_flush(struct super_block *super)
>>>>>>>>  #endif
>>>>>>>>      spin_unlock(&ent->guard);
>>>>>>>>      if (wake_up_ent)
>>>>>>>> -        wake_up(&ent->wait);
>>>>>>>> +        wake_up_process(ent->tsk);
>>>>>>>>  }
>>>>>>>>
>>>>>>>>  #define ENTD_CAPTURE_APAGE_BURST SWAP_CLUSTER_MAX
>>>>>>>> @@ -304,7 +304,7 @@ int write_page_by_ent(struct page *page, struct
>>>>>>>> writeback_control *wbc)
>>>>>>>>      ent->nr_todo_reqs++;
>>>>>>>>      list_add_tail(&rq.link, &ent->todo_list);
>>>>>>>>      if (ent->nr_todo_reqs == 1)
>>>>>>>> -        wake_up(&ent->wait);
>>>>>>>> +        wake_up_process(ent->tsk);
>>>>>>>>
>>>>>>>>      spin_unlock(&ent->guard);
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> To unsubscribe from this list: send the line "unsubscribe
>>>>>>>> reiserfs-devel" in
>>>>>>>> the body of a message to majordomo@vger.kernel.org
>>>>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>> --
>>>>> To unsubscribe from this list: send the line "unsubscribe reiserfs-devel" in
>>>>> the body of a message to majordomo@vger.kernel.org
>>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>>
>
>

Re: [reiser4] Point

[Alli Quaknaa]

Correct me if I'm wrong, but the "emerge glibc" and compiling kernel
are not actually happening on the reiser4 partition (if it is
/usr/portage only) and they IMHO actually lower the stress on the
reiser4 partition, because they consume CPU power that would be used
by emerge sync and bonnie (which do stress-test the reiser4
partition).

But maybe I'm wrong, or it was the point to get emerge sync & bonnie
working in an environment where they have to compete for CPU time with
other processes?

(Please bear with me, I'm not a developer and I'm newbie, but I use
Gentoo so the things above seemed weird to me).
al-Quaknaa

Re: [reiser4] Point

[Ryan Hope]

Well bonnie is running on the reiser4 partition and so is emerge
--sync.... I figured a better stress of reiser4 would be to see how it
hold up when the whole system is loaded as well, not just reiser4
working alone

On Wed, Aug 13, 2008 at 11:59 AM, Alli Quaknaa <alquaknaa@gmail.com> wrote:
> Correct me if I'm wrong, but the "emerge glibc" and compiling kernel
> are not actually happening on the reiser4 partition (if it is
> /usr/portage only) and they IMHO actually lower the stress on the
> reiser4 partition, because they consume CPU power that would be used
> by emerge sync and bonnie (which do stress-test the reiser4
> partition).
>
> But maybe I'm wrong, or it was the point to get emerge sync & bonnie
> working in an environment where they have to compete for CPU time with
> other processes?
>
> (Please bear with me, I'm not a developer and I'm newbie, but I use
> Gentoo so the things above seemed weird to me).
> al-Quaknaa
>

Re: [reiser4] Point

[Alli Quaknaa]

OK, I wasn't sure if it was intended this way.

Anyway - I've just got a new notebook and I'm putting my / on reiser4
and I've already got to a little trouble, when I tried to kill some
processes and the kill didn't work, and dmesg reported some oops. I
know this isn't very useful (cause I don't have the oops or other
specific info), I just want to point out that there will probabyl be a
higher number of problematic situations for teh fs to handle on / than
anywhere else. I will report whatever is to come and I will try to
help with testing as much as possible :)

Thanks for the great fs you're working on :)
al-Quaknaa