* [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
@ 2009-02-03 8:42 Artem Bityutskiy
2009-02-03 9:13 ` Nick Piggin
0 siblings, 1 reply; 6+ messages in thread
From: Artem Bityutskiy @ 2009-02-03 8:42 UTC (permalink / raw)
To: linux-fsdevel; +Cc: npiggin, LKML
Hi,
commit 05fe478dd04e02fa230c305ab9b5616669821dd3
Author: Nick Piggin <npiggin@suse.de>
Date: Tue Jan 6 14:39:08 2009 -0800
mm: write_cache_pages integrity fix
broke wbc->nr_to_write handling. Here is the fix.
I'm not 100% sure I got things right, because I am far not expert in the
area. Please, review it. The patch fixes my UBIFS issues, which are
caused by the fact that wbc->nr_to_write is not updated.
======================================================================
From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
Date: Mon, 2 Feb 2009 18:33:49 +0200
Subject: [PATCH] write-back: fix nr_to_write counter
Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke @wbc->nr_to_write.
'write_cache_pages()' changes it in the loop, but restores the original
value from @nr_to_write at the end, because of this code:
if (!wbc->no_nrwrite_index_update) {
if (wbc->range_cyclic || (range_whole && nr_to_write > 0))
mapping->writeback_index = done_index;
wbc->nr_to_write = nr_to_write;
}
Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
wbc->nr_to_write, while we should not. This patch fixes this behavior.
Also, I think wbc->nr_to_write should be changed in all cases, not only
when wbc->sync_mode == WB_SYNC_NONE.
Also, I add a comment explaining why we do not stop writing back.
Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
---
mm/page-writeback.c | 21 +++++++++++++++------
1 files changed, 15 insertions(+), 6 deletions(-)
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index b493db7..13a2b8e 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -1051,13 +1051,22 @@ continue_unlock:
}
}
- if (wbc->sync_mode == WB_SYNC_NONE) {
- wbc->nr_to_write--;
- if (wbc->nr_to_write <= 0) {
- done = 1;
- break;
- }
+ if (nr_to_write > 0)
+ nr_to_write--;
+ else if (wbc->sync_mode == WB_SYNC_NONE) {
+ /*
+ * We stop writing back only if we are not
+ * doing integrity sync. In case of integrity
+ * sync we have to keep going because someone
+ * may be concurrently dirtying pages, and we
+ * might have synced a lot of newly appeared
+ * dirty pages, bud have not synced all of the
+ * old dirty pages.
+ */
+ done = 1;
+ break;
}
+
if (wbc->nonblocking && bdi_write_congested(bdi)) {
wbc->encountered_congestion = 1;
done = 1;
--
1.6.0.6
--
Best regards,
Artem Bityutskiy (Битюцкий Артём)
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
2009-02-03 8:42 [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter Artem Bityutskiy
@ 2009-02-03 9:13 ` Nick Piggin
2009-02-03 9:24 ` Nick Piggin
2009-02-03 9:32 ` Artem Bityutskiy
0 siblings, 2 replies; 6+ messages in thread
From: Nick Piggin @ 2009-02-03 9:13 UTC (permalink / raw)
To: dedekind, Andrew Morton; +Cc: linux-fsdevel, npiggin, LKML
On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
> Hi,
>
> commit 05fe478dd04e02fa230c305ab9b5616669821dd3
> Author: Nick Piggin <npiggin@suse.de>
> Date: Tue Jan 6 14:39:08 2009 -0800
>
> mm: write_cache_pages integrity fix
>
> broke wbc->nr_to_write handling. Here is the fix.
>
> I'm not 100% sure I got things right, because I am far not expert in the
> area. Please, review it. The patch fixes my UBIFS issues, which are
> caused by the fact that wbc->nr_to_write is not updated.
> ======================================================================
>
> From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> Date: Mon, 2 Feb 2009 18:33:49 +0200
> Subject: [PATCH] write-back: fix nr_to_write counter
>
> Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke @wbc->nr_to_write.
> 'write_cache_pages()' changes it in the loop, but restores the original
> value from @nr_to_write at the end, because of this code:
>
> if (!wbc->no_nrwrite_index_update) {
> if (wbc->range_cyclic || (range_whole && nr_to_write > 0))
> mapping->writeback_index = done_index;
> wbc->nr_to_write = nr_to_write;
> }
The commit you quote only moves nr_to_write to not take effect for
WB_SYNC_ALL (ie. data integrity) writeout. And makes no other change
to write_cache_pages.
I thought your problem might have been that you were calling this
with WB_SYNC_ALL and expecting it to heed nr_to_write, however...
> Also, I think wbc->nr_to_write should be changed in all cases, not only
> when wbc->sync_mode == WB_SYNC_NONE.
... you mention this here like it is an *additional* issue on top
of your problem. So I fail to see how my commit could have caused
this problem?
> Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
> wbc->nr_to_write, while we should not. This patch fixes this behavior.
And I don't know what you mean by this because the patch doesn't
fix any problem there AFAIKS.
Anyway, I did probably not pay enough attention to ubifs when making
this change, and if it wants wbc->nr_to_write updated even for data
integrity syncs, I don't see the harm in that. So I don't have any
objection to your patch. Thanks.
Can you cc stable@kernel.org when a final version gets merged upstream
please?
>
> Also, I add a comment explaining why we do not stop writing back.
>
> Signed-off-by: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> ---
> mm/page-writeback.c | 21 +++++++++++++++------
> 1 files changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index b493db7..13a2b8e 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -1051,13 +1051,22 @@ continue_unlock:
> }
> }
>
> - if (wbc->sync_mode == WB_SYNC_NONE) {
> - wbc->nr_to_write--;
> - if (wbc->nr_to_write <= 0) {
> - done = 1;
> - break;
> - }
> + if (nr_to_write > 0)
> + nr_to_write--;
> + else if (wbc->sync_mode == WB_SYNC_NONE) {
> + /*
> + * We stop writing back only if we are not
> + * doing integrity sync. In case of integrity
> + * sync we have to keep going because someone
> + * may be concurrently dirtying pages, and we
> + * might have synced a lot of newly appeared
> + * dirty pages, bud have not synced all of the
> + * old dirty pages.
> + */
> + done = 1;
> + break;
> }
> +
> if (wbc->nonblocking && bdi_write_congested(bdi)) {
> wbc->encountered_congestion = 1;
> done = 1;
> --
> 1.6.0.6
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
2009-02-03 9:13 ` Nick Piggin
@ 2009-02-03 9:24 ` Nick Piggin
2009-02-03 9:32 ` Artem Bityutskiy
1 sibling, 0 replies; 6+ messages in thread
From: Nick Piggin @ 2009-02-03 9:24 UTC (permalink / raw)
To: dedekind; +Cc: Andrew Morton, linux-fsdevel, npiggin, LKML
On Tuesday 03 February 2009 20:13:56 Nick Piggin wrote:
> On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
> > Hi,
> >
> > commit 05fe478dd04e02fa230c305ab9b5616669821dd3
> > Author: Nick Piggin <npiggin@suse.de>
> > Date: Tue Jan 6 14:39:08 2009 -0800
> >
> > mm: write_cache_pages integrity fix
> >
> > broke wbc->nr_to_write handling. Here is the fix.
> >
> > I'm not 100% sure I got things right, because I am far not expert in the
> > area. Please, review it. The patch fixes my UBIFS issues, which are
> > caused by the fact that wbc->nr_to_write is not updated.
> > ======================================================================
> >
> > From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> > Date: Mon, 2 Feb 2009 18:33:49 +0200
> > Subject: [PATCH] write-back: fix nr_to_write counter
> >
> > Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke @wbc->nr_to_write.
> > 'write_cache_pages()' changes it in the loop, but restores the original
> > value from @nr_to_write at the end, because of this code:
> >
> > if (!wbc->no_nrwrite_index_update) {
> > if (wbc->range_cyclic || (range_whole && nr_to_write >
> > 0)) mapping->writeback_index = done_index;
> > wbc->nr_to_write = nr_to_write;
> > }
>
> The commit you quote only moves nr_to_write to not take effect for
> WB_SYNC_ALL (ie. data integrity) writeout. And makes no other change
> to write_cache_pages.
>
> I thought your problem might have been that you were calling this
> with WB_SYNC_ALL and expecting it to heed nr_to_write, however...
>
> > Also, I think wbc->nr_to_write should be changed in all cases, not only
> > when wbc->sync_mode == WB_SYNC_NONE.
>
> ... you mention this here like it is an *additional* issue on top
> of your problem. So I fail to see how my commit could have caused
> this problem?
>
> > Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
> > wbc->nr_to_write, while we should not. This patch fixes this behavior.
>
> And I don't know what you mean by this because the patch doesn't
> fix any problem there AFAIKS.
>
> Anyway, I did probably not pay enough attention to ubifs when making
> this change, and if it wants wbc->nr_to_write updated even for data
> integrity syncs, I don't see the harm in that. So I don't have any
> objection to your patch. Thanks.
>
> Can you cc stable@kernel.org when a final version gets merged upstream
> please?
What I mean to say is that while I think the patch is not a problem,
and actually should be merged if it is required for ubifs, I would
like to just see the changelog clarified...
Also if you add a nice comment there, maybe mention that nr_to_write
must reflect the number of pages written even for the WB_SYNC_ALL case
(arguably the data integrity issue which you comment on is more
obvious in comparison to the nr_to_write requirement of callers).
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
2009-02-03 9:13 ` Nick Piggin
2009-02-03 9:24 ` Nick Piggin
@ 2009-02-03 9:32 ` Artem Bityutskiy
2009-02-03 9:45 ` Nick Piggin
1 sibling, 1 reply; 6+ messages in thread
From: Artem Bityutskiy @ 2009-02-03 9:32 UTC (permalink / raw)
To: Nick Piggin; +Cc: Andrew Morton, linux-fsdevel, npiggin, LKML
On Tue, 2009-02-03 at 20:13 +1100, Nick Piggin wrote:
> On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
> > Hi,
> >
> > commit 05fe478dd04e02fa230c305ab9b5616669821dd3
> > Author: Nick Piggin <npiggin@suse.de>
> > Date: Tue Jan 6 14:39:08 2009 -0800
> >
> > mm: write_cache_pages integrity fix
> >
> > broke wbc->nr_to_write handling. Here is the fix.
> >
> > I'm not 100% sure I got things right, because I am far not expert in the
> > area. Please, review it. The patch fixes my UBIFS issues, which are
> > caused by the fact that wbc->nr_to_write is not updated.
> > ======================================================================
> >
> > From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> > Date: Mon, 2 Feb 2009 18:33:49 +0200
> > Subject: [PATCH] write-back: fix nr_to_write counter
> >
> > Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke @wbc->nr_to_write.
> > 'write_cache_pages()' changes it in the loop, but restores the original
> > value from @nr_to_write at the end, because of this code:
> >
> > if (!wbc->no_nrwrite_index_update) {
> > if (wbc->range_cyclic || (range_whole && nr_to_write > 0))
> > mapping->writeback_index = done_index;
> > wbc->nr_to_write = nr_to_write;
> > }
>
> The commit you quote only moves nr_to_write to not take effect for
> WB_SYNC_ALL (ie. data integrity) writeout. And makes no other change
> to write_cache_pages.
Nick, I'm sorry if my e-mail looked like I'm blame you, I referred the
commit because git-bisect pointed to it and I though it for me :-)
And I apologize if I write stupid things.
Here is the commit 05fe478dd04e02fa230c305ab9b5616669821dd3:
diff --git a/mm/filemap.c b/mm/filemap.c
index f9d8818..9c5e623 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -210,7 +210,7 @@ int __filemap_fdatawrite_range(struct address_space *mapping, loff_t start,
int ret;
struct writeback_control wbc = {
.sync_mode = sync_mode,
- .nr_to_write = mapping->nrpages * 2,
+ .nr_to_write = LONG_MAX,
.range_start = start,
.range_end = end,
};
diff --git a/mm/page-writeback.c b/mm/page-writeback.c
index 2e847cd..5edca67 100644
--- a/mm/page-writeback.c
+++ b/mm/page-writeback.c
@@ -963,8 +963,10 @@ retry:
}
}
- if (--nr_to_write <= 0)
- done = 1;
+ if (wbc->sync_mode == WB_SYNC_NONE) {
+ if (--wbc->nr_to_write <= 0)
+ done = 1;
+ }
if (wbc->nonblocking && bdi_write_congested(bdi)) {
wbc->encountered_congestion = 1;
done = 1;
It makes the following changes:
1. Decrement wbc->nr_to_write instead of nr_to_write
2. Decrement wbc->nr_to_write _only_ if wbc->sync_mode == WB_SYNC_NONE
3. If synced nr_to_write pages, stop only if if wbc->sync_mode ==
WB_SYNC_NONE, otherwise keep going.
The commit message talks only about item 3, at leaset as I understand
it. I do not quote the commit message because it is large. Thus, I
thought changes 1 and 2 were not intentional.
In my patch I try to
1. Undo changes 1 and 2
2. Add a comment explaining change 3 (it very useful to have comments in
_code_, not only in the commit)
> I thought your problem might have been that you were calling this
> with WB_SYNC_ALL and expecting it to heed nr_to_write, however...
Err, my problem is that wbc->nr_to_write is not updated.
> > Also, I think wbc->nr_to_write should be changed in all cases, not only
> > when wbc->sync_mode == WB_SYNC_NONE.
>
> ... you mention this here like it is an *additional* issue on top
> of your problem. So I fail to see how my commit could have caused
> this problem?
This is the change number 2 in your commit.
> > Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
> > wbc->nr_to_write, while we should not. This patch fixes this behavior.
>
> And I don't know what you mean by this because the patch doesn't
> fix any problem there AFAIKS.
This is about change number 1. In the "for" loop you change
wbc->nr_to_write, instead of nr_to_write. But before the function
returns, it writes nr_to_write back to wbc->nr_to_write, so the
result is that the caller sees wbc->nr_to_write unchanged.
> Anyway, I did probably not pay enough attention to ubifs when making
> this change, and if it wants wbc->nr_to_write updated even for data
> integrity syncs, I don't see the harm in that. So I don't have any
> objection to your patch. Thanks.
It is just how things were _before_ your patch.
> Can you cc stable@kernel.org when a final version gets merged upstream
> please?
Sure, I just want to get blessing of one of MM/FS gurus. E.g. you
Acked-by would be enough.
Thanks.
--
Best regards,
Artem Bityutskiy (Битюцкий Артём)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
2009-02-03 9:32 ` Artem Bityutskiy
@ 2009-02-03 9:45 ` Nick Piggin
2009-02-03 9:47 ` Artem Bityutskiy
0 siblings, 1 reply; 6+ messages in thread
From: Nick Piggin @ 2009-02-03 9:45 UTC (permalink / raw)
To: dedekind; +Cc: Andrew Morton, linux-fsdevel, npiggin, LKML
On Tuesday 03 February 2009 20:32:49 Artem Bityutskiy wrote:
> On Tue, 2009-02-03 at 20:13 +1100, Nick Piggin wrote:
> > On Tuesday 03 February 2009 19:42:22 Artem Bityutskiy wrote:
> > > Hi,
> > >
> > > commit 05fe478dd04e02fa230c305ab9b5616669821dd3
> > > Author: Nick Piggin <npiggin@suse.de>
> > > Date: Tue Jan 6 14:39:08 2009 -0800
> > >
> > > mm: write_cache_pages integrity fix
> > >
> > > broke wbc->nr_to_write handling. Here is the fix.
> > >
> > > I'm not 100% sure I got things right, because I am far not expert in
> > > the area. Please, review it. The patch fixes my UBIFS issues, which are
> > > caused by the fact that wbc->nr_to_write is not updated.
> > > ======================================================================
> > >
> > > From: Artem Bityutskiy <Artem.Bityutskiy@nokia.com>
> > > Date: Mon, 2 Feb 2009 18:33:49 +0200
> > > Subject: [PATCH] write-back: fix nr_to_write counter
> > >
> > > Commit 05fe478dd04e02fa230c305ab9b5616669821dd3 broke
> > > @wbc->nr_to_write. 'write_cache_pages()' changes it in the loop, but
> > > restores the original value from @nr_to_write at the end, because of
> > > this code:
> > >
> > > if (!wbc->no_nrwrite_index_update) {
> > > if (wbc->range_cyclic || (range_whole && nr_to_write >
> > > 0)) mapping->writeback_index = done_index; wbc->nr_to_write =
> > > nr_to_write;
> > > }
> >
> > The commit you quote only moves nr_to_write to not take effect for
> > WB_SYNC_ALL (ie. data integrity) writeout. And makes no other change
> > to write_cache_pages.
>
> Nick, I'm sorry if my e-mail looked like I'm blame you, I referred the
> commit because git-bisect pointed to it and I though it for me :-)
>
> And I apologize if I write stupid things.
No, you are probably right to blame me ;) but I just am not quite sure
if I read your description correctly.
> Here is the commit 05fe478dd04e02fa230c305ab9b5616669821dd3:
>
> diff --git a/mm/filemap.c b/mm/filemap.c
> index f9d8818..9c5e623 100644
> --- a/mm/filemap.c
> +++ b/mm/filemap.c
> @@ -210,7 +210,7 @@ int __filemap_fdatawrite_range(struct address_space
> *mapping, loff_t start, int ret;
> struct writeback_control wbc = {
> .sync_mode = sync_mode,
> - .nr_to_write = mapping->nrpages * 2,
> + .nr_to_write = LONG_MAX,
> .range_start = start,
> .range_end = end,
> };
> diff --git a/mm/page-writeback.c b/mm/page-writeback.c
> index 2e847cd..5edca67 100644
> --- a/mm/page-writeback.c
> +++ b/mm/page-writeback.c
> @@ -963,8 +963,10 @@ retry:
> }
> }
>
> - if (--nr_to_write <= 0)
> - done = 1;
> + if (wbc->sync_mode == WB_SYNC_NONE) {
> + if (--wbc->nr_to_write <= 0)
> + done = 1;
> + }
> if (wbc->nonblocking && bdi_write_congested(bdi)) {
> wbc->encountered_congestion = 1;
> done = 1;
>
> It makes the following changes:
> 1. Decrement wbc->nr_to_write instead of nr_to_write
Ah, ok drat yes I missed this. Indeed this is a stupid bug :(
> 2. Decrement wbc->nr_to_write _only_ if wbc->sync_mode == WB_SYNC_NONE
> 3. If synced nr_to_write pages, stop only if if wbc->sync_mode ==
> WB_SYNC_NONE, otherwise keep going.
>
> The commit message talks only about item 3, at leaset as I understand
> it. I do not quote the commit message because it is large. Thus, I
> thought changes 1 and 2 were not intentional.
2 was more or less intentional but I glossed over ubifs. At any rate
I'm happy if you restore the old behaviour.
> In my patch I try to
> 1. Undo changes 1 and 2
> 2. Add a comment explaining change 3 (it very useful to have comments in
> _code_, not only in the commit)
>
> > I thought your problem might have been that you were calling this
> > with WB_SYNC_ALL and expecting it to heed nr_to_write, however...
>
> Err, my problem is that wbc->nr_to_write is not updated.
>
> > > Also, I think wbc->nr_to_write should be changed in all cases, not only
> > > when wbc->sync_mode == WB_SYNC_NONE.
> >
> > ... you mention this here like it is an *additional* issue on top
> > of your problem. So I fail to see how my commit could have caused
> > this problem?
>
> This is the change number 2 in your commit.
OK, it just wasn't clear to me from the changelog (I missed noticing bug #1).
> > > Well, in case of @wbc->no_nrwrite_index_update != 0, we do change
> > > wbc->nr_to_write, while we should not. This patch fixes this behavior.
> >
> > And I don't know what you mean by this because the patch doesn't
> > fix any problem there AFAIKS.
>
> This is about change number 1. In the "for" loop you change
> wbc->nr_to_write, instead of nr_to_write. But before the function
> returns, it writes nr_to_write back to wbc->nr_to_write, so the
> result is that the caller sees wbc->nr_to_write unchanged.
>
> > Anyway, I did probably not pay enough attention to ubifs when making
> > this change, and if it wants wbc->nr_to_write updated even for data
> > integrity syncs, I don't see the harm in that. So I don't have any
> > objection to your patch. Thanks.
>
> It is just how things were _before_ your patch.
>
> > Can you cc stable@kernel.org when a final version gets merged upstream
> > please?
>
> Sure, I just want to get blessing of one of MM/FS gurus. E.g. you
> Acked-by would be enough.
Yes please put an Acked-by: Nick Piggin <npiggin@suse.de> on there.
But I found your comments in this mail to be easier to understand.
Perhaps incorporate them into the changelog?
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter
2009-02-03 9:45 ` Nick Piggin
@ 2009-02-03 9:47 ` Artem Bityutskiy
0 siblings, 0 replies; 6+ messages in thread
From: Artem Bityutskiy @ 2009-02-03 9:47 UTC (permalink / raw)
To: Nick Piggin; +Cc: Andrew Morton, linux-fsdevel, npiggin, LKML
On Tue, 2009-02-03 at 20:45 +1100, Nick Piggin wrote:
> > > Can you cc stable@kernel.org when a final version gets merged upstream
> > > please?
> >
> > Sure, I just want to get blessing of one of MM/FS gurus. E.g. you
> > Acked-by would be enough.
>
> Yes please put an Acked-by: Nick Piggin <npiggin@suse.de> on there.
> But I found your comments in this mail to be easier to understand.
> Perhaps incorporate them into the changelog?
Will do and re-send it.
--
Best regards,
Artem Bityutskiy (Битюцкий Артём)
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2009-02-03 9:49 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-02-03 8:42 [BUGFIX re-send] [PATCH] write-back: fix nr_to_write counter Artem Bityutskiy
2009-02-03 9:13 ` Nick Piggin
2009-02-03 9:24 ` Nick Piggin
2009-02-03 9:32 ` Artem Bityutskiy
2009-02-03 9:45 ` Nick Piggin
2009-02-03 9:47 ` Artem Bityutskiy
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).