BKCVS broken ?

BKCVS broken ?

[Stelian Pop]

The current bkcvs export is broken, several recent changesets are
missing from it.

This occurs at least in the mm/ directory, but I haven't verified
if other directories are not affected. I detected this problem
because the head of bkcvs doesn't compile anymore and shows errors
in mm/* missing symbols.

One example:

Take this changeset from Changeset,v:
-------------------------------------------------------------
1.27702
log
@[PATCH] orphaned pagecache memleak fix

Chris found that with data journaling a reiserfs pagecache may be truncate
while still pinned.  The truncation removes the page->mapping, but the page
is still listed in the VM queues because it still has buffers.  Then during
the journaling process, a buffer is marked dirty and that sets the PG_dirty
bitflag as well (in mark_buffer_dirty).  After that the page is leaked
because it's both dirty and without a mapping.

So we must allow pages without mapping and dirty to reach the PagePrivate
check.  The page->mapping will be checked again right after the PagePrivate
check.

Signed-off-by: Andrea Arcangeli <andrea@@suse.de>
Signed-off-by: Andrew Morton <akpm@@osdl.org>
Signed-off-by: Linus Torvalds <torvalds@@osdl.org>

BKrev: 4234d7beMW4wcFI6ltxdMMhApwDmuA
-------------------------------------------------------------

Looking at
http://linux.bkbits.net:8080/linux-2.6/gnupatch@4234d7beMW4wcFI6ltxdMMhApwDmuA
shows this changeset should contain a delta for mm/vmscan.c

However, mm/vmscan.c,v contains:
-------------------------------------------------------------
head    1.238;  
access;
symbols;
locks; strict; 
comment @ * @;
expand  @o@;


1.238
date    2005.03.10.17.06.39;    author pj;      state Exp;
branches;
next    1.237;  
....
1.238
log
@cpusets - big numa cpu and memory placement

(Logical change 1.27465)
@
-------------------------------------------------------------

The 'Logical change 1.27702' is missing from the file...

Stelian.
-- 
Stelian Pop <stelian.pop@fr.alcove.com>
Alcove - http://www.alcove.com

Re: BKCVS broken ?

[Catalin Marinas]

Stelian Pop <stelian.pop@fr.alcove.com> wrote:
> The current bkcvs export is broken, several recent changesets are
> missing from it.
>
> This occurs at least in the mm/ directory, but I haven't verified
> if other directories are not affected. I detected this problem
> because the head of bkcvs doesn't compile anymore and shows errors
> in mm/* missing symbols.

I noticed a similar problem a few days ago. The ChangeSet,v file
contained the logs but there were no files with the corresponding
(Logical change ...). A day later, the files corresponding to those
logs were updated. I initially blamed the non-atomicity of CVS and
rsync but, reading your e-mail, the problem might not be that simple.

BTW, is there a way for the ChangeSet,v file to be updated after all
the source files are updated (to avoid the empty patch problem if
rsync'ing when the BKCVS repository is updated)?

Catalin

Re: BKCVS broken ?

[Larry McVoy]

I'll check into it.  We've been having problems with connecting to 
master.kernel.org, yup, here you go, anyone else seeing this?

>From lm@slovax.bitmover.com  Thu Mar 17 05:06:57 2005
Date: Thu, 17 Mar 2005 05:00:57 -0800
From: root@slovax.bitmover.com (Cron Daemon)
To: lm@slovax.bitmover.com
Subject: Cron <lm@slovax> /bk-cvsexport/src/UPDATE

Read from remote host master.kernel.org: Connection timed out


On Thu, Mar 17, 2005 at 03:45:22PM +0100, Stelian Pop wrote:
> The current bkcvs export is broken, several recent changesets are
> missing from it.

-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: BKCVS broken ?

[H. Peter Anvin]

Followup to:  <20050317154434.GA24378@bitmover.com>
By author:    lm@bitmover.com (Larry McVoy)
In newsgroup: linux.dev.kernel
>
> I'll check into it.  We've been having problems with connecting to 
> master.kernel.org, yup, here you go, anyone else seeing this?
> 
> From lm@slovax.bitmover.com  Thu Mar 17 05:06:57 2005
> Date: Thu, 17 Mar 2005 05:00:57 -0800
> From: root@slovax.bitmover.com (Cron Daemon)
> To: lm@slovax.bitmover.com
> Subject: Cron <lm@slovax> /bk-cvsexport/src/UPDATE
> 
> Read from remote host master.kernel.org: Connection timed out
> 

Please Cc: any reports of badness on kernel.org to
ftpadmin@kernel.org; I would have seen this quicker that way.

Around the time the above happened the machine was pretty bogged down,
because we're preparing new hardware to replace the main server, and
were doing some very large copies.  It might have caused a timeout.

I notice a long login from you at approximately 14:00 PST; does that
mean this is no longer an issue?

	-hpa

Re: BKCVS broken ?

[Larry McVoy]

I got swamped, I'll look at this after dinner.  But you might take a look
at this: http://www.bitkeeper.com/press/2005-03-17.html which is a link
to a very simple open source BK client.  It doesn't do much except track
the head of the tree but it does that well.  It's slightly better than
that, it puts all the checkin comments in BK/ChangeLog so you don't have
to go over the wire to get those.

It's intended for someone who just wants the latest and greatest snapshot,
knows how to do cp -rp and diff -Nur, it's pretty basic.  It's not a
CVS gateway replacement but it does work for every tree on bkbits.net.
Just to be clear, we are not dropping the CVS gateway, this is "in
addition to" not "instead of".

If this turns out to be popular we can look at making a BitTorrent image
of each tree available so people can get at them without swamping us.

Don't worry about the license, it's a joke.  BSD license OK with everyone?
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: BKCVS broken ?

[Erik Andersen]

On Thu Mar 17, 2005 at 04:10:53PM -0800, Larry McVoy wrote:
> I got swamped, I'll look at this after dinner.  But you might take a look
> at this: http://www.bitkeeper.com/press/2005-03-17.html which is a link
> to a very simple open source BK client.  It doesn't do much except track
> the head of the tree but it does that well.  It's slightly better than
> that, it puts all the checkin comments in BK/ChangeLog so you don't have
> to go over the wire to get those.
> 
> It's intended for someone who just wants the latest and greatest snapshot,
> knows how to do cp -rp and diff -Nur, it's pretty basic.  It's not a
> CVS gateway replacement but it does work for every tree on bkbits.net.
> Just to be clear, we are not dropping the CVS gateway, this is "in
> addition to" not "instead of".

Thanks!  Its nice to finally have an open source tool for sucking
down the latest and greatest directly from bk.  Thus far the tool
is working perfectly at fetching source trees and at updating
them when new patches are applied.

One minor nit.  The name for the 'update' tool is a bit too
generic...  For example old (old) linux systems have an
/sbin/update util for flushing buffers, and I have plenty of
'update' scripts lying around doing odd jobs.   Perhaps a rename
to 'sfioup' might be a good idea, as that is sufficiently obscure
there is little chance of a naming collision.

 -Erik

--
Erik B. Andersen             http://codepoet-consulting.com/
--This message was written using 73% post-consumer electrons--

Re: BKCVS broken ?

[Larry McVoy]

On Thu, Mar 17, 2005 at 10:50:40PM -0700, Erik Andersen wrote:
> On Thu Mar 17, 2005 at 04:10:53PM -0800, Larry McVoy wrote:
> > I got swamped, I'll look at this after dinner.  But you might take a look
> > at this: http://www.bitkeeper.com/press/2005-03-17.html which is a link
> > to a very simple open source BK client.  It doesn't do much except track
> > the head of the tree but it does that well.  It's slightly better than
> > that, it puts all the checkin comments in BK/ChangeLog so you don't have
> > to go over the wire to get those.
> > 
> > It's intended for someone who just wants the latest and greatest snapshot,
> > knows how to do cp -rp and diff -Nur, it's pretty basic.  It's not a
> > CVS gateway replacement but it does work for every tree on bkbits.net.
> > Just to be clear, we are not dropping the CVS gateway, this is "in
> > addition to" not "instead of".
> 
> Thanks!  Its nice to finally have an open source tool for sucking
> down the latest and greatest directly from bk.  Thus far the tool
> is working perfectly at fetching source trees and at updating
> them when new patches are applied.

Great.  It _should_ just work, I tested it with patches that included
binaries which changed, it handles that.  I suspect we'll find some
case which doesn't work some day (symlinks can't be represented in 
a patch for example) but you can always reget things from scratch,
that will work for contents, permissions, symlinks, the works.

> One minor nit.  The name for the 'update' tool is a bit too
> generic...  

Hey, it's open source, I'm hoping that people will take that code and
evolve it do whatever they need.  We're willing to do what we can on
this end if people need protocol changes to support new features, 
time permitting.  Think of that code as a prototype.  It's really
simple, you can hack it trivially.

If you want us to distribute your changes then send a patch, if not
that's cool too.  You can take that and evolve it to your heart's
content.  If you need a different license to start hacking let me
know what you want, I really don't care, you can have that code 
as public domain if you like.
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: BKCVS broken ?

[Stelian Pop]

On Thu, Mar 17, 2005 at 10:38:53PM -0800, Larry McVoy wrote:

> Hey, it's open source, I'm hoping that people will take that code and
> evolve it do whatever they need.  We're willing to do what we can on
> this end if people need protocol changes to support new features, 
> time permitting.  Think of that code as a prototype.  It's really
> simple, you can hack it trivially.

	------------
	if (strncmp("bk://", p, 5)) return (1);
	------------

Any chance this could be made to work over http ?

Stelian.
-- 
Stelian Pop <stelian@popies.net>    

Re: BKCVS broken ?

[Larry McVoy]

On Fri, Mar 18, 2005 at 10:00:49AM +0100, Stelian Pop wrote:
> On Thu, Mar 17, 2005 at 10:38:53PM -0800, Larry McVoy wrote:
> 
> > Hey, it's open source, I'm hoping that people will take that code and
> > evolve it do whatever they need.  We're willing to do what we can on
> > this end if people need protocol changes to support new features, 
> > time permitting.  Think of that code as a prototype.  It's really
> > simple, you can hack it trivially.
> 
> 	------------
> 	if (strncmp("bk://", p, 5)) return (1);
> 	------------
> 
> Any chance this could be made to work over http ?

I don't see why not.  It will take some hacking though.  Can you live 
without it for a bit or is it urgent?
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com

Re: BKCVS broken ?

[Stelian Pop]

On Fri, Mar 18, 2005 at 06:13:45AM -0800, Larry McVoy wrote:

> On Fri, Mar 18, 2005 at 10:00:49AM +0100, Stelian Pop wrote:
> > On Thu, Mar 17, 2005 at 10:38:53PM -0800, Larry McVoy wrote:
> > 
> > > Hey, it's open source, I'm hoping that people will take that code and
> > > evolve it do whatever they need.  We're willing to do what we can on
> > > this end if people need protocol changes to support new features, 
> > > time permitting.  Think of that code as a prototype.  It's really
> > > simple, you can hack it trivially.
> > 
> > 	------------
> > 	if (strncmp("bk://", p, 5)) return (1);
> > 	------------
> > 
> > Any chance this could be made to work over http ?
> 
> I don't see why not.  It will take some hacking though.  Can you live 
> without it for a bit or is it urgent?

It's not urgent at all...

Thanks.

Stelian.
-- 
Stelian Pop <stelian@popies.net>

Re: BKCVS broken ?

[Hong Kong Phoey]

On Fri, 18 Mar 2005 15:21:25 +0100, Stelian Pop <stelian@popies.net> wrote:
> On Fri, Mar 18, 2005 at 06:13:45AM -0800, Larry McVoy wrote:
> 
> > On Fri, Mar 18, 2005 at 10:00:49AM +0100, Stelian Pop wrote:
> > > On Thu, Mar 17, 2005 at 10:38:53PM -0800, Larry McVoy wrote:
> > >
> > > > Hey, it's open source, I'm hoping that people will take that code and
> > > > evolve it do whatever they need.  We're willing to do what we can on
> > > > this end if people need protocol changes to support new features,
> > > > time permitting.  Think of that code as a prototype.  It's really
> > > > simple, you can hack it trivially.
> > >
> > >     ------------
> > >     if (strncmp("bk://", p, 5)) return (1);
> > >     ------------
> > >
> > > Any chance this could be made to work over http ?
> >
> > I don't see why not.  It will take some hacking though.  Can you live
> > without it for a bit or is it urgent?
> 
> It's not urgent at all...
> 

IMHO, BKCVS is just fine, what's broken is your head.

> Thanks.
> 
> Stelian.
> --
> Stelian Pop <stelian@popies.net>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: BKCVS broken ?

[Stelian Pop]

On Fri, Mar 18, 2005 at 08:38:51PM +0530, Hong Kong Phoey wrote:

> IMHO, BKCVS is just fine, what's broken is your head.

Yup, that's what I said, that cvs HEAD is broken...

:)

Stelian.
-- 
Stelian Pop <stelian@popies.net>

Re: BKCVS broken ?

[Larry McVoy]

It should be fixed now, I'm running a full tree compare to validate that.
-- 
---
Larry McVoy                lm at bitmover.com           http://www.bitkeeper.com