Missing tags in my local stable git repo

Missing tags in my local stable git repo

[Steven Rostedt]

Hi Greg,

I'm curious if you are removing tags from your git repo on kernel.org.
I've been getting the following errors on my own repos. I have a
linus.git repo that mirrors, Linus's tree, and a stable.git repo based
off of my linus.git and it clones your stable.git tree on kernel.org.
But recently after daily updates and garbage collections, my work repo
that is based off of my stable.git repo gave me this:

error: refs/tags/v2.6.31.1 does not point to a valid object!
error: refs/tags/v2.6.31.10 does not point to a valid object!
error: refs/tags/v2.6.31.11 does not point to a valid object!
error: refs/tags/v2.6.31.12 does not point to a valid object!
error: refs/tags/v2.6.31.2 does not point to a valid object!
error: refs/tags/v2.6.31.3 does not point to a valid object!
error: refs/tags/v2.6.31.4 does not point to a valid object!
error: refs/tags/v2.6.31.5 does not point to a valid object!
error: refs/tags/v2.6.31.6 does not point to a valid object!
error: refs/tags/v2.6.31.7 does not point to a valid object!
error: refs/tags/v2.6.31.8 does not point to a valid object!
error: refs/tags/v2.6.31.9 does not point to a valid object!
error: refs/tags/v2.6.33.1 does not point to a valid object!
error: refs/tags/v2.6.33.2 does not point to a valid object!
error: refs/tags/v2.6.33.3 does not point to a valid object!
error: refs/tags/v2.6.33.4 does not point to a valid object!
error: refs/tags/v2.6.33.5 does not point to a valid object!
error: refs/tags/v2.6.33.6 does not point to a valid object!
error: refs/tags/v2.6.33.7 does not point to a valid object!

This repo has the following alternates:

$ cat .git/objects/info/alternates
/home/rostedt/work/git/nobackup/linus.git/objects
/home/rostedt/work/git/nobackup/stable.git/objects


$ cat ~/work/git/nobackup/stable.git/objects/info/alternates 
/home/rostedt/work/git/nobackup/linus.git//objects

And the stable.git remotes look like:

origin  /home/rostedt/work/git/nobackup/linus.git/ (fetch)
origin  /home/rostedt/work/git/nobackup/linus.git/ (push)
stable  git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git (fetch)
stable  git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git (push)

My linus.git tree is simply a bare tree that mirrors Linus's tree:

origin  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (fetch)
origin  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (push)

I have a crontab that updates these every night with a 'git remote
update' and once a week they do a 'git gc' on these repos. I notice the
above issues after the git gc.

I'm not sure where the problem arised, but did you do any cleanups of
tags on the stable git repo?

$ git fetch stable v2.6.33.4 
error: Could not read 15b059e0f9e31f0e7f6b5397368054c005949fea
error: Could not read 5e20b95750867086fff98e5c3eaa00c3bc01d8ee
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 1 (delta 0)
Unpacking objects: 100% (1/1), done.
fatal: bad object 4640b4e7d9919e9629fe8456df94f71658431ef9
error: git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git did not send all necessary objects


Again, I'm not sure if this is something with me, or something with the
kernel.org stable tree.

Thanks,

-- Steve

Re: Missing tags in my local stable git repo

[Greg Kroah-Hartman]

On Mon, Dec 10, 2012 at 12:35:00PM -0500, Steven Rostedt wrote:
> Hi Greg,
> 
> I'm curious if you are removing tags from your git repo on kernel.org.

No, not at all, and I can see these tags in my repo, and in the one on
kernel.org.

> And the stable.git remotes look like:
> 
> origin  /home/rostedt/work/git/nobackup/linus.git/ (fetch)
> origin  /home/rostedt/work/git/nobackup/linus.git/ (push)
> stable  git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git (fetch)
> stable  git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git (push)

Can you see the tags here in your local copy of the linux-stable.git
tree?

> My linus.git tree is simply a bare tree that mirrors Linus's tree:
> 
> origin  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (fetch)
> origin  git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git (push)
> 
> I have a crontab that updates these every night with a 'git remote
> update' and once a week they do a 'git gc' on these repos. I notice the
> above issues after the git gc.
> 
> I'm not sure where the problem arised, but did you do any cleanups of
> tags on the stable git repo?

No I do not, so I don't know what is going on, sorry.

Try cloning the linux-stable.git tree from scratch to see if you still
have this issue.  Odds are, something with all of the alternates is
getting confused somewhere.

greg k-h

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 10:12 -0800, Greg Kroah-Hartman wrote:

> Can you see the tags here in your local copy of the linux-stable.git
> tree?

No I couldn't.


> Try cloning the linux-stable.git tree from scratch to see if you still
> have this issue.  Odds are, something with all of the alternates is
> getting confused somewhere.
> 

Yep, that did the trick. I just deleted my local copy and recreated it,
and all the tags are back. Sorry for the noise. Must have been a glitch
in the way alternates work and the git gc's that I do.

-- Steve

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 12:35 -0500, Steven Rostedt wrote:

> I have a crontab that updates these every night with a 'git remote
> update' and once a week they do a 'git gc' on these repos. I notice the
> above issues after the git gc.

Looking at my script that does the git gc, it runs:

 git gc --aggressive

on the stable.git tree. I wonder if that made a difference.

Oh well, the problem is now solved, and I'll just redo my stable.git
tree if it shows up again.

-- Steve

Re: Missing tags in my local stable git repo

[Greg Kroah-Hartman]

On Mon, Dec 10, 2012 at 01:47:44PM -0500, Steven Rostedt wrote:
> On Mon, 2012-12-10 at 12:35 -0500, Steven Rostedt wrote:
> 
> > I have a crontab that updates these every night with a 'git remote
> > update' and once a week they do a 'git gc' on these repos. I notice the
> > above issues after the git gc.
> 
> Looking at my script that does the git gc, it runs:
> 
>  git gc --aggressive
> 
> on the stable.git tree. I wonder if that made a difference.

I think 'git gc' might have cause the problem, as it looks like it
pruned branches that you hadn't accessed in a long time, so it cleaned
them up.

greg k-h

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 11:02 -0800, Greg Kroah-Hartman wrote:
> On Mon, Dec 10, 2012 at 01:47:44PM -0500, Steven Rostedt wrote:
> > On Mon, 2012-12-10 at 12:35 -0500, Steven Rostedt wrote:
> > 
> > > I have a crontab that updates these every night with a 'git remote
> > > update' and once a week they do a 'git gc' on these repos. I notice the
> > > above issues after the git gc.
> > 
> > Looking at my script that does the git gc, it runs:
> > 
> >  git gc --aggressive
> > 
> > on the stable.git tree. I wonder if that made a difference.

I know we're beating a dead horse here, but it would be interesting to
know where the corruption came from.

> 
> I think 'git gc' might have cause the problem, as it looks like it
> pruned branches that you hadn't accessed in a long time, so it cleaned
> them up.

That's a scary thought... git gc will just prune stuff you haven't used
in a while? I hope that's not the case.

-- Steve

Re: Missing tags in my local stable git repo

[Linus Torvalds]

On Mon, Dec 10, 2012 at 11:15 AM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
> That's a scary thought... git gc will just prune stuff you haven't used
> in a while? I hope that's not the case.

git gc should only prune stuff that is no longer *reachable*.

If the alternate tree is entirely stable, that should be perfectly fine.

HOWEVER.

Any tree that is an alternate and ever has *any* objects in it that
can become unreachable is a really really really bad idea.

Note that the "any objects" can be because it has fetched temporary
branches (even if they were never merged). So you may think that you
didn't do anything unstable, but even just using that tree to fetch
something to look at - and then not using the end result - is a major
mistake in an alternates tree.

Also, you should never *ever* make an alternate object store be
*another* alternate object store, and I think that may be the problem
in your case. Iirc git doesn't follow alternates recursively. So what
can happen is that you got objects in the "middle" alternate, that
then get pruned away by "git gc" (because they exist in the origin
repository), but they got pruned away from the furthest alternate in
the meantime, and now the objects are "two hops" away, and will not
ever be seen.

Moral of the story: be *very* careful when using alternates. Only ever
point the alternate file at a *real* repository (not another one with
alternates), and only one that never has references removed.

                     Linus

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 11:30 -0800, Linus Torvalds wrote:

> Any tree that is an alternate and ever has *any* objects in it that
> can become unreachable is a really really really bad idea.

The only alternates that I use are my local copies of stable and your
tree, which should always be safe.

> Also, you should never *ever* make an alternate object store be
> *another* alternate object store, and I think that may be the problem
> in your case. Iirc git doesn't follow alternates recursively. So what
> can happen is that you got objects in the "middle" alternate, that
> then get pruned away by "git gc" (because they exist in the origin
> repository), but they got pruned away from the furthest alternate in
> the meantime, and now the objects are "two hops" away, and will not
> ever be seen.

OK, this may be my issue, as my stable.git has my copy of your tree as
an alternate. Maybe I can merge the two trees. Although I'm not exactly
sure how to. As I have my copies as both alternates as well as remotes
for my working repos, to get them to pull everything in with a remote
update. I have in my copy of the stable.git tree the following in the
config file:

[remote "stable"]
        url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
        fetch = +refs/heads/*:refs/heads/*

Otherwise I don't get the updates from stable when I do a remote fetch
of my local copies. Would something like this work?

[remote "origin"]
        fetch = +refs/*:refs/*
        mirror = true
        url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
[remote "stable"]
        url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
        fetch = +refs/heads/*:refs/heads/*

That is, if I have a single linus-stable.git repo that all my other
repos use as an alternate and a remote, I could have this repo updated
with:

git fetch
git remote update stable

and all my other repos will get all the updates from both your tree and
the stable tree with a single remote update of this copy?

Am I making any sense? ;-)


> 
> Moral of the story: be *very* careful when using alternates. Only ever
> point the alternate file at a *real* repository (not another one with
> alternates), and only one that never has references removed.

Note, I can just keep my normal set up, as it looks like the only damage
that can happen is my copy of stable.git tree getting corrupted (as it
did), and the fix is simply to blow it away and recreate it. No big
deal.

-- Steve

Re: Missing tags in my local stable git repo

[Linus Torvalds]

On Mon, Dec 10, 2012 at 12:44 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>
> OK, this may be my issue, as my stable.git has my copy of your tree as
> an alternate. Maybe I can merge the two trees.

That should work. Of course, it would depend on there not being any
clashes in the tag namespace, but we don't do that anyway (since it
would confuse everybody), so it should all be ok.

I would almost suggest you make both my base tree and the stable tree
be remotes, instead of necessarily mirroring my tree and then putting
the stable tree on top of that. I think it's confusing to have a
mirror but than in that same tree have an extra (non-mirror) remote,
it sounds more logical to me to have two separate remotes.

But yeah, you should always just be able to recreate things since
whatever you do your alternates will never have any extra work in
them, so you could just continue with your existing setup even though
it sounds a bit iffy.

                Linus

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 13:08 -0800, Linus Torvalds wrote:

> I would almost suggest you make both my base tree and the stable tree
> be remotes, instead of necessarily mirroring my tree and then putting
> the stable tree on top of that. I think it's confusing to have a
> mirror but than in that same tree have an extra (non-mirror) remote,
> it sounds more logical to me to have two separate remotes.

I could make them both remotes. The "mirror" and fetching is more
historical than anything. I use to only have a clone of your tree, and
then when I started maintaining the stable-rt tree, I created the stable
mirror as well. My workflow with git changed between the two creations,
and what I have currently shows that.

> 
> But yeah, you should always just be able to recreate things since
> whatever you do your alternates will never have any extra work in
> them, so you could just continue with your existing setup even though
> it sounds a bit iffy.

I may just keep my current setup. I'm now thinking that this breakage
was a side effect of a previous breakage due to the brown out I had a
while back. This is the same box that corrupted my linus.git tree that I
eventually just deleted and recreated. As the cron jobs that update the
repo (and the brown out, which caused my machine to reboot, happened
during the cron job), it could have affected my stable.git repo too.

This has been like this for a week or two, when I first noticed the
missing tags. I was just busy doing other things to ask about it.

Perhaps this has nothing to do with chained alternates and git gc going
crazy. It could have just been my linus.git tree getting corrupted, and
then the git gc run on my stable.git tree shortly after, and corrupting
that too. I don't reference the stable.git tree as often as I reference
the linus.git tree, thus, I didn't put the two corruptions together.

-- Steve

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 16:42 -0500, Steven Rostedt wrote:

> Perhaps this has nothing to do with chained alternates and git gc going
> crazy. It could have just been my linus.git tree getting corrupted, and
> then the git gc run on my stable.git tree shortly after, and corrupting
> that too. I don't reference the stable.git tree as often as I reference
> the linus.git tree, thus, I didn't put the two corruptions together.
> 

Hmm, finding my post on G+:

 https://plus.google.com/u/0/116317370665849559569/posts/Nd3HJkXAYCR

That happened on Nov 13th, almost a month ago. Hmm, I would think that I
would have noticed this earlier. Maybe the corruption was delayed?

-- Steve

Re: Missing tags in my local stable git repo

[Andreas Schwab]

Steven Rostedt <rostedt@goodmis.org> writes:

> [remote "origin"]
>         fetch = +refs/*:refs/*
>         mirror = true
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> [remote "stable"]
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>         fetch = +refs/heads/*:refs/heads/*

This is your problem.  When fetching from origin all references
(including tags) not part of origin are removed.  When you mirror a repo
you cannot have another remote in the same repo.

Andreas.

-- 
Andreas Schwab, schwab@linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 22:12 +0100, Andreas Schwab wrote:
> Steven Rostedt <rostedt@goodmis.org> writes:
> 
> > [remote "origin"]
> >         fetch = +refs/*:refs/*
> >         mirror = true
> >         url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> > [remote "stable"]
> >         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
> >         fetch = +refs/heads/*:refs/heads/*
> 
> This is your problem.  When fetching from origin all references
> (including tags) not part of origin are removed.  When you mirror a repo
> you cannot have another remote in the same repo.

Note, the above was my proposal to merge stable.git with linus.git. It's
not what I currently have.

-- Steve

Re: Missing tags in my local stable git repo

[Geert Uytterhoeven]

On Mon, Dec 10, 2012 at 9:44 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
> Otherwise I don't get the updates from stable when I do a remote fetch
> of my local copies. Would something like this work?
>
> [remote "origin"]
>         fetch = +refs/*:refs/*
>         mirror = true
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> [remote "stable"]
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>         fetch = +refs/heads/*:refs/heads/*
>
> That is, if I have a single linus-stable.git repo that all my other
> repos use as an alternate and a remote, I could have this repo updated
> with:

That's actually what I have as my "master" repository.

My work repository references this as an alternate, and its .git subdirectory is
a symlink to space that is backed up (the alternates don't need backup as half
mirrors Linus' tree and the stable tree).

> git fetch
> git remote update stable

A single "git remote update" will fetch objects for both origin and the stable
remote.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Missing tags in my local stable git repo

[Geert Uytterhoeven]

On Mon, Dec 10, 2012 at 10:16 PM, Geert Uytterhoeven
<geert@linux-m68k.org> wrote:
> On Mon, Dec 10, 2012 at 9:44 PM, Steven Rostedt <rostedt@goodmis.org> wrote:
>> Otherwise I don't get the updates from stable when I do a remote fetch
>> of my local copies. Would something like this work?
>>
>> [remote "origin"]
>>         fetch = +refs/*:refs/*
>>         mirror = true
>>         url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
>> [remote "stable"]
>>         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>>         fetch = +refs/heads/*:refs/heads/*
>>
>> That is, if I have a single linus-stable.git repo that all my other
>> repos use as an alternate and a remote, I could have this repo updated
>> with:
>
> That's actually what I have as my "master" repository.

After considering Andreas's response: I don't have "mirror = true". I.e. my
master repo is a real clone, with stable added as a remote.

Gr{oetje,eeting}s,

                        Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
                                -- Linus Torvalds

Re: Missing tags in my local stable git repo

[Greg Kroah-Hartman]

On Mon, Dec 10, 2012 at 03:44:34PM -0500, Steven Rostedt wrote:
> On Mon, 2012-12-10 at 11:30 -0800, Linus Torvalds wrote:
> 
> > Any tree that is an alternate and ever has *any* objects in it that
> > can become unreachable is a really really really bad idea.
> 
> The only alternates that I use are my local copies of stable and your
> tree, which should always be safe.
> 
> > Also, you should never *ever* make an alternate object store be
> > *another* alternate object store, and I think that may be the problem
> > in your case. Iirc git doesn't follow alternates recursively. So what
> > can happen is that you got objects in the "middle" alternate, that
> > then get pruned away by "git gc" (because they exist in the origin
> > repository), but they got pruned away from the furthest alternate in
> > the meantime, and now the objects are "two hops" away, and will not
> > ever be seen.
> 
> OK, this may be my issue, as my stable.git has my copy of your tree as
> an alternate. Maybe I can merge the two trees. Although I'm not exactly
> sure how to. As I have my copies as both alternates as well as remotes
> for my working repos, to get them to pull everything in with a remote
> update. I have in my copy of the stable.git tree the following in the
> config file:
> 
> [remote "stable"]
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>         fetch = +refs/heads/*:refs/heads/*
> 
> Otherwise I don't get the updates from stable when I do a remote fetch
> of my local copies. Would something like this work?
> 
> [remote "origin"]
>         fetch = +refs/*:refs/*
>         mirror = true
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
> [remote "stable"]
>         url = git://git.kernel.org/pub/scm/linux/kernel/git/stable/linux-stable.git
>         fetch = +refs/heads/*:refs/heads/*
> 
> That is, if I have a single linus-stable.git repo that all my other
> repos use as an alternate and a remote, I could have this repo updated
> with:
> 
> git fetch
> git remote update stable
> 
> and all my other repos will get all the updates from both your tree and
> the stable tree with a single remote update of this copy?
> 
> Am I making any sense? ;-)

Kind of, but would would be even easier is if you just have one
"origin", pointing at linux-stable.git, and then every so often just do:
	git checkout master
	git pull git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

As the master branch of linux-stable.git always is a "clean" linux.git
tree, although it might lag a few -rc releases back if I have forgotten
to update it recently.  When it does get updated, all should be fine as
the master branch has no merges from anything else, so it's just a
fast-forward.

That might be simpler overall than the different repos to be pulling
from in different ways.

greg k-h

Re: Missing tags in my local stable git repo

[Steven Rostedt]

On Mon, 2012-12-10 at 13:18 -0800, Greg Kroah-Hartman wrote:

> > Am I making any sense? ;-)
> 
> Kind of,

I'm glad you understand it, because I sure don't ;-)

>  but would would be even easier is if you just have one
> "origin", pointing at linux-stable.git, and then every so often just do:
> 	git checkout master
> 	git pull git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git

Both my stable.git and my linus.git are bare repositories. No working
directory, as that would add a lot of unnecessary files and waste disk
space. You can't do a git checkout on a bare repository.

> 
> As the master branch of linux-stable.git always is a "clean" linux.git
> tree, although it might lag a few -rc releases back if I have forgotten
> to update it recently.  When it does get updated, all should be fine as
> the master branch has no merges from anything else, so it's just a
> fast-forward.

I also need the latest linus tree all the time. Unless your stable tree
has a branch from Linus's master, I will still need to directly get
Linus's tree.

> 
> That might be simpler overall than the different repos to be pulling
> from in different ways.

Unfortunately it wont work due to the requirement that the repos be
bare.

-- Steve

Re: Missing tags in my local stable git repo

[Stephen Rothwell]

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

Hi Linus,

On Mon, 10 Dec 2012 11:30:55 -0800 Linus Torvalds <torvalds@linux-foundation.org> wrote:
>
> Moral of the story: be *very* careful when using alternates. Only ever
> point the alternate file at a *real* repository (not another one with
> alternates), and only one that never has references removed.

Does that mean your tree could now be a problem, since you have recently
added and then removed a branch (some of whose commits were not eventually
committed to your master branch, I think)?

-- 
Cheers,
Stephen Rothwell                    sfr@canb.auug.org.au

[-- Attachment #2: Type: application/pgp-signature, Size: 836 bytes --]

Re: Missing tags in my local stable git repo

[Linus Torvalds]

On Mon, Dec 10, 2012 at 1:28 PM, Stephen Rothwell <sfr@canb.auug.org.au> wrote:
>
> Does that mean your tree could now be a problem, since you have recently
> added and then removed a branch (some of whose commits were not eventually
> committed to your master branch, I think)?

All of them were committed before that branch was removed, but yes, at
one point I did rebase that branch.

That said, as an alternate that should still be safe, assuming you
didn't then do any work on top of that rebased branch (ie your repo
doesn't have any references to any of those objects).

                   Linus