kernel.org git tree corrupt?

kernel.org git tree corrupt?

[Steve French]

In preparation for a merge request I do
    cg-diff -m -r origin: | diffstat -p1
(git diff -m -r origin: shows the same problem) and
    cg-log -m -r origin:
from my kernel.org cifs-2.6.git tree against the linux-2.6.git which
should only show a small set of changes (typically only affecting one
directory in the kernel source), but instead now is showing thousands
of changesets with changed lines all over the tree.   The origin is
        /pub/scm/linux/kernel/git/torvalds/linux-2.6.git
and the only operation done on cifs-2.6.git is git pull (to merge in
changes from linux-2.6.git) and applying (patch and cg-commit) small
patches (to the fs/cifs directory typically).    Even after doing git
pull again (and also git-fsck) that did not seem to help as there are
still thousands of changesets showing up when there should be fewer
than 30.   Was the linux-2.6.git rebuilt in the last four weeks with
different changeset numbers? Something seems to have drastically
changed in the kernel.org tree in the last few weeks.

Any idea how to get around this ... short of going through all patches
to fs/cifs saving the uncommitted ones off one by one by hand,
recloning the tree and reapplying them one by one?
Doing "cg-log -m -r origin: fs/cifs" shows changesets that are already
in linux-2.6.git (e.g. the vfs-wide changes to add path_put
1d957f9bf87da74f420424d16ece005202bbebd3 and
4ac9137858e08a19f29feac4e1f4df7c268b0ba5 )

The oldest patch which affects fs/cifs which incorrectly shows up as
not in mainline (ie shows up with cg-diff but is actually merged into
both mainline and my tree, and submitted and merged by someone else)
is eebd2aa355692afaf9906f62118620f1a1c19dbb dated 2/4/2008 so I assume
that the problem started around then.

Ideas for another solution short of recloning and sorting through all
of the last month or two of patches that affect this directory by
hand?


-- 
Thanks,

Steve

Re: kernel.org git tree corrupt?

[Linus Torvalds]

On Sun, 2 Mar 2008, Steve French wrote:
>
> In preparation for a merge request I do
>     cg-diff -m -r origin: | diffstat -p1
> (git diff -m -r origin: shows the same problem) and
>     cg-log -m -r origin:

First off, you really should lay off the cogito thing, it's pretty much 
guaranteed that any cogito usage will just be harder and less likely to be 
correct than just usign native git (and almost nobody will be able to help 
you any more - it's not like it's been maintained for the last year).

It *sounds* like your "origin" isn't what you expect it to be. 

> Was the linux-2.6.git rebuilt in the last four weeks with different 
> changeset numbers? Something seems to have drastically changed in the 
> kernel.org tree in the last few weeks.

Nope, nothing has been rebuilt, but there's been another merge window, so 
there's certainly been drastic changes.

> Doing "cg-log -m -r origin: fs/cifs" shows changesets that are already
> in linux-2.6.git (e.g. the vfs-wide changes to add path_put
> 1d957f9bf87da74f420424d16ece005202bbebd3 and
> 4ac9137858e08a19f29feac4e1f4df7c268b0ba5 )

In that case, your "origin" simply isn't updated, but points somewhere 
long back in history. I can only assume that cogito has done something 
wrong, like not been able to handle packed refs or something, and you have 
an updated tree but "origin" pointing to way back in the history.

> Ideas for another solution short of recloning and sorting through all
> of the last month or two of patches that affect this directory by
> hand?

Do this:

 - get rid of cogito (or at least make sure that it doesn't mess up the 
   following sequence)

 - look at your .git/config file to make sure that the remote branches are 
   sane. It should have something like this (and if it doesn't, add it):

	[remote "origin"]
		url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
		fetch = +refs/heads/*:refs/remotes/origin/*

	[branch "master"]
		remote = origin
		merge = refs/heads/master

 - look at your "origin" branch, and make sure it's a *remote* branch, not 
   the old-style local one. Ie it should *not* show up when you do a 
   plain

	git branch

   but it *should* show up (as both "origin/HEAD" and "origin/master") 
   when you do

	git branch -a

   and if it's a local branch (very possible, since you're using cogito 
   that may not even understand remote branches), then just blow it away 
   entirely with "git branch -D origin".

 - do "git fetch" to make sure that the remote branches are up-to-date

 - now do "git log origin" an it should show something recent like

	commit 038f2f725503b55ab76cfd2645915a85594710fe
	Merge: 8d07a67... 855c603...
	Author: Linus Torvalds <torvalds@woody.linux-foundation.org>
	Date:   Sun Mar 2 12:38:17 2008 -0800
	
	    Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ieee1394/linux139
	...

 - and if it does, now you should be all good, and you can dothe normal 
   "gitk origin.." and see the commits that are in your tree but not in 
   mine.

Now your setup should be ok again. I *suspect* that what has happened is 
that you got a local "origin" branch - probably through cogito - that hid 
the remote one. 

		Linus

Re: kernel.org git tree corrupt?

[Martin Langhoff]

On Mon, Mar 3, 2008 at 10:16 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>  In that case, your "origin" simply isn't updated, but points somewhere
>  long back in history. I can only assume that cogito has done something
>  wrong, like not been able to handle packed refs or something, and you have
>  an updated tree but "origin" pointing to way back in the history.

Strange - cogito over git protocol will use git itself to peek at the
references. Steve, what is your git version?

>  > Ideas for another solution short of recloning and sorting through all
>  > of the last month or two of patches that affect this directory by
>  > hand?

Linus' plan assumes you have a .git/config file. I don't think a
cogito-based checkout follows such modern conventions - here's an
alternative plan:

 - make sure you have a current git

 - get a fresh clone of linux-2.6, and in there

    # tell it about your old checkout
    $ git remote add oldlinux /path/to/your/older/linux/checkout/.git
    $ git fetch oldlinux
    # visualise what's in there
    $ gitk origin/master oldlinux/master

    # you may want to merge your old branch
    $ git merge oldlinux/master

hth,


martin

Re: kernel.org git tree corrupt?

[Steve French]

On Sun, Mar 2, 2008 at 3:40 PM, Martin Langhoff
<martin.langhoff@gmail.com> wrote:
> On Mon, Mar 3, 2008 at 10:16 AM, Linus Torvalds
>  <torvalds@linux-foundation.org> wrote:
>  >  In that case, your "origin" simply isn't updated, but points somewhere
>  >  long back in history. I can only assume that cogito has done something
>  >  wrong, like not been able to handle packed refs or something, and you have
>  >  an updated tree but "origin" pointing to way back in the history.
>
>  Strange - cogito over git protocol will use git itself to peek at the
>  references. Steve, what is your git version?

This is master.kernel.org so git version is 1.5.4.3

-- 
Thanks,

Steve

Re: kernel.org git tree corrupt?

[Steve French]

On Sun, Mar 2, 2008 at 3:16 PM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>  On Sun, 2 Mar 2008, Steve French wrote:
>  >
>  > In preparation for a merge request I do
>  >     cg-diff -m -r origin: | diffstat -p1
>  > (git diff -m -r origin: shows the same problem) and
>  >     cg-log -m -r origin:
>
>  First off, you really should lay off the cogito thing
no problem

>   - look at your .git/config file to make sure that the remote branches are
>    sane. It should have something like this (and if it doesn't, add it):
>
>         [remote "origin"]
>                 url = git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux-2.6
>                 fetch = +refs/heads/*:refs/remotes/origin/*
>
>         [branch "master"]
>                 remote = origin
>                 merge = refs/heads/master
.git/config now includes

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[remote "origin"]
        url = /pub/scm/linux/kernel/git/torvalds/linux-2.6.git
        fetch = +refs/heads/*:refs/remotes/origin/*
[branch "master"]
        remote = origin
        merge = refs/heads/master


>   - look at your "origin" branch, and make sure it's a *remote* branch, not
>    the old-style local one. Ie it should *not* show up when you do a
>    plain
>
>         git branch
now only shows "* master"

>    but it *should* show up (as both "origin/HEAD" and "origin/master")
>    when you do
>
>         git branch -a
Now "git branch -a" shows
* master
  origin/master

It is missing "origin/HEAD"
>   - now do "git log origin" an it should show something recent
git log origin and git log origin/master both return
"ambiguous argument 'origin': unknown revision"

I must be misunderstanding the syntax/manpage of git-log but "git diff
-m -r origin/master"
now works fine and displays exactly what I expect so things have improved.



>  Now your setup should be ok again. I *suspect* that what has happened is
>  that you got a local "origin" branch - probably through cogito - that hid
>  the remote one.
This seems plausible and


-- 
Thanks,

Steve

Re: kernel.org git tree corrupt?

[Linus Torvalds]

On Sun, 2 Mar 2008, Steve French wrote:
> 
> >   - look at your "origin" branch, and make sure it's a *remote* branch, not
> >    the old-style local one. Ie it should *not* show up when you do a
> >    plain
> >
> >         git branch
> now only shows "* master"
> 
> >    but it *should* show up (as both "origin/HEAD" and "origin/master")
> >    when you do
> >
> >         git branch -a
> Now "git branch -a" shows
> * master
>   origin/master
> 
> It is missing "origin/HEAD"

Ahh, yeah, my bad. The origin/HEAD thing will be created if you use either 
clone or "git remote add -m master" to create the remote. But when I asked 
you to do it by just editing the config file, you now have to create that 
HEAD pointer manually too.

You can do

	git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master

to create the thing (it just creates a symbolic ref from the origin/HEAD 
remote ref to the origin/master one, so now git will know that when you 
talk about 'origin', it is supposed to just use the master branch of 
that remote).

> >   - now do "git log origin" an it should show something recent
> git log origin and git log origin/master both return
> "ambiguous argument 'origin': unknown revision"

Well, origin/master should have worked, but the fact that plain "origin" 
didn't work is due to exactly the lack of HEAD file for that remote branch 
(for your kernel tree, there's only one remote branch, so it may be 
"obvious" that origin must be talking about the master branch, but if 
there are multiple branches at the origin it's not obvious *which* branch 
should be considered the default one, which is why we use HEAD)

> I must be misunderstanding the syntax/manpage of git-log but "git diff
> -m -r origin/master"
> now works fine and displays exactly what I expect so things have improved.

Well, if "origin/master" works in that situation, then "origin/master" 
should have worked in "git log" too, so maybe you mistyped?

Anyway, with the HEAD link added for the remote 'origin', you should be 
able to use plain 'origin' like you're used to.

			Linus

Re: kernel.org git tree corrupt?

[Steve French]

On Mon, Mar 3, 2008 at 10:21 AM, Linus Torvalds
<torvalds@linux-foundation.org> wrote:
>
>  On Sun, 2 Mar 2008, Steve French wrote:
>  > Now "git branch -a" shows
>  > * master
>  >   origin/master
>  >
>  > It is missing "origin/HEAD"
>
>  Ahh, yeah, my bad. The origin/HEAD thing will be created if you use either
>  clone or "git remote add -m master" to create the remote. But when I asked
>  you to do it by just editing the config file, you now have to create that
>  HEAD pointer manually too.
>
>  You can do
>
>         git symbolic-ref refs/remotes/origin/HEAD refs/remotes/origin/master
>
>  to create the thing (it just creates a symbolic ref from the origin/HEAD
Worked.

Shaggy pointed me to the git-request-pull script which is
easier than the previous scripts I used and now works fine on
the cifs-2.6.git tree so will send merge request.


-- 
Thanks,

Steve