process committed files in post-receive hook

process committed files in post-receive hook

[Hao]

Hi guys,

I am writing a post-receive hook in Python that examines the content of some 
files (the HEAD rev). Because the repo is a bare one on the server. My current 
approach is to check out a working copy on the server and run 'git pull' in post-
receive to get the most up-to-date version, and then process files in the 
working copy.

I have two questions. First, is there a way that I can access file content in a 
bare repo without checking out a working copy? If this is not possible, my 
approach would be reasonable. However, when 'git pull' was called in the python 
script post-receive when a commit occurs, it gives an error.

remote: fatal: Not a git repository: '.'

The call in python is

subprocess.Popen(["git", "pull"], cwd="/Users/git/ts.git.workingcopy")

I read from a post (http://stackoverflow.com/questions/4043609/) that GIT_DIR is 
causing this error. Is it safe to unset GIT_DIR in post-receive?

Thanks a lot.

Re: process committed files in post-receive hook

[Michael Schubert]

On 12/10/2011 11:29 AM, Hao wrote:
> I am writing a post-receive hook in Python that examines the content of some 
> files (the HEAD rev). Because the repo is a bare one on the server. My current 
> approach is to check out a working copy on the server and run 'git pull' in post-
> receive to get the most up-to-date version, and then process files in the 
> working copy.

You could do something like this as a post-receive hook:

#!/bin/sh

test_dir=$(mktemp -d /tmp/test.XXXXXXXXXX)
GIT_WORK_TREE=$test_dir git checkout -f
/usr/local/bin/check.py $test_dir
rm -rf $test_dir

Re: process committed files in post-receive hook

[Ivan Heffner]

On Sat, Dec 10, 2011 at 3:21 AM, Michael Schubert <mschub@elegosoft.com> wrote:
> On 12/10/2011 11:29 AM, Hao wrote:
>> I am writing a post-receive hook in Python that examines the content of some
>> files (the HEAD rev). Because the repo is a bare one on the server. My current
>> approach is to check out a working copy on the server and run 'git pull' in post-
>> receive to get the most up-to-date version, and then process files in the
>> working copy.
>

You can actually use a combination of git ls-files and git cat-file -p
in order to list and look at te content of files on the remote without
checking out an entire working tree.

Re: process committed files in post-receive hook

[Alexey Shumkin]

> I have two questions. First, is there a way that I can access file
> content in a bare repo without checking out a working copy? 

$ git show <commit>:<filename>

e.g. for bare repo of Git the following command
$ git show v1.7.8:Documentation/RelNotes/1.7.8.txt
outputs
--->8---
Git v1.7.8 Release Notes
========================

Updates since v1.7.7
--------------------

 * Some git-svn, git-gui, git-p4 (in contrib) and msysgit updates.

 * Updates to bash completion scripts.
--->8---

Re: process committed files in post-receive hook

[Neal Kreitzinger]

On 12/10/2011 4:29 AM, Hao wrote:
> Hi guys,
>
> I am writing a post-receive hook in Python that examines the content
> of some files (the HEAD rev). Because the repo is a bare one on the
> server. My current approach is to check out a working copy on the
> server and run 'git pull' in post- receive to get the most up-to-date
> version, and then process files in the working copy.
>
> I have two questions. First, is there a way that I can access file
> content in a bare repo without checking out a working copy? If this
> is not possible, my approach would be reasonable. However, when 'git
> pull' was called in the python script post-receive when a commit
> occurs, it gives an error.
>
> remote: fatal: Not a git repository: '.'
>
> The call in python is
>
> subprocess.Popen(["git", "pull"],
> cwd="/Users/git/ts.git.workingcopy")
>
> I read from a post (http://stackoverflow.com/questions/4043609/) that
> GIT_DIR is causing this error. Is it safe to unset GIT_DIR in
> post-receive?
>
The specific processing you intend to perform on the files would
determine which of the access techniques is appropriate for you.
Generally speaking, I think a checkout in a non-bare repo makes sense. 
You could limit it to a shallow clone (see git-clone manpage) to save space.

Another way to get the files is git-archive (creates tar file), that you 
could extract to a dir for processing.

In both cases, you need to consider the default permissions in play with 
git-checkout and git-archive if permissions are important in your 
processing.

v/r,
neal

Re: process committed files in post-receive hook

[Hao Wang]

Thank you all for providing the options. Just so you know I finally went 
with Alexey's suggestion. I used 'git show' to get both a list of files 
in a directory and the content of each file. It works great on a bare 
repository so there is no need to check out a copy on the server.

Below is the python code in my post-receive hook for this task, where 
rev is something like 'HEAD:directory_name' for the first function and 
'HEAD:directory/filename' for the second function.

# get a list of rule files using git show
def getRuleFileList(rev):
     # run git show
     p = subprocess.Popen(['git', 'show', rev], stdout=subprocess.PIPE)
     p.wait()
     if p.returncode != 0: return None # error

     # parse output
     i = 0
     filelist = []
     for line in p.stdout.readlines():
         filelist.append(line)
     p.stdout.close()
     return filelist

# read the content of a file
def readfile(rev):
     # run git show
     p = subprocess.Popen(['git', 'show', rev], stdout=subprocess.PIPE)
     p.wait()
     if p.returncode != 0: return None # error
     return p.stdout.read()

Hao

On 12/14/11 5:04 PM, Neal Kreitzinger wrote:
> On 12/10/2011 4:29 AM, Hao wrote:
>> Hi guys,
>>
>> I am writing a post-receive hook in Python that examines the content
>> of some files (the HEAD rev). Because the repo is a bare one on the
>> server. My current approach is to check out a working copy on the
>> server and run 'git pull' in post- receive to get the most up-to-date
>> version, and then process files in the working copy.
>>
>> I have two questions. First, is there a way that I can access file
>> content in a bare repo without checking out a working copy? If this
>> is not possible, my approach would be reasonable. However, when 'git
>> pull' was called in the python script post-receive when a commit
>> occurs, it gives an error.
>>
>> remote: fatal: Not a git repository: '.'
>>
>> The call in python is
>>
>> subprocess.Popen(["git", "pull"],
>> cwd="/Users/git/ts.git.workingcopy")
>>
>> I read from a post (http://stackoverflow.com/questions/4043609/) that
>> GIT_DIR is causing this error. Is it safe to unset GIT_DIR in
>> post-receive?
>>
> The specific processing you intend to perform on the files would
> determine which of the access techniques is appropriate for you.
> Generally speaking, I think a checkout in a non-bare repo makes sense.
> You could limit it to a shallow clone (see git-clone manpage) to save
> space.
>
> Another way to get the files is git-archive (creates tar file), that you
> could extract to a dir for processing.
>
> In both cases, you need to consider the default permissions in play with
> git-checkout and git-archive if permissions are important in your
> processing.
>
> v/r,
> neal

Re: process committed files in post-receive hook

[Jeff King]

On Wed, Dec 14, 2011 at 06:02:11PM -0800, Hao Wang wrote:

> Thank you all for providing the options. Just so you know I finally
> went with Alexey's suggestion. I used 'git show' to get both a list
> of files in a directory and the content of each file. It works great
> on a bare repository so there is no need to check out a copy on the
> server.

If you are scripting, we usually encourage the use of "plumbing"
commands whose output is guaranteed not to change ("show" is a
"porcelain" command intended to be used by end-users, and it's possible
that its behavior might change from version to version).

The plumbing command to get a directory listing for a tree is "git
ls-tree" (try the "--name-only" option for terse output, and use "-z" if
you want to be robust in the face of filenames with funny characters).

> # get a list of rule files using git show
> def getRuleFileList(rev):
>     # run git show
>     p = subprocess.Popen(['git', 'show', rev], stdout=subprocess.PIPE)
>     p.wait()
>     if p.returncode != 0: return None # error
> 
>     # parse output
>     i = 0
>     filelist = []
>     for line in p.stdout.readlines():
>         filelist.append(line)
>     p.stdout.close()
>     return filelist

Doesn't this put "tree HEAD:foo", as printed by "git show", at the top
of your filelist? Another reason to use ls-tree.

> # read the content of a file
> def readfile(rev):
>     # run git show
>     p = subprocess.Popen(['git', 'show', rev], stdout=subprocess.PIPE)
>     p.wait()
>     if p.returncode != 0: return None # error
>     return p.stdout.read()

The plumbing for this is "git cat-file blob ...".

-Peff

Re: process committed files in post-receive hook

[Hao Wang]

> If you are scripting, we usually encourage the use of "plumbing"
> commands whose output is guaranteed not to change ("show" is a
> "porcelain" command intended to be used by end-users, and it's possible
> that its behavior might change from version to version).
>
> The plumbing command to get a directory listing for a tree is "git
> ls-tree" (try the "--name-only" option for terse output, and use "-z" if
> you want to be robust in the face of filenames with funny characters).

Jeff, thank you for the information. This is really helpful.

>> # get a list of rule files using git show
>> def getRuleFileList(rev):
>>      # run git show
>>      p = subprocess.Popen(['git', 'show', rev], stdout=subprocess.PIPE)
>>      p.wait()
>>      if p.returncode != 0: return None # error
>>
>>      # parse output
>>      i = 0
>>      filelist = []
>>      for line in p.stdout.readlines():
>>          filelist.append(line)
>>      p.stdout.close()
>>      return filelist
>
> Doesn't this put "tree HEAD:foo", as printed by "git show", at the top
> of your filelist? Another reason to use ls-tree.

Yes, the first two items ("tree HEAD:foo" and an empty line) are removed 
later from filelist.

Hao