Using git to store /etc, redux

Using git to store /etc, redux

[David Härdeman]

I recently had the idea to store and track /etc using git. When googling 
the topic I came across the "Using git to store /etc" thread from the 
end of last year which provided some interesting details on what would 
be necessary.

It seems the file metadata (owner, group, mode, xattrs, etc) was the big 
stumbling point, so I wrote up a tool over the last few days which 
allows the metadata to be stored in a separate file which can be stored 
along with the rest of the data in the repo (or separately).

This is also useful for tripwire type checks and for other types of 
storage which drops some of the metadata (tar comes to mind)...

The tool (metastore) is available from: 
git://git.hardeman.nu/metastore.git

Not completely cleaned up yet (it lacks a real README and some Makefile 
targets) but I hope it might be useful to others (it sure is to me).

Please CC me on any replies.

-- 
David Härdeman

Re: Using git to store /etc, redux

[david]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2001 bytes --]

On Sat, 19 May 2007, David Härdeman wrote:

> I recently had the idea to store and track /etc using git. When googling the 
> topic I came across the "Using git to store /etc" thread from the end of last 
> year which provided some interesting details on what would be necessary.
>
> It seems the file metadata (owner, group, mode, xattrs, etc) was the big 
> stumbling point, so I wrote up a tool over the last few days which allows the 
> metadata to be stored in a separate file which can be stored along with the 
> rest of the data in the repo (or separately).
>
> This is also useful for tripwire type checks and for other types of storage 
> which drops some of the metadata (tar comes to mind)...
>
> The tool (metastore) is available from: git://git.hardeman.nu/metastore.git
>
> Not completely cleaned up yet (it lacks a real README and some Makefile 
> targets) but I hope it might be useful to others (it sure is to me).
>
> Please CC me on any replies.

as I understand the issue, the problem isn't creating a tool to store the 
metadata, but in integrating things with git.

when checking something in a pre-commit hook needs to run the tool to 
store the data.

git supports this and it's pretty simple to do this.

however when checking things out there are approaches

1. modify git to have a post-checkout hook to set the metadata to match
    what was stored at checkin and accept the fact that this leaves a
    window where the file has the wrong metadata on it (between when the
    file is written and when the hook runs), or use a staging area to have
    copies of the files during check-in and check-out

2. modify git to know that it needs to check some files out before any
    others and use an expernal program to write the files to disk. Then
    this program can use the data stored at checkin to write the files with
    the appropriate metadata

unforutnantly until one of these is done by someone the utility of 
programs like your metastore are limited.

David Lang

Re: Using git to store /etc, redux

[David Härdeman]

On Sat, May 19, 2007 at 04:37:54PM -0700, david@lang.hm wrote:
>On Sat, 19 May 2007, David Härdeman wrote:
>
>>I recently had the idea to store and track /etc using git. When googling 
>>the topic I came across the "Using git to store /etc" thread from the end 
>>of last year which provided some interesting details on what would be 
>>necessary.
>>
>>It seems the file metadata (owner, group, mode, xattrs, etc) was the big 
>>stumbling point, so I wrote up a tool over the last few days which allows 
>>the metadata to be stored in a separate file which can be stored along 
>>with the rest of the data in the repo (or separately).
>>
>>This is also useful for tripwire type checks and for other types of 
>>storage which drops some of the metadata (tar comes to mind)...
>>
>>The tool (metastore) is available from: git://git.hardeman.nu/metastore.git
>>
>>Not completely cleaned up yet (it lacks a real README and some Makefile 
>>targets) but I hope it might be useful to others (it sure is to me).
>>
>>Please CC me on any replies.
>
>as I understand the issue, the problem isn't creating a tool to store the 
>metadata, but in integrating things with git.

That is also important of course, the problem is that there are many 
different scenarios for how people might want to work with the metadata 
(e.g. whether changed metadata should be stored automatically or only 
with user interaction, etc).

For the "store /etc in git" solution which is what got me into this, it 
might be enough to have a pre-commit hook if all changes are made in 
/etc and committed to /etc/.git periodically (meaning there are no real 
checkouts to speak of).

>when checking something in a pre-commit hook needs to run the tool to 
>store the data.
>
>git supports this and it's pretty simple to do this.

Yes, I already have hook scripts in my local setup which does this, 
it was not clear whether pre-commit hooks could change the commit by 
adding more files to be committed but it seems to work, essentially 
the pre-commit hook is just:

metastore -s
git-add .metadata

>however when checking things out there are approaches
>
>1. modify git to have a post-checkout hook to set the metadata to match
>    what was stored at checkin and accept the fact that this leaves a
>    window where the file has the wrong metadata on it (between when the
>    file is written and when the hook runs), or use a staging area to have
>    copies of the files during check-in and check-out

Right, I use a non-hook script for this right now which changes umask to 
0077, pulls the changes, shows the difference in metadata and asks for 
confirmation and then applies the metadata (which undoes the effects of 
the umask setting).

I think I'll add both scripts to my git repo as examples soon.

-- 
David Härdeman

Re: Using git to store /etc, redux

[Jan Hudec]

[-- Attachment #1: Type: text/plain, Size: 1420 bytes --]

On Sat, May 19, 2007 at 19:48:15 +0200, David Härdeman wrote:
> I recently had the idea to store and track /etc using git. When googling 
> the topic I came across the "Using git to store /etc" thread from the 
> end of last year which provided some interesting details on what would 
> be necessary.
> 
> It seems the file metadata (owner, group, mode, xattrs, etc) was the big 
> stumbling point, so I wrote up a tool over the last few days which 
> allows the metadata to be stored in a separate file which can be stored 
> along with the rest of the data in the repo (or separately).
> 
> This is also useful for tripwire type checks and for other types of 
> storage which drops some of the metadata (tar comes to mind)...
> 
> The tool (metastore) is available from: 
> git://git.hardeman.nu/metastore.git
> 
> Not completely cleaned up yet (it lacks a real README and some Makefile 
> targets) but I hope it might be useful to others (it sure is to me).
> 
> Please CC me on any replies.

Have you looked at IsiSetup (http://www.isisetup.ch/, linked from
http://git.or.cz/gitwiki/InterfacesFrontendsAndTools) yet? It's a front-end
to git specifically targeted for versioning configuration. From a quick
glance at it's web I don't see whether it already stores the metadata you
describe, but in either case it could be interesting for you.

-- 
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Using git to store /etc, redux

[David Härdeman]

On Mon, May 21, 2007 20:32, Jan Hudec wrote:
> Have you looked at IsiSetup (http://www.isisetup.ch/, linked from
> http://git.or.cz/gitwiki/InterfacesFrontendsAndTools) yet? It's a
> front-end
> to git specifically targeted for versioning configuration. From a quick
> glance at it's web I don't see whether it already stores the metadata you
> describe, but in either case it could be interesting for you.

Yes, it was mentioned in the previous thread about tracking /etc in git.
I've looked at it but it seemed to not store any metadata. Also, it seemed
like overkill for just keeping a history of my /etc.

It also used a lot of hairy "grep -v" and similar tricks to run git but to
mangle the output before it's shown to the user. I'd prefer to interact
directly with git which I'm already familiar with...and git already has
everything I need for tracking /etc with the exception of the metadata.

-- 
David Härdeman