Use case: GIT to manage transactions in a CMS?

Use case: GIT to manage transactions in a CMS?

["J. David Ibáñez"]

Hello,

I am working on a project (a content management system) where the data
is stored as files and folders.

Currently, for persistance and transactions we use the ZODB [1] object
database. But our goal is to move away from the ZODB and use directly
the file system, as it will allow us to use all the good unix tools.

We are using git to manage the source code. And now we are exploring git
to see if it can do the job of transactions, so that each transaction in
the system will be a git commit.

One problem we have found is that we can not commit empty directories (what
we need to do). Any idea how to solve or work-around this constraint?

Any suggestions and input on this exotic use case for git will be very
welcomed.



[1] http://www.zope.org/Wikis/ZODB/FrontPage

-- 
J. David Ibáñez
Itaapy <http://www.itaapy.com>         Tel +33 (0)1 42 23 67 45
9 rue Darwin, 75018 Paris              Fax +33 (0)1 53 28 27 88 

Re: Use case: GIT to manage transactions in a CMS?

[Andreas Ericsson]

J. David Ibáñez wrote:
> Hello,
> 
> I am working on a project (a content management system) where the data
> is stored as files and folders.
> 
> Currently, for persistance and transactions we use the ZODB [1] object
> database. But our goal is to move away from the ZODB and use directly
> the file system, as it will allow us to use all the good unix tools.
> 
> We are using git to manage the source code. And now we are exploring git
> to see if it can do the job of transactions, so that each transaction in
> the system will be a git commit.
> 
> One problem we have found is that we can not commit empty directories (what
> we need to do). Any idea how to solve or work-around this constraint?
> 

$ touch empty/dir/.placeholder


> Any suggestions and input on this exotic use case for git will be very
> welcomed.
> 

Sounds cool. I'll have to give it a whirl when you've got something to show.

-- 
Andreas Ericsson                   andreas.ericsson@op5.se
OP5 AB                             www.op5.se
Tel: +46 8-230225                  Fax: +46 8-230231

Re: Use case: GIT to manage transactions in a CMS?

["J. David Ibáñez"]

Andreas Ericsson wrote:

> J. David Ibáñez wrote:
>
>> Hello,
>>
>> I am working on a project (a content management system) where the data
>> is stored as files and folders.
>>
>> Currently, for persistance and transactions we use the ZODB [1] object
>> database. But our goal is to move away from the ZODB and use directly
>> the file system, as it will allow us to use all the good unix tools.
>>
>> We are using git to manage the source code. And now we are exploring git
>> to see if it can do the job of transactions, so that each transaction in
>> the system will be a git commit.
>>
>> One problem we have found is that we can not commit empty directories
>> (what
>> we need to do). Any idea how to solve or work-around this constraint?
>>
>
> $ touch empty/dir/.placeholder
>
>

I would like to have something better, no so much of a work-around.

Maybe I could hack something, are there some docs explaining the
internal format of git objects? or, where to look in the source to
find this info?


>> Any suggestions and input on this exotic use case for git will be very
>> welcomed.
>>

Another point, the application is written in Python, right now I
have to open a shell to run the git commands.

It would be nice if the core functionality was implemented in a C
library, then with a Python wrapper I could use git without going
through the shell.

Is this something to be expected? There will be a "libgit" sometime
in the future?


> Sounds cool. I'll have to give it a whirl when you've got something to
> show.
>

Probably I will have to hack something else to have ACID transactions,
in the short-term. But "git" is at least interesting, since it could
open a world of possibilities. If I ever have something to show, I will
tell you.


-- 
J. David Ibáñez
Itaapy <http://www.itaapy.com>         Tel +33 (0)1 42 23 67 45
9 rue Darwin, 75018 Paris              Fax +33 (0)1 53 28 27 88 

Re: Use case: GIT to manage transactions in a CMS?

[Junio C Hamano]

"J. David Ibáñez" <jdavid@itaapy.com> writes:

> Maybe I could hack something, are there some docs explaining the
> internal format of git objects? or, where to look in the source to
> find this info?

The index file is mostly dealt with by read-cache.c (despite its
name, it has both read and write routines).  The easiest way to
understand the tree object format is by reading write-tree.c.

But the format is the easiest part.  You have to realize that
this is a nontrivial task.

The merge algorithm assumes that it can detect a presense of
directory by finding a blob under that.  You need to teach
'read-tree -m' that sometimes there is an empty directory
recorded in the index file.  This is very important to avoid
directory-file conflicts during the merge.

Tree object side is probably easier because we _can_ write an
empty tree object, and presumably three tree-walker
implementations (one used in read-tree and object layer, one in
diff-tree and another in merge-tree) should all handle empty
trees gracefully, but still its callers need to be verified.

> Another point, the application is written in Python, right now
> I have to open a shell to run the git commands.

I always have a shell open so it does not sound a big deal to me
;-).

> It would be nice if the core functionality was implemented in a C
> library, then with a Python wrapper I could use git without going
> through the shell.
>
> Is this something to be expected? There will be a "libgit" sometime
> in the future?

Yes, read the list archive.

But not yet.  The core level has been fluctuating quite a bit.  The
latest round being the object layer bookkeeping changes.