why still no empty directory support in git

why still no empty directory support in git

[Ping Yin]

Yes, i know this topic has been discussed for many times. However, i
am still not clear about the real reason.

So which is the reason?

1. it's by design, intentional
2. unclear logic, for example, whether to remove the directory after
the last file in it is deleted
3. hard work, no one has picked it yet
4. hardly done in current model

Ping Yin

Re: why still no empty directory support in git

[Asheesh Laroia]

On Tue, 30 Dec 2008, Ping Yin wrote:

> 2. unclear logic, for example, whether to remove the directory after the 
> last file in it is deleted

This is the thing I dislike most about git: that it sometimes calls 
rmdir() for me.  At least, one should be able to turn it off in a 
per-repository basis.  I'm going to see how hard a patch that would be to 
write.

> 3. hard work, no one has picked it yet

This is what I recall.

-- Asheesh.

-- 
Writing is turning one's worst moments into money.
 		-- J.P. Donleavy

Re: why still no empty directory support in git

[Jeff Whiteside]

funny, i thought it was 1, by design.

but i forget why a tree object couldn't point to an empty blob.

Re: why still no empty directory support in git

[Robin Rosenberg]

tisdag 30 december 2008 06:28:58 skrev Jeff Whiteside:
> funny, i thought it was 1, by design.
Sure, but designs can be changed.

> 
> but i forget why a tree object couldn't point to an empty blob.

It can, but that's not the same.

You can have an empty tree, but the index doesn't store them, 
so they would be lost on checkout/commit. Linus sketched a solution, 
but nobody took the bait. Seems doable if anyone really wants it, but
I'm certain it adds a lot of special cases.

Look for a discussion [RFC PATCH] Re: Empty directories... posted on 2007-07-19.
It's in the middle of a long thread.

-- robin

Re: why still no empty directory support in git

[Junio C Hamano]

Robin Rosenberg <robin.rosenberg.lists@dewire.com> writes:

> You can have an empty tree, but the index doesn't store them, so they
> would be lost on checkout/commit. Linus sketched a solution, but nobody
> took the bait. Seems doable if anyone really wants it, but I'm certain
> it adds a lot of special cases.

I think the original poster covered that "a lot of special cases" as
"unclear semantics", and there are more.  Do you want to have the presense
of empty directory "sticky"?  Perhaps it later becomes non-empty at some
point; will the "will always present" attribute kept then?  What happens
when such a directory becomes empty later?  What should happen when a
branch that has such a directory with "sticky existence" and another
branch with the same directory but without the stickiness are merged?

But I think one bigger reason missing from the list is that many people
loudly talked about "wants", but nobody made convincing argument on
"needs" of such a feature.

Re: why still no empty directory support in git

[Ping Yin]

On Tue, Dec 30, 2008 at 2:58 PM, Robin Rosenberg
<robin.rosenberg.lists@dewire.com> wrote:

> You can have an empty tree, but the index doesn't store them,
> so they would be lost on checkout/commit. Linus sketched a solution,
> but nobody took the bait. Seems doable if anyone really wants it, but
> I'm certain it adds a lot of special cases.
>
> Look for a discussion [RFC PATCH] Re: Empty directories... posted on 2007-07-19.
> It's in the middle of a long thread.
>
Thanks for pointing me to that thread. For other's convenience, the
begin of the thread is
http://article.gmane.org/gmane.comp.version-control.git/52813

Re: why still no empty directory support in git

[Asheesh Laroia]

On Tue, 30 Dec 2008, Asheesh Laroia wrote:

> On Tue, 30 Dec 2008, Ping Yin wrote:
>
>> 2. unclear logic, for example, whether to remove the directory after the 
>> last file in it is deleted
>
> This is the thing I dislike most about git: that it sometimes calls rmdir() 
> for me.  At least, one should be able to turn it off in a per-repository 
> basis.  I'm going to see how hard a patch that would be to write.

Well, changing this behavior seems to be "as easy as" changing 
unlink_entry in unpack_trees.c to not always rmdir(). The most naive thing 
I can think of is to have unlink_entry in unpack_trees check against the 
git config. It's probably more sensible for unpack_trees to be passed an 
argument that determines if it rmdir()s; that argument could be set via 
argv at "git unpack-tree" time, which could be set out of a configuration 
value read at "git" time.

Would a change of the "more sensible" kind possibly be accepted by the git 
maintainer?

I ask about this because I'm using git to track email in Maildir 
repositories, and in that vein I'm getting bitten by git's removal of 
empty directories.

-- Asheesh.

-- 
You will gain money by an illegal action.

Re: why still no empty directory support in git

[demerphq]

2008/12/30 Asheesh Laroia <asheesh@asheesh.org>:
> On Tue, 30 Dec 2008, Asheesh Laroia wrote:
>
>> On Tue, 30 Dec 2008, Ping Yin wrote:
>>
>>> 2. unclear logic, for example, whether to remove the directory after the
>>> last file in it is deleted
>>
>> This is the thing I dislike most about git: that it sometimes calls
>> rmdir() for me.  At least, one should be able to turn it off in a
>> per-repository basis.  I'm going to see how hard a patch that would be to
>> write.
>
> Well, changing this behavior seems to be "as easy as" changing unlink_entry
> in unpack_trees.c to not always rmdir(). The most naive thing I can think of
> is to have unlink_entry in unpack_trees check against the git config. It's
> probably more sensible for unpack_trees to be passed an argument that
> determines if it rmdir()s; that argument could be set via argv at "git
> unpack-tree" time, which could be set out of a configuration value read at
> "git" time.
>
> Would a change of the "more sensible" kind possibly be accepted by the git
> maintainer?
>
> I ask about this because I'm using git to track email in Maildir
> repositories, and in that vein I'm getting bitten by git's removal of empty
> directories.

Add a .exists to each directory.  There is precedent for such an
approach in other systems.

cheers,
Yves


-- 
perl -Mre=debug -e "/just|another|perl|hacker/"

Re: why still no empty directory support in git

[Asheesh Laroia]

On Tue, 30 Dec 2008, demerphq wrote:

> 2008/12/30 Asheesh Laroia <asheesh@asheesh.org>:

>> I ask about this because I'm using git to track email in Maildir 
>> repositories, and in that vein I'm getting bitten by git's removal of 
>> empty directories.
>
> Add a .exists to each directory.  There is precedent for such an 
> approach in other systems.

Delivering mail into a Maildir is a three-step process.  Let's say 
we are delivering to a Maildir spool stored in ~/Maildir.

(1)

The message is written out to ~/Maildir/tmp/some_filename.

(2)

When the message is complete, it is rename()d to 
~/Maildir/new/some_name.

(3)

When a mail user agent reads the Maildir spool, it checks new/ for new 
mail. If there is a message there, it renames it to 
~/Maildir/cur/some_other_filename and announces to the user, "You've got 
mail!"

So, let's say I take your suggestion.

$ touch ~/Maildir/new/.exists
$ git add ~/Maildir/new/.exists && git commit -m "La di da"

Now a spec-compliant Maildir user agent will attempt to deliver this new 
"email message" of zero bytes into the mail spool and assign it a message 
UID.  Doing so will remove it from Maildir/new.

Then I do "git pull" to get the new messages from my mail server's Maildir 
repository for my email.  This causes git read-tree to eventually be run. 
If the new tree has no unprocessed email, git runs rmdir() on 
~/Maildir/new/.

Now if I want to write a new email to ~/Maildir/ (such as due to copying 
an email from another folder), the Maildir user agent suddenly finds 
itself in a strange place: new/ does not exist, violating the definition 
of a Maildir. This breaks mail processing for that ~/Maildir/ folder.

This is because git is removing these directories. There is a strict 
incompatibility between git rmdir()ing empty directories behind my back 
and Maildir systems.

I hope that explains the issue I face, both to Junio and to Yves.

Note that for me, there is no issue with how to handle merging of empty 
directories, or what happens if these empty directories become files, or 
which empty directories to keep around; if git just never rmdir()s any 
directories for me, and otherwise acts identically to now, that would 
solve my problem. I can look into preparing an RFC patch that creates a 
mode like that.

-- Asheesh.

-- 
You will be the last person to buy a Chrysler.

Re: why still no empty directory support in git

[David Brown]

On Tue, Dec 30, 2008 at 03:58:46AM -0500, Asheesh Laroia wrote:

> This is because git is removing these directories. There is a strict 
> incompatibility between git rmdir()ing empty directories behind my back and 
> Maildir systems.

So, would there be a hook that would run at all of the times git might
remove the directories, and the hook could just put them back if
missing?

The post-merge hook is certainly one place, but there are likely
others.  You might also want one in post-checkout, but I'm guessing
that switching branches is going to be less frequent in a maildir
directory.

David

Re: why still no empty directory support in git

[Jeff King]

On Tue, Dec 30, 2008 at 03:58:46AM -0500, Asheesh Laroia wrote:

> So, let's say I take your suggestion.
>
> $ touch ~/Maildir/new/.exists
> $ git add ~/Maildir/new/.exists && git commit -m "La di da"
>
> Now a spec-compliant Maildir user agent will attempt to deliver this new  
> "email message" of zero bytes into the mail spool and assign it a message  
> UID.  Doing so will remove it from Maildir/new.

No. The maildir spec says:

  A unique name can be anything that doesn't contain a colon (or slash)
  and doesn't start with a dot.
     -- http://cr.yp.to/proto/maildir.html

where a "unique name" is the filename used for a message. In practice,
every maildir implementation I have seen ignores files starting with a
dot. Do you have one that doesn't?

-Peff

Re: why still no empty directory support in git

[Johannes Schindelin]

Hi,

On Thu, 1 Jan 2009, Jeff King wrote:

> On Tue, Dec 30, 2008 at 03:58:46AM -0500, Asheesh Laroia wrote:
> 
> > So, let's say I take your suggestion.
> >
> > $ touch ~/Maildir/new/.exists
> > $ git add ~/Maildir/new/.exists && git commit -m "La di da"
> >
> > Now a spec-compliant Maildir user agent will attempt to deliver this new  
> > "email message" of zero bytes into the mail spool and assign it a message  
> > UID.  Doing so will remove it from Maildir/new.
> 
> No. The maildir spec says:
> 
>   A unique name can be anything that doesn't contain a colon (or slash)
>   and doesn't start with a dot.
>      -- http://cr.yp.to/proto/maildir.html
> 
> where a "unique name" is the filename used for a message. In practice,
> every maildir implementation I have seen ignores files starting with a
> dot. Do you have one that doesn't?

For the record, I am using Git to manage my mails, and never had any 
problems after installing a hook which marks new empty directories with 
.gitignore.

Ciao,
Dscho

Re: why still no empty directory support in git

[Asheesh Laroia]

On Fri, 2 Jan 2009, Johannes Schindelin wrote:

> Hi,

Hi

*wipes the egg off his face*

> On Thu, 1 Jan 2009, Jeff King wrote:
>
>> On Tue, Dec 30, 2008 at 03:58:46AM -0500, Asheesh Laroia wrote:
>>
>>> So, let's say I take your suggestion.
>>>
>>> $ touch ~/Maildir/new/.exists
>>> $ git add ~/Maildir/new/.exists && git commit -m "La di da"
>>>
>>> Now a spec-compliant Maildir user agent will attempt to deliver this new
>>> "email message" of zero bytes into the mail spool and assign it a message
>>> UID.  Doing so will remove it from Maildir/new.
>>
>> No. The maildir spec says:
>>
>>   A unique name can be anything that doesn't contain a colon (or slash)
>>   and doesn't start with a dot.

Oops.  I never actually tried this...

> For the record, I am using Git to manage my mails, and never had any 
> problems after installing a hook which marks new empty directories with 
> .gitignore.

I'll give that a shot, and my apologies for the noise on the list with 
regard to this particular example.

I do still believe that git shouldn't rmdir() empty directories behind the 
user's back, but with this particular use case gone I'm no longer as 
adamant as before.

My apologies for not having tested this earlier; I will test it shortly, 
but there's every reason to think that Johannes and Jeff are right!

-- Asheesh.

-- 
It's interesting to think that many quite distinguished people have
bodies similar to yours.

Re: why still no empty directory support in git

[Asheesh Laroia]

On Thu, 1 Jan 2009, Jeff King wrote:

> On Tue, Dec 30, 2008 at 03:58:46AM -0500, Asheesh Laroia wrote:
>
>> So, let's say I take your suggestion.
>>
>> $ touch ~/Maildir/new/.exists
>> $ git add ~/Maildir/new/.exists && git commit -m "La di da"
>>
>> Now a spec-compliant Maildir user agent will attempt to deliver this new
>> "email message" of zero bytes into the mail spool and assign it a message
>> UID.  Doing so will remove it from Maildir/new.
>
> No. The maildir spec says:
>
>  A unique name can be anything that doesn't contain a colon (or slash)
>  and doesn't start with a dot.
>     -- http://cr.yp.to/proto/maildir.html
>
> where a "unique name" is the filename used for a message. In practice,
> every maildir implementation I have seen ignores files starting with a
> dot. Do you have one that doesn't?

My apologies. This works just fine, and I'm a dolt.

Happy new year!

(I'm still academically interested in how to avoid the rmdir(), but as I 
said before, that's a topic for someone else to pick up now.)

-- Asheesh.

-- 
English literature's performing flea.
 		-- Sean O'Casey on P. G. Wodehouse

Re: why still no empty directory support in git

[Liu Yubao]

Ping Yin wrote:
> Yes, i know this topic has been discussed for many times. However, i
> am still not clear about the real reason.
> 
> So which is the reason?
> 
> 1. it's by design, intentional
It's saied somewhere git is a "stupid content tracker", it cares file content
not file name, and empty directories will complicate the merge algorithm
unnecessarily.

> 2. unclear logic, for example, whether to remove the directory after
> the last file in it is deleted
> 3. hard work, no one has picked it yet
> 4. hardly done in current model
> 
> Ping Yin

Re: why still no empty directory support in git

[Anatol Pomozov]

Hi

On Mon, Dec 29, 2008 at 10:09 PM, Liu Yubao <yubao.liu@gmail.com> wrote:
>
> Ping Yin wrote:
> > Yes, i know this topic has been discussed for many times. However, i
> > am still not clear about the real reason.
> >
> > So which is the reason?
> >
> > 1. it's by design, intentional
> It's saied somewhere git is a "stupid content tracker", it cares file content
> not file name, and empty directories will complicate the merge algorithm
> unnecessarily.

Could you please explain how will it complicate merging. What is the
difference between merging 2 directories with 0 and 5 files and
merging 2 directories with 5 and 10 files?

+1 that git should respect empty directories. Git should handle file
content and not decide for user does he want to keep empty directory
in the source tree or not.

--
anatol

Re: why still no empty directory support in git

[Johannes Schindelin]

Hi,

On Tue, 30 Dec 2008, Ping Yin wrote:

> Yes, i know this topic has been discussed for many times.

We have empty directory support in Git.  It works like this: for 
directories that you do want to keep, you add an empty .gitignore file.

No problem at all,
Dscho

Re: why still no empty directory support in git

[Michael Gaber]

[-- Attachment #1: Type: text/plain, Size: 446 bytes --]

Johannes Schindelin schrieb:
> Hi,
> 
> On Tue, 30 Dec 2008, Ping Yin wrote:
> 
>> Yes, i know this topic has been discussed for many times.
> 
> We have empty directory support in Git.  It works like this: for 
> directories that you do want to keep, you add an empty .gitignore file.
> 
> No problem at all,
> Dscho

well if i understood him correctly his use-case would soon remove that
.whatever-file so it doesn't solve the problem

Michael

[-- Attachment #2: S/MIME Cryptographic Signature --]
[-- Type: application/x-pkcs7-signature, Size: 3656 bytes --]