handling symlinks proposal

handling symlinks proposal

[Christian Thaeter]

Git currently keep symlinks always as symlink, I would like to see some
optional functionality when handling symlinks.

Sometimes it is not intended to store information external to a source
tree in git. Problems are that it might break a checkout, just loosing
information or even have security implications.

What are are the diffrent kinds of symlinks:
 - they can point to a file local to the tree
 - or point to a file outside of the tree

 - they can be absolute
 - or relative

how can we handle symlinks:
 - 'keep' what git currently does, store the symlink as it is
 - 'file' store it as file (dereference it), information that it was a
    symlink gets lost, checkouts will produce a file.
 - 'relative' store it as a relative symlink
 - 'absolute' store it as a absolute symlink
 - 'error' refuse to checkin and abort commit
 - 'warn' display a warning

How to add this to git?
make config options (defaults in [core] ?)
 symlink[.{local|extern}?_?{absolute|relative}?] = action

per branch overrides in [branch]

Example:
[core]
  # in-tree symlinks shall not break a checkout to another place
  symlink.local = relative
  # we don't want out-of-tree symlinks, store them as file
  symlink.extern_absolute = file
  # a relative symlink out-of-tree is likely a typo/error?
  symlink.extern_relative = error

[branch]
  # some local feature branch keeps all symlinks
  myfeature.symlink = keep

Default will be 'keep' so without this options it is what git currently
does.

Additional thoughts:
An existing symlink in the tree is not altered by check in, only the
repository is affected.
New checkouts or switching branches may alter it.
Merging between branches where one stores something as file and the
other symlink should be straightforward, does this need to be resolved?

So far thats just an idea how I would like it. If People like it this
way or we can refine it even better, I am going to implement it sometime
next.

Re: handling symlinks proposal

[René Scharfe]

Christian Thaeter schrieb:
> Git currently keep symlinks always as symlink, I would like to see some
> optional functionality when handling symlinks.
> 
> Sometimes it is not intended to store information external to a source
> tree in git. Problems are that it might break a checkout, just loosing
> information or even have security implications.
[...]
> how can we handle symlinks:
>  - 'keep' what git currently does, store the symlink as it is
>  - 'file' store it as file (dereference it), information that it was a
>     symlink gets lost, checkouts will produce a file.
>  - 'relative' store it as a relative symlink
>  - 'absolute' store it as a absolute symlink
>  - 'error' refuse to checkin and abort commit
>  - 'warn' display a warning
> 
> How to add this to git?

Why would we want to?  Could you perhaps give examples on how a symlink
in a git repo could break checkouts, cause loss of information or have
security implications?

You're symlink handling methods 'file', 'relative', 'absolute', 'error',
'warn' aren't very useful if you check in files (and symlinks) manually,
because if you add the links by hand then you can make sure they point
to the right place (or are no symlinks, but plain files), after all.

So they're mainly intended for initial checkins and merges with external
repos, right?  But in these cases you have to trust foreign code anyway
(simple example: you need to be sure 'make all' doesn't run a command
like 'rm -rf $HOME'), so you have to take a closer look at that stuff
anyway -- symlinks are just one part of what needs checking.

Thanks,

Re: handling symlinks proposal

[Linus Torvalds]

On Sun, 26 Nov 2006, Christian Thaeter wrote:
>
> Git currently keep symlinks always as symlink, I would like to see some
> optional functionality when handling symlinks.

I can pretty much guarantee that you really don't want to do this, and 
that you'd be _much_ happier using some explicit wrappers around git to 
handle your special needs, than try to teach the SCM to handle symlinks 
specially.

The issues become all the same as with some of the idiotic things that 
other SCM's do - like $Id keyword substitution - but _much_ worse.

A simple example: what is "git diff HEAD" supposed to say, once you've 
committed the symlink as something else than a symlink? It's obviously 
still a symlink in your source tree.. Or what about "git reset --hard", or 
"git checkout other-branch-that-also-had-a-symlink"?

In other words, trivial and simple operations that have clear and 
well-defined semantics suddenly become totally insane, with no semantics 
that make sense what-so-ever. 

By not "keeping" symlinks as symlinks, what you actually did was to 
totally destroy the ability to handle symlinks AT ALL. The simplest core 
operations suddenly cannot work, because you've broken the 1:1 link 
between "original source tree" and "tree that actually got committed".

> how can we handle symlinks:
>  - 'keep' what git currently does, store the symlink as it is

Right. This is the only sane thing, if you want to support symlinks at 
all.

>  - 'file' store it as file (dereference it), information that it was a
>     symlink gets lost, checkouts will produce a file.

This is what some other systems do, by just not knowing anything about 
symlinks at all. It's certainly self-consistent - you can just drop all 
"lstat()" calls, and replace them with regular "stat()" calls. 

It causes:

 - (obviously) inability to handle symlinks correctly

 - security problems and serious confusion when you _do_ have a symlink
   (what happens when you check out another branch, and the symlink points 
   to outside the tree? A system that is truly unaware of symlinks will 
   generally just overwrite the file OUTSIDE the tree, which is clearly 
   bogus, but equally bogus is to just do an "unlink + create" which will 
   break the symlink.

so it's generally MUCH worse than any alternative.

>  - 'relative' store it as a relative symlink
>  - 'absolute' store it as a absolute symlink

Neither of these make any real sense. You can really only store it in the 
form it is now, because you simply _cannot_ make it a "relative" or 
"absolute" symlink (the transformation depends on how things are mounted).

Not to mention that you get all the problems with "what does 'git diff' 
mean?" that I already alluded to above..

>  - 'error' refuse to checkin and abort commit

Sure, we can just say "don't allow symlinks". This is probably what we'd 
have to do for a native Windows client, for example, or for anything else 
that simply doesn't support the concept (VMS port of git, anyone?).

>  - 'warn' display a warning

Again, there's nothing wrong with this, but what's the point, really? 
Especially a big enough point to merit a per-branch config option?

Re: handling symlinks proposal

[Christian Thaeter]

Linus Torvalds wrote:
> 
> On Sun, 26 Nov 2006, Christian Thaeter wrote:
>> Git currently keep symlinks always as symlink, I would like to see some
>> optional functionality when handling symlinks.
> 
> I can pretty much guarantee that you really don't want to do this, and 
> that you'd be _much_ happier using some explicit wrappers around git to 
> handle your special needs, than try to teach the SCM to handle symlinks 
> specially.

ok agreed, i'll write some script which one can use on a tree to
sanitize symlinks on given rules and/or be used as a pre-commit check.
That has the pretty same effect for my needs and is more sane like you
outlined.