fast-import and core.autocrlf option

fast-import and core.autocrlf option

[Dmitry Kakurin]

It looks to me that CR/LF conversion does not happen during
fast-import even if I have core.autocrlf set to 'input'.
Is this a bug or is there a reason for that?

Re: fast-import and core.autocrlf option

[Johannes Schindelin]

Hi,

On Sun, 22 Jul 2007, Dmitry Kakurin wrote:

> It looks to me that CR/LF conversion does not happen during
> fast-import even if I have core.autocrlf set to 'input'.
> Is this a bug or is there a reason for that?

fast-import works on blobs only.  So it is expected.  Besides, it would 
slow down fast-import tremendously if you wanted to introduce that 
behaviour.  Therefore I suggest just rolling your own dos2unix instead of 
trying to play cute games with fast-import.

Ciao,
Dscho

Re: fast-import and core.autocrlf option

[Shawn O. Pearce]

Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
> On Sun, 22 Jul 2007, Dmitry Kakurin wrote:
> 
> > It looks to me that CR/LF conversion does not happen during
> > fast-import even if I have core.autocrlf set to 'input'.
> > Is this a bug or is there a reason for that?
> 
> fast-import works on blobs only.  So it is expected.  Besides, it would 
> slow down fast-import tremendously if you wanted to introduce that 
> behaviour.  Therefore I suggest just rolling your own dos2unix instead of 
> trying to play cute games with fast-import.

Right, in fast-import we only process blobs as raw blobs.
Its rare that we have a file path associated with the blob data
at the time that we are actually processing the blob itself.  E.g.
applications can send us blobs up front, before they even start to
send us commits and path information.

So if we were to offer the CRLF->LF conversion feature in fast-import
it would need to be an option supplied at the time the 'data'
command issued, rather than based upon the gitattributes system
that is normally used for working tree operations.

It wouldn't be to difficult to add such an option to the data
command, but I'm not going to be able to get around to it anytime
soon.  This might be an easy change for someone else to attempt if
they were interested in some Git hacking.  Should all be isolated
to the cmd_data() routine in fast-import.c, and just needs to call
crlf_to_git() in convert.c, which is uh, static.  :-)

-- 
Shawn.

Re: fast-import and core.autocrlf option

[Junio C Hamano]

"Shawn O. Pearce" <spearce@spearce.org> writes:

> Johannes Schindelin <Johannes.Schindelin@gmx.de> wrote:
>> On Sun, 22 Jul 2007, Dmitry Kakurin wrote:
>> 
>> > It looks to me that CR/LF conversion does not happen during
>> > fast-import even if I have core.autocrlf set to 'input'.
>> > Is this a bug or is there a reason for that?
>> 
>> fast-import works on blobs only.  So it is expected.  Besides, it would 
>> slow down fast-import tremendously if you wanted to introduce that 
>> behaviour.  Therefore I suggest just rolling your own dos2unix instead of 
>> trying to play cute games with fast-import.
>
> Right, in fast-import we only process blobs as raw blobs.
> Its rare that we have a file path associated with the blob data
> at the time that we are actually processing the blob itself.  E.g.
> applications can send us blobs up front, before they even start to
> send us commits and path information.

Don't the front-ends usually have path information already when
they feed you a blob data, especially most of them operate on
per-file history?  If that is the case,...

> So if we were to offer the CRLF->LF conversion feature in fast-import
> it would need to be an option supplied at the time the 'data'
> command issued, rather than based upon the gitattributes system
> that is normally used for working tree operations.

... the "option" could be "this came from such and such path"
instead of "this is DOS data, please apply crlf".

Re: fast-import and core.autocrlf option

[Shawn O. Pearce]

Junio C Hamano <gitster@pobox.com> wrote:
> "Shawn O. Pearce" <spearce@spearce.org> writes:
> > Right, in fast-import we only process blobs as raw blobs.
> > Its rare that we have a file path associated with the blob data
> > at the time that we are actually processing the blob itself.  E.g.
> > applications can send us blobs up front, before they even start to
> > send us commits and path information.
> 
> Don't the front-ends usually have path information already when
> they feed you a blob data, especially most of them operate on
> per-file history?  If that is the case,...

Depends.  A frontend can feed huge streams of blobs and use marks
to make us remember their SHA-1s, then later feed us those marks
and connect them to paths.  Such a frontend may not have exact path
information available when it feeds the blobs to us.  Then again
maybe it does.  Depends on how the source information was organized.

> > So if we were to offer the CRLF->LF conversion feature in fast-import
> > it would need to be an option supplied at the time the 'data'
> > command issued, rather than based upon the gitattributes system
> > that is normally used for working tree operations.
> 
> ... the "option" could be "this came from such and such path"
> instead of "this is DOS data, please apply crlf".

How do we setup .gitattributes?  Should it be read from a branch
in memory, or the working directory?

If it should be read from a branch, which branch?  What if the
frontend does not want the .gitattributes file in the history
of the import?  Putting it into a branch just to cause CRLF->LF
translation during import would require filter-branch afterwards
to strip out the file.

I'd rather make things very explicit.  If the frontend wants us to
apply a filter to a data chunk before we work further on it then
the frontend should give us the .gitattributes rules to apply as
part of the data command.

-- 
Shawn.