On "interpret-trailers" standalone tool

On "interpret-trailers" standalone tool

[Junio C Hamano]

So far I've mostly been ignoring how the command line would look
like, because the intermediate goal to my mind was to have it as a
hook that are added by people better versed with Git than an average
end-user, and if the command line interface had to change then they
are capable of updating it, so it is more acceptable than the usual
end-user tools to break compatibility between an early prototype and
later versions, and because the final goal would be to libify the
internal logic and integrate it into places we would invoke hooks,
making the standalone command irrelevant.

However, I started to care ;-)  For example, wouldn't it be nice if
you can do

    $ git format-patch -5 --cover-letter -o +my-series/ my-topic
    $ git interpret-trailers "some args" ./+my-series/0*.patch

to fix-up the "trailers" portion of the proposed log message in the
formatted patches?  There may be other possible uses that having a
standalone tool would be helpful, even after we removed the need for
such a tool from commit, rebase, etc. by integrating the internal
logic to the implementation of these commands.

However, I am wondering if the current "everything on the command
line is instruction to the command" is too limiting to allow the use
of the tool both as a filter and as a tool that can work on one or
more files named on the command line.  If we start from there, the
only way to later add "these arguments are names of the files to be
operated on" is to add "--file <file1> --file <file2>..." options,
which feels quite backwards as a UNIX tool.

It would be easier to explain and understand if the command line
option set is modeled after things like "cat" or "sed", where
non-option arguments are filenames, instructions are given in the
form of "--option <arg>" (e.g. "-e 's/foo/bar/'" given to sed), and
having no non-option arguments on the command line signals that the
tool is working as a filter.

Re: On "interpret-trailers" standalone tool

[Christian Couder]

From: Junio C Hamano <gitster@pobox.com>
>
> So far I've mostly been ignoring how the command line would look
> like,

I don't really feel this way ;-)

> because the intermediate goal to my mind was to have it as a
> hook that are added by people better versed with Git than an average
> end-user, and if the command line interface had to change then they
> are capable of updating it, so it is more acceptable than the usual
> end-user tools to break compatibility between an early prototype and
> later versions, and because the final goal would be to libify the
> internal logic and integrate it into places we would invoke hooks,
> making the standalone command irrelevant.
> 
> However, I started to care ;-)  For example, wouldn't it be nice if
> you can do
> 
>     $ git format-patch -5 --cover-letter -o +my-series/ my-topic
>     $ git interpret-trailers "some args" ./+my-series/0*.patch
> 
> to fix-up the "trailers" portion of the proposed log message in the
> formatted patches?  There may be other possible uses that having a
> standalone tool would be helpful, even after we removed the need for
> such a tool from commit, rebase, etc. by integrating the internal
> logic to the implementation of these commands.
> 
> However, I am wondering if the current "everything on the command
> line is instruction to the command" is too limiting to allow the use
> of the tool both as a filter and as a tool that can work on one or
> more files named on the command line.  If we start from there, the
> only way to later add "these arguments are names of the files to be
> operated on" is to add "--file <file1> --file <file2>..." options,
> which feels quite backwards as a UNIX tool.

Yeah, except that we could add for example a '-o' option that would
take a directory as argument and that would mean that the command
should operate on all the files in this directory. It would be like
the -o option of the format-patch command.

> It would be easier to explain and understand if the command line
> option set is modeled after things like "cat" or "sed", where
> non-option arguments are filenames, instructions are given in the
> form of "--option <arg>" (e.g. "-e 's/foo/bar/'" given to sed), and
> having no non-option arguments on the command line signals that the
> tool is working as a filter.

Yeah, that's an interesting idea. I am not against making yet another
number of changes to "git interpret-trailers" to make something like
the above possible. But I think there are a few things that should be
discussed first.

First, if you think that the command might often be used along with
format-patch, then it might be interesting for the user to have the
command behave like format-patch instead of like cat or sed. This
means that we could add the -o option I suggest above. And we don't
need to do it now. We could add this option later instead of having to
make the command work on many files now.

Second, if the command should accept a patch as input instead of just
a commit message, or both, this means that the command should have a
way to tell if it is passed a patch, and then locate the commit
message part in the patch. This means yet other changes to the
command. Maybe these changes could be made later, in another series,
or when the need arises to use it on full patches.

Third, if trailers arguments are passed to the command using an option
like "-z token=value" or "-z token:value", it would be nice to the
user for consistency if the same option could be used when passing the
same arguments to "git commit" and perhaps other commands like "git
rebase", "git cherry-pick" and so on. This means that we now have to
choose carefully the name of this option. Perhaps we can just give it
a long name now like --trailer and care later about a short name, but
I think it would not be very nice to the user to only have a long name
for this option as it will very often be used.

Fourth, some users might want the command to be passed some files as
input, but they might not want the command to modify these input
files. They might prefer the command to write its ouput into another
set of output files. Maybe a syntax like cat or sed is not very well
suited for this kind of use, while having a -o option for the output
directory and a -i option for the input directory (if different from
the output dir) would be nicer.

Thanks,
Christian.

Re: On "interpret-trailers" standalone tool

[Junio C Hamano]

Christian Couder <chriscool@tuxfamily.org> writes:

>> However, I am wondering if the current "everything on the command
>> line is instruction to the command" is too limiting to allow the use
>> of the tool both as a filter and as a tool that can work on one or
>> more files named on the command line.  If we start from there, the
>> only way to later add "these arguments are names of the files to be
>> operated on" is to add "--file <file1> --file <file2>..." options,
>> which feels quite backwards as a UNIX tool.
>
> Yeah, except that we could add for example a '-o' option that would
> take a directory as argument and that would mean that the command
> should operate on all the files in this directory. It would be like
> the -o option of the format-patch command.

For output for which users do not know offhand what files are to be
produced, giving a single directory with -o makes tons of sense, but
for input, naming each individual file (and with help with shell
globs *) is a lot more natural UNIX tool way, I would think.  "Take
everything from this directory" cannot be substitute for that, even
though the reverse (i.e. by naming the input files with "dir/*") is
true.  It is not a viable replacement.

> First, if you think that the command might often be used along with
> format-patch,

... I am not singling out format-patch output.  Any text file/stream
that has the commit log message may benefit from the "trailers" filter,
and format-patch output is merely one very obvious example.  As to
the detection of the end of commit log message, the current "EOF is
where the log message ends (but we would remote trailing blank line)"
can easily be updated to "EOF or the first three-dash line".

> Third, if trailers arguments are passed to the command using an
> option like "-z token=value" or "-z token:value", it would be nice
> to the user for consistency if the same option could be used when
> passing the same arguments to "git commit" and perhaps other
> commands like "git rebase", "git cherry-pick" and so on. This
> means that we now have to choose carefully the name of this
> option. Perhaps we can just give it a long name now like --trailer
> and care later about a short name,...

Absolutely.  That is a very sensible way to go.

> Fourth, some users might want the command to be passed some files as
> input, but they might not want the command to modify these input
> files. They might prefer the command to write its ouput into another
> set of output files. Maybe a syntax like cat or sed is not very well
> suited for this kind of use, while having a -o option for the output
> directory and a -i option for the input directory (if different from
> the output dir) would be nicer.

Sure.  I would expect we would require something like Perl's '-i'
(in-place rewrite) option for this sequence to really work:

	git format-patch -o there -5
	git that-command --options -i there/*

and without, I would expect the output to come to its standard
output.

Thanks.

Re: On "interpret-trailers" standalone tool

[Christian Couder]

On Mon, Apr 14, 2014 at 11:41 PM, Junio C Hamano <gitster@pobox.com> wrote:
> Christian Couder <chriscool@tuxfamily.org> writes:
>
>> Yeah, except that we could add for example a '-o' option that would
>> take a directory as argument and that would mean that the command
>> should operate on all the files in this directory. It would be like
>> the -o option of the format-patch command.
>
> For output for which users do not know offhand what files are to be
> produced, giving a single directory with -o makes tons of sense, but
> for input, naming each individual file (and with help with shell
> globs *) is a lot more natural UNIX tool way, I would think.

Yeah, but the "git interpret-trailers" command is a special, because,
if it takes files as arguments, then it is logical that its output
would be also files and not stdout. (See also at the end of this
message.)

> "Take
> everything from this directory" cannot be substitute for that, even
> though the reverse (i.e. by naming the input files with "dir/*") is
> true.  It is not a viable replacement.
>
>> First, if you think that the command might often be used along with
>> format-patch,
>
> ... I am not singling out format-patch output.  Any text file/stream
> that has the commit log message may benefit from the "trailers" filter,
> and format-patch output is merely one very obvious example.  As to
> the detection of the end of commit log message, the current "EOF is
> where the log message ends (but we would remote trailing blank line)"
> can easily be updated to "EOF or the first three-dash line".

Ok, I think that it's an interesting feature anyway, so I can add it
now instead of later.

>> Third, if trailers arguments are passed to the command using an
>> option like "-z token=value" or "-z token:value", it would be nice
>> to the user for consistency if the same option could be used when
>> passing the same arguments to "git commit" and perhaps other
>> commands like "git rebase", "git cherry-pick" and so on. This
>> means that we now have to choose carefully the name of this
>> option. Perhaps we can just give it a long name now like --trailer
>> and care later about a short name,...
>
> Absolutely.  That is a very sensible way to go.

Ok, I will use "--trailer" then. As I said in my previous message,
this unfortunately means that the command will not be very user
friendly until we integrate it with other commands like "git commit"
and find a short option name that hopefully work for all the commands.

>> Fourth, some users might want the command to be passed some files as
>> input, but they might not want the command to modify these input
>> files. They might prefer the command to write its ouput into another
>> set of output files. Maybe a syntax like cat or sed is not very well
>> suited for this kind of use, while having a -o option for the output
>> directory and a -i option for the input directory (if different from
>> the output dir) would be nicer.
>
> Sure.  I would expect we would require something like Perl's '-i'
> (in-place rewrite) option for this sequence to really work:
>
>         git format-patch -o there -5
>         git that-command --options -i there/*
>
> and without, I would expect the output to come to its standard
> output.

If the input comes from stdin, then I agree that the command should be
a filter, so its output should be on stdout. But if the input comes
from files given as arguments, then I would say that the command
should behave as an editor and by default it should edit its input
file inplace. Its input and output files should be different only if
it is given one or more special option,

Otherwise the example you gave:

    $ git format-patch -5 --cover-letter -o +my-series/ my-topic
    $ git interpret-trailers "some args" ./+my-series/0*.patch

would result in having on stdout all the patches edited by "git
interpret-trailers".
How would people could then easily send these edited patches?

Thanks,
Christian.

Re: On "interpret-trailers" standalone tool

[Junio C Hamano]

Christian Couder <christian.couder@gmail.com> writes:

> If the input comes from stdin, then I agree that the command should be
> a filter, so its output should be on stdout. But if the input comes
> from files given as arguments, then I would say that the command
> should behave as an editor and by default it should edit its input
> file inplace.

I do not see where that "should" comes from.  I am looking at this
more from the angle of obtaining a useful building block, while you
seem to be thinking of this as a specialized tool for a narrow set
of specifkc tasks.

Thinking of examples in "sort" and "sed", I would say "read multiple
files, futz with the contents and spit out a single output stream"
is the way people expect a command to operate without being told to
operate in some other way.  Overwriting existing files should never
be the default for safety---otherwise you would require people who
want safety to do something like:

    cp realfile tmp
    futz tmp
    verify tmp
    mv tmp realfile

On the other hand, a usual "sort/sed/cat"-like command, even without
the "in-place edit" option, can be used as in-place with

    mv realfile tmp
    futz tmp >realfile

easily, and is more flexible as a building block.  Of course, that
does not rule out an option to work in-place (e.g. in a similar way
to "sort -o file file", or "perl -i -e 'y/a-z/A-Z/' frotz nitfol").

> Its input and output files should be different only if
> it is given one or more special option,
>
> Otherwise the example you gave:
>
>     $ git format-patch -5 --cover-letter -o +my-series/ my-topic
>     $ git interpret-trailers "some args" ./+my-series/0*.patch
>
> would result in having on stdout all the patches edited by "git
> interpret-trailers".

Didn't I mention that I do not mind "-i" already if in-place edit is
desired?  Add "-i" to the command line arguments among "some args",
and your complaints will disappear, no?

Re: On "interpret-trailers" standalone tool

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ...  I am looking at this
> more from the angle of obtaining a useful building block, while you
> seem to be thinking of this as a specialized tool for a narrow set
> of specifkc tasks.

By the way, I am speaking with a bitter experience of designing the
original "format-patch" command line parameters, thinking that "This
is a specialized tool to let me send what Linus hasn't picked up
yet, so the command should take where Linus is and where I am".

Not using the A..B syntax turned out to be a poor choice but that
realization came long after the ship sailed---back then, A..B syntax
was relatively new and it was unclear that it would become the
universal syntax we use throughout the system to denote a range of
commits in the DAG.

The design mistake for starting at a specialized tool for a narrow
set of specific tasks took considerable effort to retrofit the
general syntax while not breaking the traditional usage.  I am being
cautious here because I do not see us making the same mistake.

Re: On "interpret-trailers" standalone tool

[Junio C Hamano]

Junio C Hamano <gitster@pobox.com> writes:

> ...  I am being
> cautious here because I do not see us making the same mistake.

s/do not/& want to/

Sorry for the noise.