[BUG] Autocompletion fails with "bash: words: bad array subscript"

[BUG] Autocompletion fails with "bash: words: bad array subscript"

[Sverre Rabbelier]

Heya,

[cced: git completion people]

This happens if I try use ctrl-shift-r (reverse-i-search) for the
string `git commit -am "S`, resulting in the following:

(reverse-i-search)`git commit -am "S': git commit -am "Set new Melange
version number to 2-0-20110501 in app.yaml.template."

If I then hit tab, I get:

$ bash: words: bad array subscriptversion number to 2-0-20110501 in
app.yaml.template."

Hitting tab again gives:

bash: words: bad array subscript

Display all 3032 possibilities? (y or n)

I have no clue how to debug this, other than that it doesn't happen if
I don't source ~/code/git/contrib/completion/git-completion.bash.

Anyone have any idea's?

-- 
Cheers,

Sverre Rabbelier

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Jeff King]

On Tue, May 10, 2011 at 10:13:11PM +0200, Sverre Rabbelier wrote:

> This happens if I try use ctrl-shift-r (reverse-i-search) for the
> string `git commit -am "S`, resulting in the following:
> 
> (reverse-i-search)`git commit -am "S': git commit -am "Set new Melange
> version number to 2-0-20110501 in app.yaml.template."
> 
> If I then hit tab, I get:
> 
> $ bash: words: bad array subscriptversion number to 2-0-20110501 in
> app.yaml.template."

I can reproduce it here pretty easily.

> Hitting tab again gives:
> 
> bash: words: bad array subscript
> 
> Display all 3032 possibilities? (y or n)
> 
> I have no clue how to debug this, other than that it doesn't happen if
> I don't source ~/code/git/contrib/completion/git-completion.bash.
> 
> Anyone have any idea's?

You can try "set -x" which will show you what was executing. Of course
that will spew pages of output. So you'll want to use something like
"script", or if you're man enough, "exec 2>stderr" and then do the whole
thing blind. ;)

It looks like we set $cword too low at some point, as the problematic
code seems to be:

  + upargs+=(-v $vprev "${words[cword - 1]}")
  bash: words: bad array subscript

but I haven't figured out yet where that happens.

-Peff

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Jeff King]

On Tue, May 10, 2011 at 04:31:01PM -0400, Jeff King wrote:

> It looks like we set $cword too low at some point, as the problematic
> code seems to be:
> 
>   + upargs+=(-v $vprev "${words[cword - 1]}")
>   bash: words: bad array subscript
> 
> but I haven't figured out yet where that happens.

Hrm. That code doesn't appear in our completion at all. We provide our
own _get_comp_words_by_ref, but if it is already defined, we use
whatever is there. So on my box, the problematic code comes from
/etc/bash_completion. And I think it is a bug there, as this is one of
the first things called (so git's completion hasn't had a change to
introduce any bugs yet :) ).

-Peff

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Sverre Rabbelier]

Heya,

[also +Jonathan, who I seem to have forgotten to cc]

On Tue, May 10, 2011 at 22:39, Jeff King <peff@peff.net> wrote:
> Hrm. That code doesn't appear in our completion at all. We provide our
> own _get_comp_words_by_ref, but if it is already defined, we use
> whatever is there. So on my box, the problematic code comes from
> /etc/bash_completion. And I think it is a bug there, as this is one of
> the first things called (so git's completion hasn't had a change to
> introduce any bugs yet :) ).

Most curious, since it doesn't happen when I don't source git
completion. Perhaps it'll happen for any completion, or maybe we're
using the completion wrong somehow? I'm on Debian wheezy/sid.

-- 
Cheers,

Sverre Rabbelier

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Jeff King]

On Tue, May 10, 2011 at 10:47:49PM +0200, Sverre Rabbelier wrote:

> On Tue, May 10, 2011 at 22:39, Jeff King <peff@peff.net> wrote:
> > Hrm. That code doesn't appear in our completion at all. We provide our
> > own _get_comp_words_by_ref, but if it is already defined, we use
> > whatever is there. So on my box, the problematic code comes from
> > /etc/bash_completion. And I think it is a bug there, as this is one of
> > the first things called (so git's completion hasn't had a change to
> > introduce any bugs yet :) ).
> 
> Most curious, since it doesn't happen when I don't source git
> completion. Perhaps it'll happen for any completion, or maybe we're
> using the completion wrong somehow? I'm on Debian wheezy/sid.

Right. Bash calls into our __git() completion function, which calls the
implementation of _get_comp_words_by_ref from /etc/bash_completion,
which has the bug. If you don't source git completion, then you are just
getting bash's default file completion.

-Peff

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Sverre Rabbelier]

Heya,

On Tue, May 10, 2011 at 23:01, Jeff King <peff@peff.net> wrote:
> Right. Bash calls into our __git() completion function, which calls the
> implementation of _get_comp_words_by_ref from /etc/bash_completion,
> which has the bug. If you don't source git completion, then you are just
> getting bash's default file completion.

So should we file this bug with bash's completion people? Also, how
did you manage to reproduce? It doesn't happen all the time for me.

-- 
Cheers,

Sverre Rabbelier

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Jeff King]

On Tue, May 10, 2011 at 11:02:53PM +0200, Sverre Rabbelier wrote:

> On Tue, May 10, 2011 at 23:01, Jeff King <peff@peff.net> wrote:
> > Right. Bash calls into our __git() completion function, which calls the
> > implementation of _get_comp_words_by_ref from /etc/bash_completion,
> > which has the bug. If you don't source git completion, then you are just
> > getting bash's default file completion.
> 
> So should we file this bug with bash's completion people?

Probably, but it would be nice to reduce it to a smaller test case (or
one that happens just with completions shipped by Debian) just to rule
out anything git is doing.

> Also, how did you manage to reproduce?  It doesn't happen all the time
> for me.

Initially, with:

  $ git init repo && cd repo
  $ echo content >file && git add file
  $ git commit -am foo
  $ <C-R>git commit -am f<Tab>

but I also get it with just:

  $ cd /anywhere ;# do not have to even be in a git repo
  $ <C-R>git<Tab>

-Peff

[PATCH] completion: fix array indexing error after reverse history search

[SZEDER Gábor]

In v1.7.4-rc0~11^2~2 (bash: get --pretty=m<tab> completion to work
with bash v4, 2010-12-02) we started to use _get_comp_words_by_ref()
to access completion-related variables.  This function is usually
provided by the bash-completion package, but if it's not present when
git-completion.bash is loaded, then we use our simplified
re-implementation of that function.  We also provide a bare bones
_get_comp_words_by_ref() implementation that works with zsh.

Unfortunately (and interestingly!), since the recent commit da4902a
(completion: remove unnecessary _get_comp_words_by_ref() invocations,
2011-04-28) the same bug can be triggered in all of these three
implementations when doing reverse history search:

 - Hit ctrl-r to start reverse history search.
 - Search for a git command, e.g. by typing 'git '.
   The prompt will look like this: (reverse-i-search)`git ': git diff
 - Press TAB to attempt completion.

This will lead to an error like:

  $ bash: words: bad array subscript

The reason for this is that since commit da4902a we always tell
_get_comp_words_by_ref() to set the $prev variable in _git(), which
should contain the word preceeding the word containing the current
cursor position.  The value of this variable is assigned in all three
_get_comp_words_by_ref() implementations with something like

  prev=${COMP_WORDS[COMP_CWORD-1]}

However, in the above bug-triggering input the cursor is considered to
be at the very beginning of the command line, i.e. in the nullth word,
so the array index in the above line ends up to be -1, hence the
error.  In this case the only sensible value for $prev would be an
empty string, because there is no previous word on the command line.

The same applies to the _get_comp_words_by_ref() invocation in
_gitk(), too.

This patch fixes both of our _get_comp_words_by_ref() implementations
by assigning an empty string to $prev when there can't be any previous
word.  It also works around the bug in _get_comp_words_by_ref() from
the bash-completion package by not asking for the $prev variable at
all when there can't be any previous word.  In the latter case we
check the value of $COMP_CWORD; we usually refrain from using this
variable directly in completion functions because of the word
splitting changes introduced in bash v4 (see v1.7.4-rc0~11^2~2), but
in this case, i.e. at the nullth word the different word splitting
rules don't make a difference.

Noticed-by: Sverre Rabbelier <srabbelier@gmail.com>
Signed-off-by: SZEDER Gábor <szeder@ira.uka.de>
---

On Tue, May 10, 2011 at 10:13:11PM +0200, Sverre Rabbelier wrote:
> This happens if I try use ctrl-shift-r (reverse-i-search) for the
> string `git commit -am "S`, resulting in the following:
> 
> (reverse-i-search)`git commit -am "S': git commit -am "Set new Melange
> version number to 2-0-20110501 in app.yaml.template."
> 
> If I then hit tab, I get:
> 
> $ bash: words: bad array subscriptversion number to 2-0-20110501 in
> app.yaml.template."

Nice catch, although I have no idea why anyone would attempt
completion at that point.


Best,
Gábor

 contrib/completion/git-completion.bash |   26 ++++++++++++++++++++++----
 1 files changed, 22 insertions(+), 4 deletions(-)

diff --git a/contrib/completion/git-completion.bash b/contrib/completion/git-completion.bash
index 00691fc..0b12c66 100755
--- a/contrib/completion/git-completion.bash
+++ b/contrib/completion/git-completion.bash
@@ -445,7 +445,11 @@ _get_comp_words_by_ref ()
 			cur=$cur_
 			;;
 		prev)
-			prev=${words_[$cword_-1]}
+			if [ "$cword_" = 0 ]; then
+				prev=""
+			else
+				prev=${words_[$cword_-1]}
+			fi
 			;;
 		words)
 			words=("${words_[@]}")
@@ -466,7 +470,11 @@ _get_comp_words_by_ref ()
 			cur=${COMP_WORDS[COMP_CWORD]}
 			;;
 		prev)
-			prev=${COMP_WORDS[COMP_CWORD-1]}
+			if [ "$COMP_CWORD" = 0 ]; then
+				prev=""
+			else
+				prev=${COMP_WORDS[COMP_CWORD-1]}
+			fi
 			;;
 		words)
 			words=("${COMP_WORDS[@]}")
@@ -2611,7 +2619,12 @@ _git ()
 	fi
 
 	local cur words cword prev
-	_get_comp_words_by_ref -n =: cur words cword prev
+	if [ "$COMP_CWORD" = 0 ]; then
+		_get_comp_words_by_ref -n =: cur words cword
+		prev=""
+	else
+		_get_comp_words_by_ref -n =: cur words cword prev
+	fi
 	while [ $c -lt $cword ]; do
 		i="${words[c]}"
 		case "$i" in
@@ -2662,7 +2675,12 @@ _gitk ()
 	fi
 
 	local cur words cword prev
-	_get_comp_words_by_ref -n =: cur words cword prev
+	if [ "$COMP_CWORD" = "0" ]; then
+		_get_comp_words_by_ref -n =: cur words cword
+		prev=""
+	else
+		_get_comp_words_by_ref -n =: cur words cword prev
+	fi
 
 	__git_has_doubledash && return
 
-- 
1.7.5.1.248.g2ba0c6

Re: [PATCH] completion: fix array indexing error after reverse history search

[Sverre Rabbelier]

Heya,

2011/5/11 SZEDER Gábor <szeder@ira.uka.de>:
> Nice catch, although I have no idea why anyone would attempt
> completion at that point.

Wow, amazing turnaround, will test tomorrow :). I hit tab instead of
enter because I don't want to accidentally run the command.

-- 
Cheers,

Sverre Rabbelier

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[SZEDER Gábor]

Hi,


On Tue, May 10, 2011 at 05:10:16PM -0400, Jeff King wrote:
> On Tue, May 10, 2011 at 11:02:53PM +0200, Sverre Rabbelier wrote:
> 
> > On Tue, May 10, 2011 at 23:01, Jeff King <peff@peff.net> wrote:
> > > Right. Bash calls into our __git() completion function, which calls the
> > > implementation of _get_comp_words_by_ref from /etc/bash_completion,
> > > which has the bug. If you don't source git completion, then you are just
> > > getting bash's default file completion.
> > 
> > So should we file this bug with bash's completion people?
> 
> Probably, but it would be nice to reduce it to a smaller test case (or
> one that happens just with completions shipped by Debian) just to rule
> out anything git is doing.

Thinking a bit more about it, you don't even need to search history to
reproduce.  Try this:

  _foo ()
  {
          local prev
          _get_comp_words_by_ref prev
  }
  complete -F _foo foo

Then type "foo", go back to the very beginning of the command line,
and then press TAB, and the same "bash: words: bad array subscript"
error appears.

So the bug is definitely not git-related.  The fix would be a check
along the lines of the first two hunks of the patch I just sent out.
Oddly enough, the bash-completion folks had a similar check in the now
deprecated _get_pword() function, that didn't made it into
_get_comp_words_by_ref()...


Best,
Gábor

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[Jonathan Nieder]

SZEDER Gábor wrote:

> Thinking a bit more about it, you don't even need to search history to
> reproduce.

Yep.  Doing

 $ . /etc/bash_completion
 $ ls<^A><TAB>
 bash: words: bad array subscript

produces the same.

> So the bug is definitely not git-related.  The fix would be a check
> along the lines of the first two hunks of the patch I just sent out.

FWIW it looks like the bash-completion lib adopted a different fix
recently: see [1] (_init_completion: Indicate that completion should
not continue if cword == 0, 2011-05-02).

[1] http://git.debian.org/?p=bash-completion/bash-completion.git;a=commitdiff;h=457dbf6061eea5f2d1e3bccacf1691265f7321cc

Re: [BUG] Autocompletion fails with "bash: words: bad array subscript"

[SZEDER Gábor]

Hi,


On Tue, May 10, 2011 at 06:45:52PM -0500, Jonathan Nieder wrote:
> SZEDER Gábor wrote:
> > So the bug is definitely not git-related.  The fix would be a check
> > along the lines of the first two hunks of the patch I just sent out.
> 
> FWIW it looks like the bash-completion lib adopted a different fix
> recently: see [1] (_init_completion: Indicate that completion should
> not continue if cword == 0, 2011-05-02).
> 
> [1] http://git.debian.org/?p=bash-completion/bash-completion.git;a=commitdiff;h=457dbf6061eea5f2d1e3bccacf1691265f7321cc

That doesn't fix the issue at hand.  _init_completion() invokes
_get_comp_words_by_ref() before that check to set all variables [1],
which will in turn try to access the -1th element of the array when
setting $prev, producing the same error.


Best,
Gábor


[1]: http://git.debian.org/?p=bash-completion/bash-completion.git;a=blob;f=bash_completion;h=e88e2fc8cd97f2a43173b45a1448451bb53e55ab;hb=457dbf6061eea5f2d1e3bccacf1691265f7321cc#l708