git filters don't get applied to dotfiles

git filters don't get applied to dotfiles

[Adrien LEMAIRE]

I'd like to report a bug regarding git filters not being applied to
files beginning with a dot character "."
Using git version 2.24.1
Please let me know if there is a better way to report bugs. The github
page only mentions this email.

To give a bit of context, I've followed this arch linux suggestion to
track my dotfiles using the "bare repository and alias method".

.gitattributes
> mailrc filter=remove-smtp-pass
> .mailrc filter=remove-smtp-pass


.dotfiles/config
> [core]
> repositoryformatversion = 0
> filemode = true
> bare = true
>
> [status]
> showUntrackedFiles = no
>
> [filter "remove-smtp-pass"]
> clean = "sed -e 's/gmail.com:.*@smtp/gmail.com:PASSWORD@smtp/'"


The file I'm trying to commit is called .mailrc, but to confirm the
bug, I copied it as mailrc (without leading dot). When adding both
files, the filter only gets applied to the file without leading dot.

> $ config diff --cached
> diff --git a/.mailrc b/.mailrc
> new file mode 100644
> index 0000000..7ac2abb
> --- /dev/null
> +++ b/.mailrc
> @@ -0,0 +1,4 @@
> +account gmail {
> +  set v15-compat
> +  set mta=smtp://lemaire.adrien%40gmail.com:IamApassword@smtp.gmail.com:587 smtp-use-starttls
> +}
> diff --git a/mailrc b/mailrc
> new file mode 100644
> index 0000000..2698128
> --- /dev/null
> +++ b/mailrc
> @@ -0,0 +1,4 @@
> +account gmail {
> +  set v15-compat
> +  set mta=smtp://lemaire.adrien%40gmail.com:PASSWORD@smtp.gmail.com:587 smtp-use-starttls
> +}
> $ config check-attr -a -- mailrc
> mailrc: filter: remove-smtp-pass
> $ config check-attr -a -- .mailrc
> .mailrc: filter: remove-smtp-pass


As reported by check-attr, the .mailrc file is correctly assigned to
the filter, but the diff output shows the password.

Cordially
Adrien

Re: git filters don't get applied to dotfiles

[Dennis Kaarsemaker]

On Fri, 2019-12-27 at 16:51 +0900, Adrien LEMAIRE wrote:
> I'd like to report a bug regarding git filters not being applied to
> files beginning with a dot character "."
> Using git version 2.24.1
> Please let me know if there is a better way to report bugs. The github
> page only mentions this email.

<snip reproduction recipe>

I was not able to reproduce this in the git test suite with a quick
patch (see below). Your output does not show any git add command, is it
possible that you added the changes before configuring the filter?

If you set GIT_TRACE=2 in your environment before doing the git add of
the .mailrc file, you should see it run the filter command. It should
look something like:

+ git add test test.t test.i .mailrc
trace: built-in: git add test test.t test.i .mailrc
trace: run_command: ./rot13.sh
trace: run_command: ./rot13.sh

(which is a part of the output of GIT_TRACE=2 ./t0021-conversion.sh -x
-v -i)


diff --git t/t0021-conversion.sh t/t0021-conversion.sh
index 6c6d77b51a..32c27d513b 100755
--- t/t0021-conversion.sh
+++ t/t0021-conversion.sh
@@ -77,6 +77,7 @@ test_expect_success setup '
 
        {
            echo "*.t filter=rot13"
+           echo ".mailrc filter=rot13"
            echo "*.i ident"
        } >.gitattributes &&
 
@@ -88,9 +89,10 @@ test_expect_success setup '
        cat test >test.t &&
        cat test >test.o &&
        cat test >test.i &&
-       git add test test.t test.i &&
+       cat test >.mailrc &&
+       git add test test.t test.i .mailrc &&
        rm -f test test.t test.i &&
-       git checkout -- test test.t test.i &&
+       git checkout -- test test.t test.i .mailrc &&
 
        echo "content-test2" >test2.o &&
        echo "content-test3 - filename with special characters" >"test3 '\''sq'\'',\$x=.o"
@@ -102,6 +104,7 @@ test_expect_success check '
 
        test_cmp test.o test &&
        test_cmp test.o test.t &&
+       test_cmp test.o .mailrc &&
 
        # ident should be stripped in the repository
        git diff --raw --exit-code :test :test.i &&
@@ -110,9 +113,12 @@ test_expect_success check '
        test "z$id" = "z$embedded" &&
 
        git cat-file blob :test.t >test.r &&
+       git cat-file blob :.mailrc >.mailrc.r &&
 
        ./rot13.sh <test.o >test.t &&
-       test_cmp test.r test.t
+       ./rot13.sh <test.o >.mailrc &&
+       test_cmp test.r test.t &&
+       test_cmp .mailrc.r .mailrc
 '
 
 # If an expanded ident ever gets into the repository, we want to make sure that

Re: git filters don't get applied to dotfiles

[Adrien LEMAIRE]

Hi Dennis, and thanks for looking into this.

I cannot reproduce this issue anymore, and it works as expected:

$ GIT_TRACE=2 config add .mailrc
23:31:05.135580 git.c:439               trace: built-in: git add .mailrc
23:31:05.135902 run-command.c:663       trace: run_command: 'sed -e
'\''s/gmail.com:.*@smtp/gmail.com:PASSWORD@smtp/'\'''

$ config diff --cached
diff --git a/.mailrc b/.mailrc
new file mode 100644
index 0000000..2698128
--- /dev/null
+++ b/.mailrc
@@ -0,0 +1,4 @@
+account gmail {
+  set v15-compat
+  set mta=smtp://lemaire.adrien%40gmail.com:PASSWORD@smtp.gmail.com:587
smtp-use-starttls
+}

To answer your question, yes I first added the file without a filter.
But I'm pretty sure I did a `config restore --staged .mailrc` after
creating the filter (and I actually repeated the operation several
times before contacting you the other day), but I must have been wrong
about that.
I didn't know about the GIT_TRACE environment variable. Thank you for
teaching me something, and sorry about the false bug report.

Best regards
Adrien

On Mon, Dec 30, 2019 at 1:02 AM Dennis Kaarsemaker
<dennis@kaarsemaker.net> wrote:
>
> On Fri, 2019-12-27 at 16:51 +0900, Adrien LEMAIRE wrote:
> > I'd like to report a bug regarding git filters not being applied to
> > files beginning with a dot character "."
> > Using git version 2.24.1
> > Please let me know if there is a better way to report bugs. The github
> > page only mentions this email.
>
> <snip reproduction recipe>
>
> I was not able to reproduce this in the git test suite with a quick
> patch (see below). Your output does not show any git add command, is it
> possible that you added the changes before configuring the filter?
>
> If you set GIT_TRACE=2 in your environment before doing the git add of
> the .mailrc file, you should see it run the filter command. It should
> look something like:
>
> + git add test test.t test.i .mailrc
> trace: built-in: git add test test.t test.i .mailrc
> trace: run_command: ./rot13.sh
> trace: run_command: ./rot13.sh
>
> (which is a part of the output of GIT_TRACE=2 ./t0021-conversion.sh -x
> -v -i)
>
>
> diff --git t/t0021-conversion.sh t/t0021-conversion.sh
> index 6c6d77b51a..32c27d513b 100755
> --- t/t0021-conversion.sh
> +++ t/t0021-conversion.sh
> @@ -77,6 +77,7 @@ test_expect_success setup '
>
>         {
>             echo "*.t filter=rot13"
> +           echo ".mailrc filter=rot13"
>             echo "*.i ident"
>         } >.gitattributes &&
>
> @@ -88,9 +89,10 @@ test_expect_success setup '
>         cat test >test.t &&
>         cat test >test.o &&
>         cat test >test.i &&
> -       git add test test.t test.i &&
> +       cat test >.mailrc &&
> +       git add test test.t test.i .mailrc &&
>         rm -f test test.t test.i &&
> -       git checkout -- test test.t test.i &&
> +       git checkout -- test test.t test.i .mailrc &&
>
>         echo "content-test2" >test2.o &&
>         echo "content-test3 - filename with special characters" >"test3 '\''sq'\'',\$x=.o"
> @@ -102,6 +104,7 @@ test_expect_success check '
>
>         test_cmp test.o test &&
>         test_cmp test.o test.t &&
> +       test_cmp test.o .mailrc &&
>
>         # ident should be stripped in the repository
>         git diff --raw --exit-code :test :test.i &&
> @@ -110,9 +113,12 @@ test_expect_success check '
>         test "z$id" = "z$embedded" &&
>
>         git cat-file blob :test.t >test.r &&
> +       git cat-file blob :.mailrc >.mailrc.r &&
>
>         ./rot13.sh <test.o >test.t &&
> -       test_cmp test.r test.t
> +       ./rot13.sh <test.o >.mailrc &&
> +       test_cmp test.r test.t &&
> +       test_cmp .mailrc.r .mailrc
>  '
>
>  # If an expanded ident ever gets into the repository, we want to make sure that
>