Bug 561666 - git-crypt filter stopped working
Summary: git-crypt filter stopped working
Status: VERIFIED FIXED
Alias: None
Product: JGit
Classification: Technology
Component: JGit (show other bugs)
Version: 5.6   Edit
Hardware: PC Linux
: P3 normal (vote)
Target Milestone: 5.6.2   Edit
Assignee: Thomas Wolf CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-04-01 19:46 EDT by Andrei Pozolotin CLA
Modified: 2020-04-03 12:28 EDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Andrei Pozolotin CLA 2020-04-01 19:46:01 EDT
RE: https://github.com/AGWA/git-crypt

1. git-crypt filter stopped working
after upgrade from eclipse 2019-06 to 2020-03

2. any attempt to commit anything in eclipse egit/jgit
results in exception, for example, snippet from eclipse log:

!ENTRY org.eclipse.egit.ui 4 0 2020-04-01 18:30:43.116
!MESSAGE Failed to add resource to index
!STACK 0
org.eclipse.jgit.api.errors.FilterFailedException: Execution of filter command '"git-crypt" clean' on file 'roles.d/aws/func/server/setup/files/etc/readme.md' failed with return code '127', message on stderr: '"git-crypt" clean: "git-cry
pt": command not found
'
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.filterClean(WorkingTreeIterator.java:503)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.filterClean(WorkingTreeIterator.java:460)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.possiblyFilteredInputStream(WorkingTreeIterator.java:417)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.possiblyFilteredInputStream(WorkingTreeIterator.java:402)
        at org.eclipse.jgit.treewalk.WorkingTreeIterator.getEntryContentLength(WorkingTreeIterator.java:605)
        at org.eclipse.jgit.api.AddCommand.call(AddCommand.java:200)
        at org.eclipse.egit.ui.internal.actions.CommitActionHandler$3.run(CommitActionHandler.java:216)
        at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

3. relevant project .gitattributes file content:


* filter=git-crypt diff=git-crypt

/.gitignore       !filter !diff
/.gitmodules      !filter !diff
/.gitattributes   !filter !diff


4. git-crypt program is actually present:

# which git-crypt
/usr/bin/git-crypt

5. both git and git-crypt continue to work fine
from command line, outside of eclipse

6. the program search path reported by bash is identical 
to the path reported inside running eclipse

7. is there any way to enable an option to log/print
actual path used by jgit/egit when invoking an external filter?
Comment 1 Thomas Wolf CLA 2020-04-02 07:25:35 EDT
(In reply to Andrei Pozolotin from comment #0)
> org.eclipse.jgit.api.errors.FilterFailedException: Execution of filter
> command '"git-crypt" clean' on file
> 'roles.d/aws/func/server/setup/files/etc/readme.md' failed with return code
> '127', message on stderr: '"git-crypt" clean: "git-cry
> pt": command not found

Why is the command "git-crypt" instead of just git-crypt?

Can you show the filter definition from the git config?
Comment 2 Andrei Pozolotin CLA 2020-04-02 09:45:08 EDT
1. this is how it is setup by git-crypt by default,
after:
-------------
git-crypt init
-------------

in the location of:

-------------
[project repository]/.git/config
-------------

-------------

[core]
        repositoryformatversion = 0
        filemode = true
        bare = false
        logallrefupdates = true
[filter "git-crypt"]
        smudge = \"git-crypt\" smudge
        clean = \"git-crypt\" clean
        required = true
[diff "git-crypt"]
        textconv = \"git-crypt\" diff

-------------

2. removing quotes does solve the problem, thank you!

3. now you will say "it is git-crypt" problem :-)

4. only: how did it work before? 
git-crypt did not change
with eclipse change from eclipse 2019-06 to 2020-03:
https://www.archlinux.org/packages/community/x86_64/git-crypt/
Last Updated:	2018-01-09 05:16 UTC

5. I also cross-posted this at github/git-crypt:
https://github.com/AGWA/git-crypt/issues/205
Comment 3 Thomas Wolf CLA 2020-04-02 10:16:47 EDT
(In reply to Andrei Pozolotin from comment #2)
> 2. removing quotes does solve the problem, thank you!
> 
> 3. now you will say "it is git-crypt" problem :-)
No, I won't.

> 4. only: how did it work before? git-crypt did not change
But JGit did. See bug 558577.

JGit < 5.7.0 did

  $ sh -c '<cmd> "$@"' <cmd> <args>

(with your filter settings sh -c '"git-crypt" "$@"' "git-crypt" clean )

In bug 558577 a user had problems with backslashes. So we (well, I)
changed that to

  $ sh -c '$0 "$@"' <cmd> <args>

(with your filter settings sh -c '$0 "$@"' "git-crypt" clean )

since that avoids that problem.
 
As it turns out now this probably wasn't the correct idea.

In any case: the backslash problem has in the meantime been fixed better for the platform where it matters (Windows/Cygwin). Linux users will normally not have file names or paths containing backslashes. So it's probably best to undo this $0 change.

Though we will have to check what happens when core.hooksDir is set to a path containing blanks. (Hooks are run via the same mechanism.)
Comment 4 Andrei Pozolotin CLA 2020-04-02 11:42:57 EDT
git-crypt is confident it is "not them"
https://github.com/AGWA/git-crypt/issues/205

-------------

Quoting is used just like anywhere in a shell: to avoid issues if the path includes a space or other special characters. Rather than checking the resolved binary name and path for issues, it just quotes it by default to be safe. This should be just fine. Whatever Eclipse is doing is wrong, they should handle this as an already ready to use shell command the way POSIX would handle it, not try to add more escaping.

-------------
Comment 5 Thomas Wolf CLA 2020-04-02 12:29:28 EDT
(In reply to Andrei Pozolotin from comment #4)
> git-crypt is confident it is "not them"
> https://github.com/AGWA/git-crypt/issues/205
> 
> -------------
> 
> Quoting is used just like anywhere in a shell: to avoid issues if the path
> includes a space or other special characters. Rather than checking the
> resolved binary name and path for issues, it just quotes it by default to be
> safe. This should be just fine. Whatever Eclipse is doing is wrong, they
> should handle this as an already ready to use shell command the way POSIX
> would handle it, not try to add more escaping.
> 
> -------------

Or whatever. We don't escape, we _do_ pass it on as is. But depending on the way one does that, the shell decides to do its own magic or not. And in fact with the $0 it is passed on verbatim, and then the quoted version generated by git-crypt init fails. What is correct, though, is that we need to fix this.
Comment 6 Eclipse Genie CLA 2020-04-02 17:21:26 EDT
New Gerrit change created: https://git.eclipse.org/r/160405
Comment 7 Eclipse Genie CLA 2020-04-03 03:22:05 EDT
Gerrit change https://git.eclipse.org/r/160405 was merged to [stable-5.6].
Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=2640d38f140fd9239395b9756c8b1f828ed37dd4
Comment 8 Thomas Wolf CLA 2020-04-03 11:55:07 EDT
EGit nightly should have the fix for this. Update site at https://download.eclipse.org/egit/updates-nightly .
Comment 9 Andrei Pozolotin CLA 2020-04-03 12:13:11 EDT
1. I confirm that this resolves the issue:
* https://download.eclipse.org/egit/updates-nightly/
* Eclipse EGit 5.8.0.202004031526 org.eclipse.egit

2. thank you so much for your attention