Community
Participate
Working Groups
Newer GPG version may store private keys in .gnupg/private-keys-v1.d/ in a format that JGit cannot handle. This may affect any *.key file in that directory. This can occur is gpg-agent 2.2.22 or newer is used, or even with older gpg-agent versions if option enable-extended-key-format is given, either on the command line or in the gpg-agent.conf.[1] The new format is described at [2]. BouncyCastle has no support yet for this.[3] The old format is a binary serialized s-expression for the (possibly encrypted) private key. There are several problems here: 1. The new format is a key-value format similar to e-mail headers (or to headers in git objects), with continuation lines. The key is just one of the values. One must parse this key-value format with continuation lines to extract the "Key:" value. Note that GPG matches keys case-insensitively, so the existing RawParseUtils cannot be used. But it is of course possible to parse this format. 2. The key value itself is a human-readable externalization of the s-expression; it cannot be passed as is to BouncyCastle's SExprParser. The value needs to be converted first to the binary serialized form. This is also possible, but means one has to write a fairly complete s-expression scanner that handles at least lists, tokens, hex values, and quoted strings. (I think that's all of the atoms that occur in GPG private keys.) The quoted string format does not correspond to any of the existing QuotedString formats in JGit. Again, this is doable. 3. Encrypted keys are encrypted using AES/OCB/NoPadding. This requires a custom PBEProtectionRemoverFactory; the default JcePBEProtectionRemoverFactory is hard-wired to AES/CBC/NoPadding. Also doable. But then there's a show-stopper: OCB has been removed from the BouncyCastle bundle in Orbit![4] Probably due to concerns over the patents on OCB, which, AFAIK, are valid in the U.S. only. * Are these concerns still valid after the move of the Eclipse Foundation to Europe? Or could we now include a full-fledged BC in Orbit? * BC apparently is fine with having OCB in their code base based on the license grant for open-source at [5]. Why is Eclipse not happy with this? After, Orbit only OSGi-fies the BC bundles. We're not even trying to implement OCB; we just want to _use_ it. If Eclipse cannot have a BC that includes OCB, what other possibilities would there be to solve this problem? [1] https://www.gnupg.org/documentation/manuals/gnupg/Agent-Options.html [2] https://github.com/gpg/gnupg/blob/master/agent/keyformat.txt [3] https://github.com/bcgit/bc-java/issues/794 [4] https://dev.eclipse.org/ipzilla/show_bug.cgi?id=9421#c10 [5] https://www.cs.ucdavis.edu/~rogaway/ocb/license.htm
In order to discuss with the Eclipse lawyers if we can include OCB in BC in Orbit and use it from JGit we would need to open a CQ. We could try to use BC as a works-with dependency and not ship it from Eclipse. The user would have to download and install it separately. For that we would need to convince upstream to add OSGi headers to the manifest. And it would be still bad from usability aspect. Don't know if that's feasible at all.
(In reply to Matthias Sohn from comment #1) > In order to discuss with the Eclipse lawyers if we can include OCB in BC in > Orbit and use it from JGit we would need to open a CQ. > > We could try to use BC as a works-with dependency and not ship it from > Eclipse. The user would have to download and install it separately. For that > we would need to convince upstream to add OSGi headers to the manifest. And > it would be still bad from usability aspect. Don't know if that's feasible > at all. That would run into problems if the Orbit pcprov bundle was installed in an Eclipse (perhaps via some other project, not EGit/JGit). There's be two copies, and both might provide the same IUs with maybe even the same versions, and both would want to register their security provider with the name "BC". Here's another idea that also doesn't work: * Use Eclipse's "Create plug-in from existing jar", using the upstream BC prov jar from maven. Give the new plug-in a MANIFEST.MF that exports all packages with right version, and make it an OSGi _fragment_ with the Orbit BC prov bundle as host. If that worked, one could distribute such a bundle from outside Eclipse. Evidently this _does_ make the missing classes available, but attempting to load any of them then produces an exception because the loader sees different signatures on files in the same package. :-( Another idea (haven't tried it yet) is perhaps to detect the "openpgp-s2k3-ocb-aes" in the key value after having extracted it (point 1 in comment 0), throw a special exception, catch it in the top-level GpgSigner.signObject(), and then handle it by shelling out to gpg. Which of course adds complications for finding the right gpg executable, spawning an external process, passing the data and getting the result. And it comes with a non-negligible performance cost. There is git config gpg.program where the user could specify which executable to use. Or we could always shell out to gpg for signing. But firing up an external process for each signature may be a bit expensive?
(In reply to Thomas Wolf from comment #2) > Another idea (haven't tried it yet) [... shelling out to gpg for AES-OCB encrypted secret keys] Tried it now. That would be a possibility. A prototype works. > Or we could always shell out to gpg for signing. This is maybe not such a good idea. The problem is that we'd like to ask for a passphrase only if needed. If we keep the integration with the JGit CredentialsProvider, we have no good way to hook that into the external GPG process; so we might have to ask for a passphrase always, even if it wouldn't be needed because in the end the key is not encrypted. Or we could remove all passphrase handling for GPG signing completely, and let GPG do its thing. It usually comes with a GUI prompt helper (pinentry-mac and likewise on other systems), so it would pop up its own prompt dialog when needed.
(In reply to Thomas Wolf from comment #3) > (In reply to Thomas Wolf from comment #2) > > Another idea (haven't tried it yet) > [... shelling out to gpg for AES-OCB encrypted secret keys] > > Tried it now. That would be a possibility. A prototype works. > > > Or we could always shell out to gpg for signing. > > This is maybe not such a good idea. > [...] > Or we could remove all passphrase handling for GPG signing completely, and > let GPG do its thing. After more thoughts, I think we should: * For JGit: in org.eclipse.jgit.gpg.bc, go the full mile and do steps 1, 2, and 3 from comment 0. It'll work in environments where a full-fledged BC is available. * For EGit: implement the "external GPG" approach for signing; use BC only for signature verification. Don't use org.eclipse.jgit.gpg.bc but our own implementation. This would sign by shelling out to GPG, and leave all passphrase handling also to GPG. I tried that, and it works great on Mac. It also has the big advantage that one doesn't have to rely on internals of GPG, and it should in one fell swoop solve all (or most) of the bug reports we got about "no signing key found" or "integrate with the keystore": we'll get integration with the keystore and with gpg-agent "for free".
(In reply to Thomas Wolf from comment #4) > After more thoughts, I think we should: > > * For JGit: in org.eclipse.jgit.gpg.bc, go the full mile and do steps 1, 2, > and 3 from comment 0. It'll work in environments where a full-fledged BC > is available. After more experiments: to make it work, we'd also have to copy two BouncyCastle helper classes (SExprParser and SXprUtils) and then adapt SExprParser. That class doesn't work with keys decrypted from AES/OCB-encrypted data. Such keys don't have a hash (they don't need one, OCB takes care of that), but the BouncyCastle SExprParser always expects one. The alternative for JGit is to wait for a fix for upstream issue 794.[1] [1] https://github.com/bcgit/bc-java/issues/794
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/175332
(In reply to Eclipse Genie from comment #6) > New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/175332 And the change for EGit is https://git.eclipse.org/r/c/egit/egit/+/175336 .
Gerrit change https://git.eclipse.org/r/c/jgit/jgit/+/175332 was merged to [master]. Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=bdc48aeac756cc0471618b06d793083e63109ee0
New Gerrit change created: https://git.eclipse.org/r/c/jgit/jgit/+/176595
Gerrit change https://git.eclipse.org/r/c/jgit/jgit/+/176595 was merged to [master]. Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=704ccdc096e4f5cf2670c5c58eaf19fe1fdf4df3
Fixed in JGit so that it works if used together with the standard maven artifacts of BouncyCastle. The Eclipse version of the Bouncy Castle bundles excludes OCB; but Egit now has the option to use an external GPG executable.
