NEW DATE! Bugzilla will undergo maintenance 2024-03-28 18h00 CET. Bugzilla will be placed in read-only mode at that time.

Some Eclipse Foundation services are deprecated, or will be soon. Please ensure you've read this important communication.
Bug 549832 - Eclipse git cannot pull/push files to a remote origin with authenticated PROXY
Summary: Eclipse git cannot pull/push files to a remote origin with authenticated PROXY
Status: RESOLVED FIXED
Alias: None
Product: EGit
Classification: Technology
Component: Core (show other bugs)
Version: 5.3   Edit
Hardware: PC Windows 7
: P3 major (vote)
Target Milestone: 5.6   Edit
Assignee: Project Inbox CLA
QA Contact:
URL:
Whiteboard:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-08-07 03:55 EDT by Panos Kavalagios CLA
Modified: 2019-10-08 13:11 EDT (History)
4 users (show)

See Also:


Attachments
eclipse-workspace.log (4.32 KB, text/plain)
2019-08-08 05:37 EDT, Panos Kavalagios CLA
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Panos Kavalagios CLA 2019-08-07 03:55:48 EDT
Eclipse git does not work with authenticated PROXY credentials. It displays the following error when attempting to clone/pull/push to a remote repository:

https://github.com/xxx: 407 Proxy Authentication Required:

The PROXY is setup correctly in both .gitconfig and Eclipse though. You can download from within Eclipse and the command line git works fine and downloads and uploads the project to github.
Comment 1 Jay Arthanareeswaran CLA 2019-08-07 05:31:42 EDT
This looks like a network or credential issue rather than an eclipse. In any case moving to Egit for a response.
Comment 2 Panos Kavalagios CLA 2019-08-07 07:16:28 EDT
I strongly believe it is Egit issue. To sum up, because it was not clear:

- Eclipse PROXY setup in Preferences -> General -> Network Connections is OK! You can download and install software from repos and marketplace just fine.
- Git command line PROXY setup in %userprofile%\.gitconfig is OK! You can use "git pull" and "git push" command line from https://git-scm.com/ just fine.
- Eclipse Egit PROXY setup (probably the same of Eclipse or gitcongig?) is NOT OK! It cannot download or upload with Team -> pull/push, but it works for local repository with commit and sync commands.
Comment 3 Thomas Wolf CLA 2019-08-08 05:22:56 EDT
Not enough information. Which EGit version? Which Java version? Which Eclipse version? What kind of proxy (http/socks4/socks5)? What kind of proxy authentication mechanisms? Any stack traces in the log? Is a wire-trace of the connection messages between Eclipse and the proxy server available (including message headers and content)?
Comment 4 Panos Kavalagios CLA 2019-08-08 05:37:09 EDT
Created attachment 279520 [details]
eclipse-workspace.log

Please find my answers below:

- EGit version 5.3.0
- Java version 1.8.211
- Eclipse version 2019-03
- HTTP PROXY
- Proxy authentication mechanisms is username and password
- Stack traces included in the eclipse-workspace.log
- No wire-trace of the connection messages between Eclipse and the proxy server are available, since this is a corporate environment. Any suggestion to collect them with a user space program?
Comment 5 Thomas Wolf CLA 2019-08-08 06:24:12 EDT
Wireshark is one possibility. If you can't or are not allowed to install that, there's some built-in logging in Java; see https://stackoverflow.com/questions/1445919/how-to-enable-wire-logging-for-a-java-httpurlconnection-traffic .
Comment 6 Panos Kavalagios CLA 2019-08-08 09:53:48 EDT
I have installed wireshark on a test machine that have admin rights and captured the traffic to proxy port 8012. Is it possible to send it only to you? I wouldn't like to disclose any sensitive information about my credentials and network.
Comment 7 Thomas Wolf CLA 2019-08-08 12:41:32 EDT
I've sent you an e-mail with my PGP key. Feel free to send the log encrypted via e-mail to me.
Comment 8 Michael Keppler CLA 2019-08-09 01:53:55 EDT
Please fix this warning from your log first, to avoid that you are actually using some different settings in Eclipse than with command line git:

!MESSAGE Warning: The environment variable HOME is not set. The following directory will be used to store the Git
user global configuration and to define the default location to store repositories: 'C:\Users\kavalpa'. If this is
not correct please set the HOME environment variable and restart Eclipse. Otherwise Git for Windows and
EGit might behave differently since they see different configuration options.
This warning can be switched off on the Team > Git > Confirmations and Warnings preference page.
Comment 9 Panos Kavalagios CLA 2019-08-09 05:50:01 EDT
The wireshark information has been submitted.

I confirm the chosen HOME is correct. So, both are using the correct .gitconfig under my home directory C:\Users\kavalpa.
Comment 10 Thomas Wolf CLA 2019-08-09 19:15:18 EDT
Thanks, got the wire-trace. Didn't help much. For some reason, the JDK UrlConnection would just not continue after the 407 with the normal proxy authentication using the scheme advertised by your proxy (BASIC). There's just a single HTTP CONNECT request, followed by the 407 reply, then nothing more.

After a lot of head scratching I managed to set up a http proxy locally and gave this a try. And lo and behold, I actually could reproduce your problem.

After some hours of debugging I finally found the reason: https://bugs.openjdk.java.net/browse/JDK-8210814

Since Java 8, "basic" authentication for proxy authentication for tunneling https traffic through an http proxy is disabled by default!

There's a system property that can be set to re-allow basic authentication for https-through-http-proxying: -Djdk.http.auth.tunneling.disabledSchemes=""

See also https://stackoverflow.com/questions/41806422/java-web-start-unable-to-tunnel-through-proxy-since-java-8-update-111
Comment 11 Thomas Wolf CLA 2019-08-10 10:29:05 EDT
I don't quite understand this move in JDK. The correct place to disable basic proxy authentication seems to me to be in the configuration of proxy server, which just should not advertise basic in this case.

But given that JDK decided to do this on the client side, what are the options?

1. Include system property -Djdk.http.auth.tunneling.disabledSchemes="" in
   eclipse.ini by default?

2. Tell users to include it there themselves if needed? Might invalidate app
   signature, for instance on Mac, but so does installing any bundle.

3. Tell users to start Eclipse with environment variable
   _JAVA_OPTIONS="-Djdk.http.auth.tunneling.disabledSchemes= " set? Not nice.

4. Tell users to edit the network config file inside the JDK to set this? Ugh...

5. I don't see any way to trick the JDK HttpUrlConnection into using pre-emptive
   proxy authentication. Setting the "Proxy-Authorization: Basic <credentials>"
   header doesn't work; JDK ignores it for setting up the tunnel.

6. Switch EGit to use the Apache Http transport instead of the JDK Http
   transport by default? I'm not 100% sure, but it appears that with Apache
   Http this might just work, or that one could at least make it use
   pre-emptive proxy auth.

(2), (3), and (4) would mean we'd just mention this somewhere in the EGit manual and then close this as "NOT_ECLIPSE", but it's not exactly user friendly.
Comment 12 Thomas Wolf CLA 2019-08-11 12:08:01 EDT
(In reply to Thomas Wolf from comment #11)
> 6. Switch EGit to use the Apache Http transport instead of the JDK Http
>    transport by default? I'm not 100% sure, but it appears that with Apache
>    Http this might just work, or that one could at least make it use
>    pre-emptive proxy auth.

After adding a CredentialsProvider in the JGit Apache HTTP code this indeed works out of the box with Apache HTTP. No need for pre-emptive proxy auth even.

We might do this for EGit 5.6.0. Same procedure as for the SSH transport: add a user preference allowing to switch between the two, set Apache HTTP as default. If no serious problems crop up, remove the preference and make
EGit unconditionally use Apache HTTP.

For EGit 5.5.0 it's IMO too late. I don't like swapping such basic stuff too close to the release date.

What do others think?
Comment 13 Panos Kavalagios CLA 2019-08-12 03:52:31 EDT
Just to confirm that the workaround with the option:

-Djdk.http.auth.tunneling.disabledSchemes=""

in eclipse.ini works fine in my case as well. You may want to lower the importance, as it is not blocker anymore.

The fix could follow the Eclipse Core approach. The same error is displayed in Eclipse in the traces, when you are trying to download a software in Help -> Install New Software or Marketplace, but then it continues. If Apache HTTP is used there, it would be better for Egit to be aligned as well.
Comment 14 Thomas Wolf CLA 2019-08-12 06:41:03 EDT
MPC (Marketplace Client) uses Apache HTTP. Don't know what "Install New Software" uses.
Comment 15 Matthias Sohn CLA 2019-08-12 08:53:49 EDT
(In reply to Thomas Wolf from comment #12)
> (In reply to Thomas Wolf from comment #11)
> > 6. Switch EGit to use the Apache Http transport instead of the JDK Http
> >    transport by default? I'm not 100% sure, but it appears that with Apache
> >    Http this might just work, or that one could at least make it use
> >    pre-emptive proxy auth.
> 
> After adding a CredentialsProvider in the JGit Apache HTTP code this indeed
> works out of the box with Apache HTTP. No need for pre-emptive proxy auth
> even.
> 
> We might do this for EGit 5.6.0. Same procedure as for the SSH transport:
> add a user preference allowing to switch between the two, set Apache HTTP as
> default. If no serious problems crop up, remove the preference and make
> EGit unconditionally use Apache HTTP.
> 
> For EGit 5.5.0 it's IMO too late. I don't like swapping such basic stuff too
> close to the release date.
> 
> What do others think?

I agree, we shouldn't do it shortly before release 5.5

+1 for doing this early in the next release
I'll create stable-5.5 branch soon
Comment 16 Eclipse Genie CLA 2019-09-25 16:08:55 EDT
New Gerrit change created: https://git.eclipse.org/r/150173
Comment 17 Eclipse Genie CLA 2019-10-07 14:00:54 EDT
New Gerrit change created: https://git.eclipse.org/r/150746
Comment 18 Eclipse Genie CLA 2019-10-08 03:41:59 EDT
Gerrit change https://git.eclipse.org/r/150173 was merged to [master].
Commit: http://git.eclipse.org/c/jgit/jgit.git/commit/?id=2abd3c43bd61a6e06e2ea0ecab2d9f769a7223d3
Comment 19 Eclipse Genie CLA 2019-10-08 13:10:21 EDT
Gerrit change https://git.eclipse.org/r/150746 was merged to [master].
Commit: http://git.eclipse.org/c/egit/egit.git/commit/?id=909896cf1a7657ac721045f673983eed57664643