How to configure Python with M$ Forefront (TMG) proxy - python

I'm trying to use Python 3.7 and PIP (on Windows 7) behind corporate firewall but I'm getting following error:
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed:
407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/pywin32/
I've tried following configurations to pass proxy access and anyone of these is not working :(
set HTTP_PROXY=http://DOMAIN\user:pass#proxyhost:port
set HTTP_PROXY=http://user:pass#proxyhost:port
set HTTP_PROXY=http://proxyhost:port

I had a problem with my organization proxy auth too and was getting 407 errors in Windows OS, finally solved it using CTNLM , so first try to config ctnlm for your proxy and test it with your browser (the config is located in %PROGRAMFILES%\Cntlm\cntlm.ini for windows and default proxy should be something like http://localhost:3128) and if it worked give it to pip install : --proxy http://localhost:3128

Related

Artifactory 7.x - pypi proxy errror

I have a local Pypi repository with simple-default layout. It has anonymous read access granted. I am using this repository to store my libraries.
After the upgrade from Artifactory 6.x to 7.x it is not possible to install any library using pip; it worked in 6.x. Following errors are returned on pip install my-python-lib:
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after
connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='host', port=8081):
Read timed out. (read timeout=30.0)")': /artifactory/api/pypi/my-python-repo/simple/my-python-lib/
Could not install packages due to an EnvironmentError: HTTPConnectionPool(host='host', port=8081):
Max retries exceeded with url: /artifactory/api/pypi/my-python-repo/simple/my-python-lib/ (Caused by
ReadTimeoutError("HTTPConnectionPool(host='host', port=8081): Read timed out. (read timeout=30.0)"))
Setting longer --default-timeout doesn't work as well.
When I try to open host/artifactory/api/pypi/my-python-repo/simple/my-python-lib/ in the browser, it returns following error (after about 2 minutes):
502 Proxy Error
The proxy server received an invalid response from an
upstream server. The proxy server could not handle the request GET
/artifactory/api/pypi/my-python-repo/simple/my-python-lib/.
Reason: Error reading from remote server
It seems that a reverse proxy(Nginx or HTTPD) is configured on top of Artifactory and Artifactory 7.x has a different configuration when compared to Artifactory 6.x. I would recommend referring to this KB article for more insights.

Testing electron applications: unable to initialize test as prescribed with Robot Framework

I am attempting to test a packaged Electron application (an .exe) with RobotFramework on a windows 10 machine. I have other RobotFramework tests for other web applications that are running fine on the same machine.
I have the chromedriver of what I believe to be the correct version within the same directory as my .robot script. I have also tried different version of the chromedriver.exe. Currently I am using chromedriver 80.0.3987.16 The package.json of my app indicates that the electron version is 5.0.1
I have created a vars.py file and have the robot script instantiate the driver with the code as indicated in https://spage.fi/electron
As prescribed by the spage.fi link: within the vars.py file:
binary_location = {"chromeOptions": {"binary": "myelectronapp.exe"}}
Note that the .exe and its associated files from the package process I've included in the same directory as the robot script in order to remove any path dependancies.
My robot snippet:
*** Settings ***
Documentation Suite description
Library SeleniumLibrary
Variables vars.py
*** Test Cases ***
Test title
[Tags] DEBUG
Log To Console calling create driver
Create Webdriver Remote desired_capabilities=${binary_location} command_executor=http://localhost:9515
My port 9515 is open via an inbound rule.
My experience has been regardless of the chromedriver version that I think aligns, that I always get a connection refusal error:
[ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000000003E3C160>: Failed to establish a new connection: [WinE
rror 10061] No connection could be made because the target machine actively refused it')': /session
x 3 retries
I'm figuring it is not a port accessibility issue because when I run chromedriver from the command line and then invoke the url at port 9515, I get the callback trace that I would expect.
Its difficult to know if this is from a syntax issue or some other connection attribute that I have overlooked.
This error message...
[ WARN ] Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<urllib3.connection.HTTPConnection object at 0x0000000003E3C160>: Failed to establish a new connection: [WinError 10061] No connection could be made because the target machine actively refused it')': /session
...implies that the ChromeDriver was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session.
Your main issue is the incompatibility between the version of the binaries you are using as follows:
You are using chromedriver=80.0.3987.16
Release Notes of chromedriver=80.0.3987.16 clearly mentions the following :
Supports Chrome version 80
Presumably you are using chrome= 79.0
Release Notes of ChromeDriver v79.0.3945.36 clearly mentions the following :
Supports Chrome 79
So there is a clear mismatch between the ChromeDriver v80.0 and the Chrome Browser v79.0
Solution
Ensure that:
Selenium is upgraded to current levels Version 3.141.59.
ChromeDriver is downgraded to ChromeDriver v79.0 level.
Chrome is updated to current Chrome Version 79.0 level. (as per ChromeDriver v79.0 release notes)
Clean your Project Workspace through your IDE and Rebuild your project with required dependencies only.
If your base Web Client version is too old, then uninstall it and install the recent GA and released version of Web Client.
Take a System Reboot.
Execute your #Test as non-root user.
Always invoke driver.quit() within tearDown(){} method to close & destroy the WebDriver and Web Client instances gracefully.

Python - Pip Install - Proxy Error - 'Cannot connect to proxy.', OSError'

I want to install some modules in a Enterprise VM in order to create some Python Scripts. I'm trying to use PIP with Proxy to do it. I'm using this command lines:
C:\Users\user>SET HTTPS_PROXY=https://user:pass#199.00.11.11:8080
C:\Users\user>SET PROXY=http://user:pass#199.00.11.11:8080
C:\Users\user>pip install datetime
To have access to my virtual machine I've this credentials:
USER: NAN/user
PASS: pass
But I am getting this error:
Collecting datetime
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/datetime/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/datetime/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/datetime/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/datetime/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/datetime/
Could not find a version that satisfies the requirement datetime (from versions: )
No matching distribution found for datetime
What I need to do in order to get the python module?
In my case, it was only because the 【VPN】 software modified the 【System Proxy】 configuration of the 【IE】 browser.
I modified it back in this way:
open 【IE】
Press 【Alt+X】 and scroll down to 【Internet options】
Click on the 【Connections】 tab
Press the 【LAN settings】 button
Uncheck the 【Proxy Server】
This is how my problem was successfully solved.
Try like this:
set HTTP_PROXY=http://199.00.11.11:8080
set HTTPS_PROXY=https://199.00.11.11:8080
First, install a proxy authentication service like CNTLM)
Second,
set http_proxy=http://username:password#proxyAddress:port
set https_proxy=https://username:password#proxyAddress:port
I also faced this issue even with proxy, adding "--isolated" helped
For eg: pip install datetime --isolated
Note: My proxy is already set in "pip config"
Use the following code
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org datetime
You can change datetime with the name of any package you want.
I recently had a similar issue doing pip3 install awscli on my local mac. I checked my env and there were no proxies set there. The solution that worked for me was disabling proxies from the network which got set on my machine for some reason. To do that
Click on Wifi
Open Network Preferences
Click Advanced button
Go to proxies tab
Uncheck Web Proxy and Secure Web Proxy
Again, I know that network proxy misconfiguration is rare and this answer may not be the most suitable answer. But this might come in useful for some folks as this is right now the first link that comes on google search for pip install ProxyError.
This works on Windows 10 for me:
Search for Proxy Settings in Windows Start Search, open it
Under Manual Proxy Setup, uncheck "Use a proxy server".
Alternatively, you may just add exceptions for the sources you want to install packages from.
Try like: Go to *control panel --> Internet options --> Connections -->LAN Settings ->mark 'use proxy server for local address' options.
I ran into the same problem and the solution in my case was to unset the HTTPS_PROXY variable, because my proxy does not accept https requests.
I found the solution in this page : https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html#https-proxy-error-http-proxy
All proxy variables are set by the os admin :
HTTP_PROXY=http://my_domain:8888
NO_PROXY=*.internal_domain,172.*,
HTTPS_PROXY=http://my_domain:8888
To update pip for example, I have to run :
unset HTTPS_PROXY ; pip install -U pip
If your proxy configration is OK,
It may be because your pip version is too high.
Try to lower your pip version
python -m pip install pip==19.3.1
Then restart your terminal.
You can try this method to see if it works

Private PyPi Repository in Closed Network

I am on a closed network (aka no access to the internet) thus I need to create my own PyPi server. I am following the instructions here:
How to set up and use a private PyPI repo
The only differences:
I did not do was set up Apache authentication. No need for it.
I put everything in /src/pypi (all pointers were updated)
I was able to get Apache operational and (supposedly) got my ~/.pip/pip.conf file operational. Here are the contents:
[global]
extra-index-url=https://pypi.myserver.com/pypi/
trusted-host = pypi.myserver.com
When I try to run the command:
pip install foobar-utils
I get the following:
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at xxx>: Failed to establish a new connection: [Errorno -2] Name or service not known',)' /simple/foobar-utils
I know it's hitting apache, I see it in my logs.
Any help would be great. Thanks
If you want to completely replace PyPI you must use index-url instead of extra-index-url. With extra-index-url pip still search packages at https://pypi.org, extra-index-url is an additional server to search.
Your pip.conf must be
[global]
index=https://pypi.myserver.com/pypi/
index-url=https://pypi.myserver.com/pypi/
trusted-host = pypi.myserver.com
index is for pip search, index-url for pip install.

Error when installing through Pip - Windows

When trying to install Django through pip we get an error message.
So it's an protocol error, and then since he has in Swedish it says something like:
"a try was made to get access to a socket in a way that is forbidden by the table of access"....
It seems like we need any admin access or something? We tried to run the command prompt as an administrator. By marking the "run as administrator" box in the command prompt settings. We are lost, any help is greatly appreciated.
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ProtocolError('Connection aborted.', error(10013, 'Ett f\xf6rs\xf6k gjordes att f\xe5 \xe5tkomst till en socket p\xe5 ett s\xe4tt som \xe4r f\xf6rbjudet av \xe5tkomstbeh\xf6righeterna'))': /simple/django/
Could not find a version that satisfies the requirement django (from versions: )
No matching distribution found for django
make sure that python.exe is allowed in your firewall settings and any antivirus firewall settings. I had the same problems, and had to allow the program under my AVG firewall settings cause it still wouldn't work even after I had allowed it under Windows firewall.

Categories