I am on a closed network (aka no access to the internet) thus I need to create my own PyPi server. I am following the instructions here:
How to set up and use a private PyPI repo
The only differences:
I did not do was set up Apache authentication. No need for it.
I put everything in /src/pypi (all pointers were updated)
I was able to get Apache operational and (supposedly) got my ~/.pip/pip.conf file operational. Here are the contents:
[global]
extra-index-url=https://pypi.myserver.com/pypi/
trusted-host = pypi.myserver.com
When I try to run the command:
pip install foobar-utils
I get the following:
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'NewConnectionError('<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at xxx>: Failed to establish a new connection: [Errorno -2] Name or service not known',)' /simple/foobar-utils
I know it's hitting apache, I see it in my logs.
Any help would be great. Thanks
If you want to completely replace PyPI you must use index-url instead of extra-index-url. With extra-index-url pip still search packages at https://pypi.org, extra-index-url is an additional server to search.
Your pip.conf must be
[global]
index=https://pypi.myserver.com/pypi/
index-url=https://pypi.myserver.com/pypi/
trusted-host = pypi.myserver.com
index is for pip search, index-url for pip install.
Related
I have a local Pypi repository with simple-default layout. It has anonymous read access granted. I am using this repository to store my libraries.
After the upgrade from Artifactory 6.x to 7.x it is not possible to install any library using pip; it worked in 6.x. Following errors are returned on pip install my-python-lib:
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after
connection broken by 'ReadTimeoutError("HTTPConnectionPool(host='host', port=8081):
Read timed out. (read timeout=30.0)")': /artifactory/api/pypi/my-python-repo/simple/my-python-lib/
Could not install packages due to an EnvironmentError: HTTPConnectionPool(host='host', port=8081):
Max retries exceeded with url: /artifactory/api/pypi/my-python-repo/simple/my-python-lib/ (Caused by
ReadTimeoutError("HTTPConnectionPool(host='host', port=8081): Read timed out. (read timeout=30.0)"))
Setting longer --default-timeout doesn't work as well.
When I try to open host/artifactory/api/pypi/my-python-repo/simple/my-python-lib/ in the browser, it returns following error (after about 2 minutes):
502 Proxy Error
The proxy server received an invalid response from an
upstream server. The proxy server could not handle the request GET
/artifactory/api/pypi/my-python-repo/simple/my-python-lib/.
Reason: Error reading from remote server
It seems that a reverse proxy(Nginx or HTTPD) is configured on top of Artifactory and Artifactory 7.x has a different configuration when compared to Artifactory 6.x. I would recommend referring to this KB article for more insights.
I am trying to install Django 1.8.11 on my Windows 10 PC, but i am getting this error when run pip install django==1.8.11:
Retrying (Retry(total=0, connect=None, read=None, redirect=None)) after connection broken by 'ConnectTimeoutError(<pip._vendor.requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x000001CE97C60D68>, 'Connection to xxxx.xxxx.xxx.xx timed out. (connect timeout=15)')': /simple/pip/
(xxxx.xxxx.xxx.xx Is an address that I use as proxy some times)
I have Python version: 3.5.4
and pip version: 9.0.1
I have checked proxy settings with netsh winhttp show proxy
Current WinHTTP proxy settings:
Direct access (no proxy server).
I am not behind any corporate proxy
My system proxy settings are
Automatically detect settings -> ON
Use setup script -> OFF
Use a proxy server -> OFF
Also tried ping pypi.python.org
Ping statistics for 151.101.4.223:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 92ms, Maximum = 102ms, Average = 95ms
So, I have access to the internet
Also i removed temp files, tried python -m pip --proxy="" install django==1.8.11, searched the registry for data or value xxxx.xxxx.xxx.xx, same result...
If anyone knows where is that xxxx.xxxx.xxx.xx configured so I can remove it, don't know what else to do.
Look into the current environment variables, try to find any proxy setting:
set | find "proxy"
If anything found — unset the variables.
List settings from the config file:
pip config list | find "proxy"
If anything found — unset the variable using pip config unset or pip config edit. See the docs for pip config at https://pip.pypa.io/en/stable/reference/pip_config/
Try installing the latest version pip install Django
https://pypi.org/project/Django/
I'm just playing with a simple example to get a basic understanding of Docker going. Here is my Docker image file:
FROM python:3.7-alpine
# copy all the files to the container
COPY . /test
WORKDIR /test
# install dependencies
RUN pip install pip_system_certs --trusted-host pypi.org --trusted-host files.pythonhosted.org --trusted-host pypi.python.org
RUN pip install -r requirements.txt
# run the command
CMD ["python", "./test_script.py"]
The trusted-host options are what allow us to get around corporate network security settings and install packages internally on windows and they seem to work in Docker too but only for some packages. For instance if my requirements.txt includes flask and requests everything is fine, but pandas and numpy give me
WARNING: Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError(SSLCertVerificationError(1, '[SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed
certificate in certificate chain (_ssl.c:1076)'))': /simple/numpy/
and fails. I think it's weird that this is working for some packages but not others.
Any help appreciated.
Using Docker Desktop in Windows 10.
I know my company's big corporate proxy removes (most) normal certificates and re-wraps them in a self-signed cert. This caused lots of similar headaches for me. I resolved it by:
Figuring out what our root cert was by visiting an internet site in Chrome, clicking on the lock in the address bar, and viewing the certification path for the site's certificate. The root CA was our internal one.
Going to the certificate management in Windows control panel and under "Trusted Root Certification" found my company's internal root cert and exported it as a "Base-64 encoded X.509" file.
Copied that certificate file into my Docker container and added it as a CA certificate to the "os" inside my container. After that, everything I ran in my container just worked.
The catch with step 3 here is that exactly how you do this is different for different flavors of linux. I don't know much about alpine, but these links might get you pointed in roughly the right direction:
https://blog.confirm.ch/adding-a-new-trusted-certificate-authority/
https://github.com/gliderlabs/docker-alpine/issues/260
Also, bonus catch - if you use python's requests library in your application, it doesn't use the system CA certs by default. If this is a problem for you, read about setting the REQUESTS_CA_BUNDLE in the accepted answer here: Python Requests - How to use system ca-certificates (debian/ubuntu)?
I was not able to solve this for days, so if you can give me any advice I would appreciate it!
When I try to install any package from python I get this message (in this case lets take flask for example):
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',)': /simple/flask/
Could not find a version that satisfies the requirement Flask (from versions: )
No matching distribution found for Flask "
I do not really understand what caused it because previously everything worked well and I was able to install packages. And, I did not install anything new in between. Also, I am really a beginner in python programming and I do not fully understand what exactly the message is trying to say. I definitely have an internet connection and I have installed e.g. Flask previously and have used it as well. The only thing that happened before everything got stack was that my internet provider has "rebooted" my internet connection (I am not sure that "reboot" is the right termini for that).
I am using Windows subsystem for Linux, Visual studio code and python version: Python 3.6.8 (default, Oct 7 2019, 12:59:55) [GCC 8.3.0] on Linux.
This sounds like a DNS issue. Pip would be trying to resolve https://pypi.org. But ultimately that has to resolve to an IP Address.
Most ISPs provide a DNS service by default. Perhaps it wasn't able to resolve it?
When you added nameserver 8.8.8.8 you're using a Google DNS server which, it seems worked as expected by resolving https://pypi.org
I solved it by adding:
nameserver 8.8.8.8
nameserver 8.8.4.4
to the file:
/etc/resolv.conf
(via $sudo nano /etc/resolv.conf and save)
BUT I still do not understand fully why it worked? and Where came the problem from?
I'm trying to use Python 3.7 and PIP (on Windows 7) behind corporate firewall but I'm getting following error:
Retrying (Retry(total=4, connect=None, read=None, redirect=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed:
407 Proxy Authentication Required ( Forefront TMG requires authorization to fulfill the request. Access to the Web Proxy filter is denied. )',))': /simple/pywin32/
I've tried following configurations to pass proxy access and anyone of these is not working :(
set HTTP_PROXY=http://DOMAIN\user:pass#proxyhost:port
set HTTP_PROXY=http://user:pass#proxyhost:port
set HTTP_PROXY=http://proxyhost:port
I had a problem with my organization proxy auth too and was getting 407 errors in Windows OS, finally solved it using CTNLM , so first try to config ctnlm for your proxy and test it with your browser (the config is located in %PROGRAMFILES%\Cntlm\cntlm.ini for windows and default proxy should be something like http://localhost:3128) and if it worked give it to pip install : --proxy http://localhost:3128