I am new to setuptools on python.
I added a package 'numpy' and 'tensorflow' to install_requires list and running python setup.py install. It does not install due to SSL issue. We use self signed SSL for https based urls.
In case of pip for individual packages I can use --cert option. As I know setuptools use pip indirectly. If so, is there anyway to force it to use --cert option when setup.py is ran?
It turns out that we have actually two problems here.
The problem about tensorflow is related some difference between egg and wheel packages.
To solve the problem about numpy and other packages it is enough to add the following line into ~/.pip/pip.conf file:
[global]
cert=/path/to/cert.crt
If necessarry you may add any other option values (including proxy) with same format.
I am new to coding. While i try to install python-forecastio using pip3 for python3,it gives me below error.
$ sudo pip3 install --trusted-host pypi.python.org python-forecastio
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Collecting python-forecastio
Could not fetch URL https://pypi.python.org/simple/python-forecastio/: There was a problem confirming the ssl certificate: Can't connect to HTTPS URL because the SSL module is not available. - skipping
Could not find a version that satisfies the requirement python-forecastio (from versions: )
No matching distribution found for python-forecastio
What is wrong is, as you may have guessed, the SSL module is not available.
So the simple way to fix the problem is to reinstall python. But the alternative is to run this simple line of code: import ssl. If you get an error you will know your python 3 installation is corrupt so you need to reinstall python, or if you don't get an error try using easy_install-3.x (what ever the module or link you want)(Use the specific python you have installed version in place of "x"). Hope I helped!
-Zeus
My python package footools needs html5lib via install_requires in setup.py.
setup.py develop fails
Installing via setup.py develop fails:
cd src/footools/
python setup.py develop
Processing dependencies for footools==2016.205
Searching for html5lib==0.9999999
Reading https://source.example.com/pypi/simple/html5lib/
Download error on https://source.example.com/pypi/simple/html5lib/:
[Errno 185090050] _ssl.c:354: error:0B084002:x509
certificate routines:X509_load_cert_crl_file:system lib --
Some packages may not be found!
Couldn't find index page for 'html5lib' (maybe misspelled?)
pip works
But direct download works:
bar#workdevel123:~/src/footools> pip install html5lib==0.9999999
/home/bar/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:79:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately
and may cause certain SSL connections to fail.
For more information, see
https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Collecting html5lib==0.9999999
/home/bar/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:79:
InsecurePlatformWarning: A true SSLContext object is not available.
This prevents urllib3 from configuring SSL appropriately and
may cause certain SSL connections to fail.
For more information,
see https://urllib3.readthedocs.org/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading https://source.example.com/pypi/packages/html5lib-0.9999999.tar.gz
Requirement already satisfied (use --upgrade to upgrade):
six in /usr/lib/python2.7/site-packages (from html5lib==0.9999999)
Installing collected packages: html5lib
Running setup.py install for html5lib
Successfully installed html5lib-0.9999999
Questions
What is the difference between these two methods?
Why are they different?
What is the correct way to install a dependency in python?
setup.py
The setup.py is not special:
import setuptools
setuptools.setup(
name='foo',
version='2016.210',
long_description=open('README.txt').read(),
packages=setuptools.find_packages(),
install_requires=[
# about twenty packages before this line
'html5lib==0.9999999'
],
include_package_data=True,
entry_points={
'console_scripts': [
'foo=foo.utils.bar:main',
],
},
)
python setup.py develop, or setuptools in general uses easy_install to satisfy dependencies which in turn uses urllib2 whereas pip uses requests. See here for easy_install vs pip.
pip is more modern and among other things has capability to uninstall packages and complies with PEP 438 -- Transitioning to release-file hosting on PyPI. You can achieve the same thing as python setup.py develop with pip install -e src/footools/, note if the project path is in the current directory use, ./footools.
The requests package bundles CA certs in the package itself, python -c 'import pip;print(pip.download.requests.certs.where())'.
setuptools uses system installed CA certs python -c 'from setuptools import ssl_support;print(ssl_support.cert_paths)'.
You have to update system installed CA certs using tools like update-ca-certificates for Ubuntu to either update CA certs automatically or download from https://curl.haxx.se/docs/caextract.html and install into one of the paths shown by setuptools or set setuptools.ssl_support.cert_paths to an empty sequence like [] and do pip install certifi.
Calling setuptools.ssl_support.find_ca_bundle() will reveal the location of CA certs.
setuptools is a collection of enhancements to the Python distutils (for Python 2.6 and up) that allow developers to more easily build and distribute Python packages, especially ones that have dependencies on other packages.
So, among other stuff, you can create packages that can be upload to Pypi, and later installed using pip (therefore distibuting your module).
That said, they actually should not be that different in the installation part. You are running the develop mode, so maybe you have to fiddle a bit with the directories or fix the authorization error.
In the Development Mode the project is deployed into a staging area (in some way similar to the process of a virtual environment)
deployment is done in such a way that changes to the project source are immediately available in the staging area(s), without needing to run a build or install step after each change.
Meaning also everything will be available for that python interpreter. It can be later unstaged.
I noticed that html5lib is being fetched from different places: /pypi/simple/in one case and /pypi/packages/ in the other.
dependency_links
A list of strings naming URLs to be searched when satisfying dependencies. These links will be used if needed to install packages specified by setup_requires or tests_require.
Back to the problem I think it is most probably the ssl issue, since in pip it handled graciously (i.e., nice warning and there is some kind of workaround), but the same does not happen with setuptools. If there is an error in the request that is not handled then Couldn't find index page for 'html5lib' could be because of that.
What is the difference between these two methods?
Nothing terribly important to you as a user, aside from their different user interfaces. They are two stops on the scenic historical train ride of Python package management. There were others along the way.
Why are they different?
Back in the day, Python didn't ship with a package management system. Third party solutions filled the void. They were designed by different people at different moments in time. If you look at other programming languages, you sometimes see similar stories; sometimes you see happier stories; sometimes more tragic.
What is the correct way to install a dependency in python?
Both of these methods are technically correct. Pip is the more modern method, and in my experience it is both more popular and handier to work with. As of Python 3.4 and up, Pip has been included in the CPython distribution and is officially 'preferred'. So you can see which way the wind is blowing.
I was trying to install a few packages in a virtual environment using pip. I was getting the error:
Downloading/unpacking pymongo
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement pymongo
No distributions at all found for pymongo
I found that some say a workaround is downgrading your pip. I downloaded and installed pip 1.2.1 Now I get the error:
(env1)PS C:\dev\virtualenvs> pip install bottle
Downloading/unpacking bottle
Cannot fetch index base URL http://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement bottle
No distributions at all found for bottle
The log says:
urlopen error [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions>
How do I correctly use virtualenvs on windows?
EDIT: I am able to use pip when not in my virtualenv.
In this case the operating system should not matter. Also this should not be a question about virtualenv (as suggested in your title). pip just cannot access the remote end that it would like to access. Does it take some time for the error message to appear (is this a timeout issue)? If yes, then this is a networking issue, maybe triggered by a firewall. You should check this first.
According to http://treq.readthedocs.org/en/latest/, treq can be installed using pip, however it is failing. I can install treq using the tar file, but I am wondering why this is failing since http://pypi.python.org/simple/treq/ has 0.1.0 and 0.2.0.
$ sudo pip install treq
Downloading/unpacking treq
Cannot fetch index base URL http://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement treq
No distributions at all found for treq
Storing complete log in <cut>.pip/pip.log
$ pip search treq
megrok.strictrequire - Checks that all grokked "view-like" components
require a permission.
streql - Constant-time string comparison
trequests - A Tornado async HTTP/HTTPS client adaptor for
python-requests
treq - A requests-like API built on top of twisted.web's
Agent
repoze.bfg.restrequest - a REST aware Request for implementing RESTful
applications with repoze.bfg
$
Just tried to install from tarfile and it can't find twisted either so wondering if there's basic issue with pip...
$ sudo python setup.py install
running install
Checking .pth file support in /usr/local/lib/python2.7/dist-packages/
<snip>
Extracting treq-0.2.0-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Adding treq 0.2.0 to easy-install.pth file
Installed /usr/local/lib/python2.7/dist-packages/treq-0.2.0-py2.7.egg
Processing dependencies for treq==0.2.0
Searching for Twisted>=12.1.0
Reading http://pypi.python.org/simple/Twisted/
Download error on http://pypi.python.org/simple/Twisted/: timed out -- Some packages may not be found!
Reading http://pypi.python.org/simple/Twisted/
Download error on http://pypi.python.org/simple/Twisted/: timed out -- Some packages may not be found!
Couldn't find index page for 'Twisted' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading http://pypi.python.org/simple/
Download error on http://pypi.python.org/simple/: timed out -- Some packages may not be found!
No local packages or download links found for Twisted>=12.1.0
error: Could not find suitable distribution for Requirement.parse('Twisted>=12.1.0')
$
The error message reads:
Download error on http://pypi.python.org/simple/Twisted/: timed out -- Some packages may not be found!
Did you check for a firewall blocking access to PyPI?