We Keep Coding

issue in installing packages using pip install

I need to install pyspark. It has a dependency on pypandoc. So I first do pip install pypandoc and then pip install pyspark and everything looks fine. However, based on some requirements I need to install my dependencies using a requirements.txt file. So I put both pypandoc and pyspark in the requirements.txt file and then I do pip install -r requirements.txt (pypandoc comes first in the file followed by pyspark), however this time the installation file with the following error
Complete output from command python setup.py egg_info:
Could not import pypandoc - required to package PySpark
Download error on https://pypi.org/simple/pypandoc/: [Errno 97] Address family not supported by protocol -- Some packages may not be found!
Couldn't find index page for 'pypandoc' (maybe misspelled?)
Download error on https://pypi.org/simple/: [Errno 97] Address family not supported by protocol -- Some packages may not be found!
No local packages or working download links found for pypandoc
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/tmp/pip-install-6vmbjchu/pyspark/setup.py", line 224, in <module>
'Programming Language :: Python :: Implementation :: PyPy']
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 144, in setup
_install_setup_requires(attrs)
File "/usr/local/lib/python3.6/site-packages/setuptools/__init__.py", line 139, in _install_setup_requires
dist.fetch_build_eggs(dist.setup_requires)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 724, in fetch_build_eggs
replace_conflicting=True,
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 782, in resolve
replace_conflicting=replace_conflicting
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1065, in best_match
return self.obtain(req, installer)
File "/usr/local/lib/python3.6/site-packages/pkg_resources/__init__.py", line 1077, in obtain
return installer(requirement)
File "/usr/local/lib/python3.6/site-packages/setuptools/dist.py", line 791, in fetch_build_egg
return cmd.easy_install(req)
File "/usr/local/lib/python3.6/site-packages/setuptools/command/easy_install.py", line 673, in easy_install
raise DistutilsError(msg)
distutils.errors.DistutilsError: Could not find suitable distribution for Requirement.parse('pypandoc')
So it looks like when I do it in this way, the pypandoc is not properly installed when it tries to install pyspark. How can I fix this issue?

Issues installing python package Six (to install Pip)

We recently uninstalled pip to do some cleanup on Mac OS X El Capitan. Now trying to re-install pip.
$ sudo easy_install pip
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 11, in <module>
sys.exit(main())
File "/Library/Python/2.7/site-packages/setuptools/command/easy_install.py", line 2270, in main
**kw
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 321, in __init__
_Distribution.__init__(self, attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (six 1.4.1 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('six>=1.6.0'))
Looks like we need to upgrade Six. So:
$ easy_install --upgrade six
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 11, in <module>
sys.exit(main())
File "/Library/Python/2.7/site-packages/setuptools/command/easy_install.py", line 2270, in main
**kw
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 321, in __init__
_Distribution.__init__(self, attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (six 1.4.1 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('six>=1.6.0'))
Now it looks like we need to upgrade Six in order to upgrade Six??? Maybe it's just a permissions issue:
$sudo easy_install --upgrade six
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 11, in <module>
sys.exit(main())
File "/Library/Python/2.7/site-packages/setuptools/command/easy_install.py", line 2270, in main
**kw
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/core.py", line 111, in setup
_setup_distribution = dist = klass(attrs)
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 321, in __init__
_Distribution.__init__(self, attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (six 1.4.1 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('six>=1.6.0'))
Nope. Same error.
I'm obviously missing something. Can someone shed some light on this?
Tried the first answer:
$ python get-pip.py
Collecting pip
Using cached pip-9.0.1-py2.py3-none-any.whl
Collecting wheel
Using cached wheel-0.29.0-py2.py3-none-any.whl
Installing collected packages: pip, wheel
Exception:
Traceback (most recent call last):
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/req/req_set.py", line 784, in install
**kwargs
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/wheel.py", line 247, in move_wheel_files
prefix=prefix,
File "/var/folders/23/49gg72xd4wb1qps4z5j9vbz80000gy/T/tmpz5ckOD/pip.zip/pip/locations.py", line 140, in distutils_scheme
d = Distribution(dist_args)
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 321, in __init__
_Distribution.__init__(self, attrs)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py", line 287, in __init__
self.finalize_options()
File "/Library/Python/2.7/site-packages/setuptools/dist.py", line 389, in finalize_options
ep.require(installer=self.fetch_build_egg)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 2324, in require
items = working_set.resolve(reqs, env, installer, extras=self.extras)
File "/Library/Python/2.7/site-packages/pkg_resources/__init__.py", line 859, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
VersionConflict: (six 1.4.1 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python), Requirement.parse('six>=1.6.0'))

Python 2.7.9+ and 3.4+
Good news! Python 3.4 (released March 2014) and Python 2.7.9 (released December 2014) ship with Pip. This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup. In shipping with a package manager, Python joins Ruby, Node.js, Haskell, Perl, Go--almost every other contemporary language with a majority open-source community. Thank you Python.
Of course, that doesn't mean Python packaging is problem solved. The experience remains frustrating. I discuss this in Stack Overflow question Does Python have a package/module management system?.
And, alas for everyone using Python 2.7.8 or earlier (a sizable portion of the community). There's no plan to ship Pip to you. Manual instructions follow.
Python 2 ≤ 2.7.8 and Python 3 ≤ 3.3
Flying in the face of its 'batteries included' motto, Python ships without a package manager. To make matters worse, Pip was--until recently--ironically difficult to install.
Official instructions
Per https://pip.pypa.io/en/stable/installing/#do-i-need-to-install-pip:
Download get-pip.py, being careful to save it as a .py file rather than .txt. Then, run it from the command prompt:
python get-pip.py
You possibly need an administrator command prompt to do this. Follow Start a Command Prompt as an Administrator (Microsoft TechNet).
Alternative instructions
The official documentation tells users to install Pip and each of its dependencies from source. That's tedious for the experienced, and prohibitively difficult for newbies.
For our sake, Christoph Gohlke prepares Windows installers (.msi) for popular Python packages. He builds installers for all Python versions, both 32 and 64 bit. You need to
Install setuptools
Install pip
For me, this installed Pip at C:\Python27\Scripts\pip.exe. Find pip.exe on your computer, then add its folder (for example, C:\Python27\Scripts) to your path (Start / Edit environment variables). Now you should be able to run pip from the command line. Try installing a package:
pip install httpie
There you go (hopefully)! Solutions for common problems are given below:
Proxy problems
If you work in an office, you might be behind a HTTP proxy. If so, set the environment variables http_proxy and https_proxy. Most Python applications (and other free software) respect these. Example syntax:
http://proxy_url:port
http://username:password#proxy_url:port
If you're really unlucky, your proxy might be a Microsoft NTLM proxy. Free software can't cope. The only solution is to install a free software friendly proxy that forwards to the nasty proxy. http://cntlm.sourceforge.net/
Unable to find vcvarsall.bat
Python modules can be part written in C or C++. Pip tries to compile from source. If you don't have a C/C++ compiler installed and configured, you'll see this cryptic error message.
Error: Unable to find vcvarsall.bat
You can fix that by installing a C++ compiler such as MinGW or Visual C++. Microsoft actually ship one specifically for use with Python. Or try Microsoft Visual C++ Compiler for Python 2.7.
Often though it's easier to check Christoph's site for your package.

TyperError installing jupyter

I'm trying to install jupyter on Ubuntu 14.04 LTS but I keep getting this error. I tried giving permissions to every folder and run the code as:
sudo -H pip3 install --user jupyter
But nothing worked. I also did the update on pip and it's running the latest version.
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.4/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python3.4/dist-packages/pip/commands/install.py", line 335, in run
wb.build(autobuilding=True)
File "/usr/local/lib/python3.4/dist-packages/pip/wheel.py", line 749, in build
self.requirement_set.prepare_files(self.finder)
File "/usr/local/lib/python3.4/dist-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/usr/local/lib/python3.4/dist-packages/pip/req/req_set.py", line 634, in _prepare_file
abstract_dist.prep_for_dist()
File "/usr/local/lib/python3.4/dist-packages/pip/req/req_set.py", line 129, in prep_for_dist
self.req_to_install.run_egg_info()
File "/usr/local/lib/python3.4/dist-packages/pip/req/req_install.py", line 412, in run_egg_info
self.setup_py, self.name,
File "/usr/local/lib/python3.4/dist-packages/pip/req/req_install.py", line 387, in setup_py
import setuptools # noqa
File "/root/.local/lib/python3.4/site-packages/setuptools/__init__.py", line 12, in <module>
import setuptools.version
File "/root/.local/lib/python3.4/site-packages/setuptools/version.py", line 1, in <module>
import pkg_resources
File "/root/.local/lib/python3.4/site-packages/pkg_resources/__init__.py", line 72, in <module>
import packaging.requirements
File "/root/.local/lib/python3.4/site-packages/packaging/requirements.py", line 59, in <module>
MARKER_EXPR = originalTextFor(MARKER_EXPR())("marker")
TypeError: __call__() missing 1 required positional argument: 'name'
Thanks for the help!

For me it works with the recent pyparsing package. Perhaps you need to upgrade your pyparsing to the recent version. If you installed it with apt-get, you need to remove the python3-pyparsing package, and install it with pip3.
sudo apt remove python3-pyparsing
sudo pip3 install pyparsing
As this answer writes, you need pyparsing>=2.0.2 to avoid this error.

I had this error as well, after a bit of digging I found this question that addresses the same error caused by some other software.
As the author states, the problem is that the call to MARKER_EXPR() should have a 'name' argument but it doesn't.
To fix this, edit line 59 of /usr/local/lib/python3.4/dist-packages/packaging/requirements.py and just add quotes to the call to MARKER_EXPR() like so:
MARKER_EXPR = originalTextFor(MARKER_EXPR(""))("marker")

Python Pip Install "Egg" Error (Pandas and other modules)

Having issues installing some python modules (for example pandas) and also had some issues installing ipython notebook.
This error seems to have come up before for other folks (Python pip install fails: invalid command egg_info), but they were using Python 2.7, whereas I'm using 2.6 (default with my Mac OS 10.6). Also I tried easy_install -U setuptools and pip install --upgrade setuptools but neither worked.
This is the error I get for installing pandas:
Command python setup.py egg_info failed with error code 1 in /private/var/folders/Wm/WmmbYincEnuCrAMtGBudAk+++TM/-Tmp-/pip_build_AWal/pandas
Storing debug log for failure in /Users/AWal/Library/Logs/pip.log
For installing iPython, this is the error that I get:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/basecommand.py", line 122, in main
status = self.run(options, args)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/commands/install.py", line 283, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/req.py", line 1435, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/req.py", line 671, in install
self.move_wheel_files(self.source_dir, root=root)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/req.py", line 901, in move_wheel_files
pycompile=self.pycompile,
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/wheel.py", line 341, in move_wheel_files
generated.extend(maker.make_multiple(['%s = %s' % kv for kv in console.items()]))
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/_vendor/distlib/scripts.py", line 316, in make_multiple
filenames.extend(self.make(specification, options))
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/_vendor/distlib/scripts.py", line 305, in make
self._make_script(entry, filenames, options=options)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/_vendor/distlib/scripts.py", line 209, in _make_script
self._write_script(scriptnames, shebang, script, filenames, ext)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/_vendor/distlib/scripts.py", line 189, in _write_script
self._fileop.write_binary_file(outname, script_bytes)
File "/Library/Python/2.6/site-packages/pip-1.5.4-py2.6.egg/pip/_vendor/distlib/util.py", line 384, in write_binary_file
with open(path, 'wb') as f:
IOError: [Errno 13] Permission denied: '/usr/local/bin/sphinx-apidoc'
Storing debug log for failure in /Users/AWal/Library/Logs/pip.log
Any thoughts on what to do?
UPDATE:
When I do sudo pip install pandas (another module that I can't install), I get a long long traceback with the end error message:
RuntimeError: Broken toolchain: cannot link a simple C program
And traceback (a sample of it of the 100 or so lines) looks like this:
Traceback (most recent call last):
File "<string>", line 17, in <module>
File "/private/tmp/pip_build_root/pandas/setup.py", line 619, in <module>
**setuptools_kwargs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 113, in setup
File "build/bdist.macosx-10.6-universal/egg/setuptools/dist.py", line 266, in __init__
File "build/bdist.macosx-10.6-universal/egg/setuptools/dist.py", line 312, in fetch_build_eggs
File "build/bdist.macosx-10.6-universal/egg/pkg_resources/__init__.py", line 753, in resolve
File "build/bdist.macosx-10.6-universal/egg/pkg_resources/__init__.py", line 1005, in best_match
File "build/bdist.macosx-10.6-universal/egg/pkg_resources/__init__.py", line 1017, in obtain
File "build/bdist.macosx-10.6-universal/egg/setuptools/dist.py", line 379, in fetch_build_egg
File "build/bdist.macosx-10.6-universal/egg/setuptools/command/easy_install.py", line 619, in easy_install

A permission Denied message would suggest you to run your command as root as suggested by Hackaholic.
try sudo pip install YOUR_PACKAGE
You might also want to have a look at virtualenv to use a more isolated environment.
Under debian based distributions, you might install python-virtualenv package with apt-get install python-virtualenv.
Then create a virtual environment names whatever by typing virtualenv whatever (this will create a folder whatever inside your current folder.
Then cd to it and type bin/activate to enter this virtual environment.
From there try again your pip install command.
Might help
UPDATE :
You might try to use the suggested solutions available here
To sumup, try to add export ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future inside ~/.bash_profile

This question comes first in Google, but the second one actually worked:
pip install -U setuptools
pip install ez_setup

virtualenvwrapper install failure - "what would cause ImportError: No module named core" etc?

I'm on osx 10.6.8, and trying to use bash to install virtualenvwrapper, and am getting back cryptic feedback (at least for me).
I was able to install virtualenv.
Perhaps someone can point me in the right direction...
the output from the failed install:
Downloading/unpacking virtualenvwrapper
Running setup.py egg_info for package virtualenvwrapper
Installed /private/tmp/pip_build_root/virtualenvwrapper/pbr-0.5.23-py2.6.egg
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py:245: UserWarning: Module pbr was already imported from /tmp/easy_install-xuBV6G/pbr-0.5.23/pbr/__init__.py, but /private/tmp/pip_build_root/virtualenvwrapper/pbr-0.5.23-py2.6.egg is being added to sys.path
working_set.add(dist)
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "/private/tmp/pip_build_root/virtualenvwrapper/setup.py", line 7, in <module>
pbr=True,
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 113, in setup
_setup_distribution = dist = klass(attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 223, in __init__
_Distribution.__init__(self,attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 270, in __init__
self.finalize_options()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 256, in finalize_options
ep.load()(self, ep.name, value)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 1907, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named core
Complete output from command python setup.py egg_info:
Installed /private/tmp/pip_build_root/virtualenvwrapper/pbr-0.5.23-py2.6.egg
/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py:245: UserWarning: Module pbr was already imported from /tmp/easy_install-xuBV6G/pbr-0.5.23/pbr/init.py, but /private/tmp/pip_build_root/virtualenvwrapper/pbr-0.5.23-py2.6.egg is being added to sys.path
working_set.add(dist)
Traceback (most recent call last):
File "", line 16, in
File "/private/tmp/pip_build_root/virtualenvwrapper/setup.py", line 7, in
pbr=True,
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/core.py", line 113, in setup
_setup_distribution = dist = klass(attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 223, in init
_Distribution.__init__(self,attrs)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/distutils/dist.py", line 270, in init
self.finalize_options()
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/setuptools/dist.py", line 256, in finalize_options
ep.load()(self, ep.name, value)
File "/System/Library/Frameworks/Python.framework/Versions/2.6/Extras/lib/python/pkg_resources.py", line 1907, in load
entry = __import__(self.module_name, globals(),globals(), ['__name__'])
ImportError: No module named core
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /private/tmp/pip_build_root/virtualenvwrapper
Storing complete log in /Users/admin/Library/Logs/pip.log

Updating pbr package using pip install -U pbr globally solved the import issue.