We Keep Coding

Pip is broken and I don't have admin permission

I have poured about 6 hours into this issue now and I'm getting farther and farther. Here's what happened.
I am using a shared Linux machine that has both Python 2.7 and Python 3.5 and I don't have admin privileges. I was using Python 3.5 and I wanted to use the package seaborn and I found out the specific labeling feature I wanted was only available in a newer version. So no problem "pip3 install seaborn --upgrade".
Everything was happening so fast but something went wrong and by the suggestion of the internet, I ended up upgrading a few other packages like matplotlib, setuptools, and eventually pip itself.
Here's where it really went wrong. Pip just broke. This happens when I run pip3 --version:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 33, in vendored
__import__(vendored_name, globals(), locals(), level=0)
ImportError: No module named 'pip._vendor.pkg_resources'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/bin/pip3", line 9, in <module>
from pip import main
File "/usr/lib/python3/dist-packages/pip/__init__.py", line 13, in <module>
from pip.exceptions import InstallationError, CommandError, PipError
File "/usr/lib/python3/dist-packages/pip/exceptions.py", line 6, in <module>
from pip._vendor.six import iteritems
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 75, in <module>
vendored("pkg_resources")
File "/usr/lib/python3/dist-packages/pip/_vendor/__init__.py", line 36, in vendored
__import__(modulename, globals(), locals(), level=0)
File "<frozen importlib._bootstrap>", line 969, in _find_and_load
File "<frozen importlib._bootstrap>", line 958, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 664, in _load_unlocked
File "<frozen importlib._bootstrap>", line 634, in _load_backward_compatible
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2927, in <module>
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2913, in _call_aside
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2952, in _initialize_master_working_set
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 956, in subscribe
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2952, in <lambda>
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2515, in activate
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2097, in declare_namespace
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2047, in _handle_ns
File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/__init__.py", line 2066, in _rebuild_mod_path
AttributeError: '_NamespacePath' object has no attribute 'sort'
The first line in the call stack is "from pip import main" so I tried running pip --version instead of pip3 and now I get:
Traceback (most recent call last):
File "/usr/bin/pip", line 9, in <module>
from pip import main
ImportError: cannot import name main
This error is all over the internet as a bug in pip itself but all every solution involves something that I don't have permission to do. I tried to uninstall and reinstall an older version of pip using "python -m pip ..." but this did not help with the pip3 error, which prompted the same error a few lines later. And I can't use this shortcut with pip3 (python3 -m pip...) because this error occurs anyway.
Worst part is that my original program is now completely dead. I can't even import seaborn at all without crashing.
Traceback (most recent call last):
File "/u/hailey/.local/lib/python3.5/site-packages/matplotlib/__init__.py", line 861, in __setitem__
cval = self.validate[key](val)
KeyError: 'axes.color_cycle'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "nn.py", line 9, in <module>
import seaborn as sns
File "/usr/lib/python3/dist-packages/seaborn/__init__.py", line 13, in <module>
set()
File "/usr/lib/python3/dist-packages/seaborn/rcmod.py", line 98, in set
set_palette(palette, color_codes=color_codes)
File "/usr/lib/python3/dist-packages/seaborn/rcmod.py", line 491, in set_palette
mpl.rcParams["axes.color_cycle"] = list(colors)
File "/u/hailey/.local/lib/python3.5/site-packages/matplotlib/__init__.py", line 868, in __setitem__
'list of valid parameters.' % (key,))
KeyError: 'axes.color_cycle is not a valid rc parameter. See rcParams.keys() for a list of valid parameters.'
At this point, I decided everything would be easier if I just downloaded my own clean version of Python in a local directory where I have privileges. So I downloaded and set up a new version of Python 3.6 in a local directory. Then I ran my original program with this and I was told I had no module named 'numpy'. Reasonable. Let's try to install numpy.
Obtaining file:///u/hailey/python36/Python-3.6.0
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/u/hailey/python36/Python-3.6.0/setup.py", line 4, in <module>
import sys, os, importlib.machinery, re, optparse
ImportError: No module named machinery
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /u/hailey/python36/Python-3.6.0/
Now I'm led to a page that recommends running "setup.py install", which sounds simple enough. But nope.
bash: /u/hailey/python36/Python-3.6.0/setup.py: Permission denied
Now I'm giving up because evidently I know nothing about Python. I would really appreciate some help with this. At this point, even just recovering the state I had yesterday would be fine as I can live without the extra label on my seaborn graph. Thanks everyone!

Python Eclipse PyDev

I am a complete newbie to Python.
I have downloaded and installed Anaconda 3.5.2 on Windows 7, then Eclipse Photon. Installed the PyDev Plugin.
I was able to run a few simple test programs in Eclipse.
Suddenly, without installing anything I get
Error processing line 1 of C:\Anaconda3\lib\site-packages\matplotlib-2.2.2-py3.6-nspkg.pth:
Failed to import the site module
Traceback (most recent call last):
File "C:\Anaconda3\lib\site.py", line 168, in addpackage
exec(line)
File "<string>", line 1, in <module>
File "C:\Anaconda3\lib\types.py", line 171, in <module>
import functools as _functools
File "C:\Anaconda3\lib\functools.py", line 21, in <module>
from collections import namedtuple
ImportError: cannot import name 'namedtuple'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Anaconda3\lib\site.py", line 541, in <module>
main()
File "C:\Anaconda3\lib\site.py", line 523, in main
known_paths = addsitepackages(known_paths)
File "C:\Anaconda3\lib\site.py", line 320, in addsitepackages
addsitedir(sitedir, known_paths)
File "C:\Anaconda3\lib\site.py", line 207, in addsitedir
addpackage(sitedir, name, known_paths)
File "C:\Anaconda3\lib\site.py", line 178, in addpackage
import traceback
File "C:\Anaconda3\lib\traceback.py", line 5, in <module>
import linecache
File "C:\Anaconda3\lib\linecache.py", line 8, in <module>
import functools
File "C:\Anaconda3\lib\functools.py", line 21, in <module>
from collections import namedtuple
ImportError: cannot import name 'namedtuple'
even with a "Hello World" which ran before
Any ideas what could cause this?

I have found the reason:
I have added a "PyDev Package" named "collections" in the IDE. This collided with something ....

You can try a few things. Have you tried updating the package in conda? It is possible it is not up to date. Also, with Eclipse (and with most IDEs) if things that were working a moment break without any changes it is a sign you should quit the program, give it a moment to settle, and try reopening and rerunning your code. Sometimes it is the tool's fault!

How to install packages through pip? [duplicate]

This question already has answers here:
Python: pip is installed but not working in windows
(2 answers)
Closed 5 years ago.
I am trying to install a python gui automator and am supposed to do so through pip. Typing pip install pyautogui in the command prompt (as administrator) gives me many exceptions. Here is the printout:
pip install pyautogui
Traceback (most recent call last):
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\_vendor\requests\packages\__init__.py", line 27, in <module>
from . import urllib3
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\_vendor\requests\packages\urllib3\__init__.py", line 8, in <module>
from .connectionpool import (
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\_vendor\requests\packages\urllib3\connectionpool.py", line 7, in <module>
from socket import error as SocketError, timeout as SocketTimeout
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\socket.py", line 49, in <module>
import _socket
zipimport.ZipImportError: can't find module '_socket'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "C:\Users\Nate4\AppData\Local\Programs\Python\Python36-32\Scripts\pip.exe\__main__.py", line 5, in <module>
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\__init__.py", line 21, in <module>
from pip._vendor.requests.packages.urllib3.exceptions import DependencyWarning
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\_vendor\requests\__init__.py", line 62, in <module>
from .packages.urllib3.exceptions import DependencyWarning
File "c:\users\nate4\appdata\local\programs\python\python36-32\lib\site-packages\pip\_vendor\requests\packages\__init__.py", line 29, in <module>
import urllib3
ModuleNotFoundError: No module named 'urllib3'

1) Go to the directory where your python is installed. (In my case it's C:\Users...\AppData\Local\Programs\Python\Python36-32)
2) Find the directory called "Scripts"
3) Hold down SHIFT and right-click on that folder. Then click "Open command window here"
4) A command prompt window will pop up. Type pip install <packagename> and press ENTER.
This method works for me when I install packages for my python through pip. Hope it helps you.

Using PYMC3 on Windows 10 - theano cannot import name 'floatX'

I'm struggling to get PYMC3 to install correctly on windows. I've tried using the Anaconda package via conda install -c conda-forge pymc3 and in a virtualenv using only pip as per the documentation.
It seems to install ok, but fails when running an import pymc3 with the following error. Research suggests that there may be some dependencies which are getting missed.
>>> import pymc3
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
ERROR (theano.sandbox.cuda): nvcc compiler not found on $PATH. Check your nvcc installation and try again.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\__init__.py", line 8, in <module>
from .sampling import *
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\sampling.py", line 11, in <module>
from .step_methods import (NUTS, HamiltonianMC, Metropolis, BinaryMetropolis,
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\step_methods\__init__.py", line 3, in <module>
from .hmc import HamiltonianMC
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\step_methods\hmc\__init__.py", line 1, in <module>
from .hmc import HamiltonianMC
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\step_methods\hmc\hmc.py", line 7, in <module>
from .base_hmc import BaseHMC
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\step_methods\hmc\base_hmc.py", line 2, in <module>
from .trajectory import get_theano_hamiltonian_functions
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\step_methods\hmc\trajectory.py", line 3, in <module>
from pymc3.theanof import join_nonshared_inputs, gradient, CallableTensor, floatX
ImportError: cannot import name 'floatX'
I can't find much out there about this floatx package.
Seems to be a windows thing - as it installed and runs no problems on my old Ubuntu box (with anaconda).
Any help gratefully received!
EDIT: Step closer. For some reason I had the path to the nvcc compiler wrong. So if you suffer the same check that you have the full path including "\bin" in your environmnental path. If you make a change don't forget to restart.
Now I get an error:
>>> import pymc3
WARNING (theano.sandbox.cuda): The cuda backend is deprecated and will be removed in the next release (v0.10). Please switch to the gpuarray backend. You can get more information about how to switch at this URL:
https://github.com/Theano/Theano/wiki/Converting-to-the-new-gpu-back-end%28gpuarray%29
Using gpu device 0: GeForce GTX 960M (CNMeM is disabled, cuDNN not available)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\__init__.py", line 4, in <module>
from .distributions import *
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\distributions\__init__.py", line 1, in <module>
from . import timeseries
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\distributions\timeseries.py", line 1, in <module>
import theano.tensor as tt
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\__init__.py", line 116, in <module>
theano.sandbox.cuda.tests.test_driver.test_nvidia_driver1()
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\sandbox\cuda\tests\test_driver.py", line 32, in test_nvidia_driver1
profile=False)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\compile\function.py", line 326, in function
output_keys=output_keys)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\compile\pfunc.py", line 486, in pfunc
output_keys=output_keys)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 1795, in orig_function
defaults)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\compile\function_module.py", line 1661, in create
input_storage=input_storage_lists, storage_map=storage_map)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\link.py", line 699, in make_thunk
storage_map=storage_map)[:3]
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\vm.py", line 1098, in make_all
self.updated_vars,
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\vm.py", line 952, in make_vm
vm = CVM(
NameError: name 'CVM' is not defined
>>>
This I fixed with the last post here: https://groups.google.com/forum/#!msg/theano-users/JoTu61_MTLk/4ZzsVyaOf2kJ. But now stuck with an error:
>>> import pymc3
Traceback (most recent call last):
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cutils.py", line 306, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: No module named 'cutils_ext.cutils_ext'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cutils.py", line 317, in <module>
from cutils_ext.cutils_ext import * # noqa
ImportError: No module named 'cutils_ext.cutils_ext'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\__init__.py", line 4, in <module>
from .distributions import *
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\distributions\__init__.py", line 1, in <module>
from . import timeseries
File "C:\Users\peter\Anaconda3\lib\site-packages\pymc3\distributions\timeseries.py", line 1, in <module>
import theano.tensor as tt
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\__init__.py", line 80, in <module>
from theano.scan_module import (scan, map, reduce, foldl, foldr, clone,
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\scan_module\__init__.py", line 41, in <module>
from theano.scan_module import scan_opt
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\scan_module\scan_opt.py", line 60, in <module>
from theano import tensor, scalar
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\tensor\__init__.py", line 9, in <module>
from theano.tensor.subtensor import *
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\tensor\subtensor.py", line 26, in <module>
import theano.gof.cutils # needed to import cutils_ext
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cutils.py", line 320, in <module>
compile_cutils()
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cutils.py", line 285, in compile_cutils
preargs=args)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cmodule.py", line 2313, in compile_str
return dlimport(lib_filename)
File "C:\Users\peter\Anaconda3\lib\site-packages\theano\gof\cmodule.py", line 302, in dlimport
rval = __import__(module_name, {}, {}, [module_name])
ImportError: DLL load failed: The specified procedure could not be found.
>>>
The investigation continues! Maybe it's time to bin windows and move to linux full time!
EDIT2: I switched to a python 2.7 environment and it seems to work ok. Had to install http://mingw.org/ but after that all ok.
Would be nice to get it to work in 3.6 though. Thoughts anyone?

Ok, I managed to cobble an install together.
I had no luck with the dependency management etc using conda install -c conda-forge pymc3, and I couldn't be sure if there were issues with locations/paths to compilers etc. So I did a re-install from the start.
I went for Anaconda3 (rather than 2) and it worked fine.
1) I followed the environment setup here https://github.com/philferriere/dlwin until I got to the point of installing Theano. But instead of the version suggested I installed the latest with conda install -c rdonnelly Theano suggested by https://github.com/nouiz. This is a nice solid environment set up - and i really recommend the msys tool - I hadn't used it before.
The test script in the philferriere guide are a good way to test function/performance.
2) Then I set up pymc3 using pip install pymc3 as they suggest here https://github.com/pymc-devs/pymc3. but NOT the latest version on git.
Hope someone finds this useful.
Refs:
https://github.com/Theano/Theano/issues/5348

virtualenv on different Python version throws ImportError

I searched around and haven't seen this posted elsewhere.
My Windows machine uses Python 2.7 as the default Python. I ran virtualenv to create a virtual environment that uses Python 3.4:
C:\virtualenvs>virtualenv -p C:\Python34\python.exe mypy3
and got an error:
Running virtualenv with interpreter C:\Python34\python.exe
Traceback (most recent call last):
File "C:\Python27\lib\site-packages\virtualenv.py", line 8, in <module>
import base64
File "C:\Python34\lib\base64.py", line 9, in <module>
import re
File "C:\Python34\lib\re.py", line 324, in <module>
import copyreg
File "C:\Python27\lib\site-packages\copyreg\__init__.py", line 2, in <module>
from future.utils import PY3
File "C:\Python27\lib\site-packages\future\utils\__init__.py", line 75, in <mo
dule>
import functools
File "C:\Python34\lib\functools.py", line 21, in <module>
from collections import namedtuple
File "C:\Python34\lib\collections\__init__.py", line 17, in <module>
from reprlib import recursive_repr as _recursive_repr
File "C:\Python27\lib\site-packages\reprlib\__init__.py", line 8, in <module>
raise ImportError('Cannot import module from python-future source folder')
ImportError: Cannot import module from python-future source folder

I needed to specify the full path of virtualenv:
C:\virtualenvs>C:\python34\Scripts\virtualenv.exe -p C:\Python34\python.exe
because I was effectively calling this:
C:\virtualenvs>C:\python27\Scripts\virtualenv.exe -p C:\Python34\python.exe
since C:\python27\Scripts is in my PATH. And the collision of Python 2.7 and 3.4 was causing the issue.