We Keep Coding

Using Ubuntu Python packages and ImportError: No module named

How to install Ubuntu's packaged modules instead of using pip?
All Python packages tell us to do the packages installation via pip, but Ubuntu has its own packaging system. I'd rather stick to using the system default packaging system instead of cooking my own using pip, but is that a good idea?
Searching for "ImportError: No module named" you'll get all kinds of such error for all kinds of different Python modules. I'm wondering if there is a general solution to use Ubuntu system default packaging system instead.
Specifically, I installed python-jsonpath-rw with
sudo apt-get install python-jsonpath-rw
but when I tried to do
import sys, json, jsonpath
I get:
Traceback (most recent call last):
File "./pyjsonpath", line 2, in <module>
import sys, json, jsonpath
ImportError: No module named jsonpath
I then searched, and found
http://blog.ingensol.pl/2015/03/need-to-make-quick-json-fixes-jsonpath.html
and followed all its commands,
$ sudo apt-get install python-jsonpath-rw
$ sudo apt-get install python-setuptools
$ sudo easy_install -U jsonpath
but am still getting the exact same errors as before.
Please help. Thx.
$ apt-cache policy python-jsonpath-rw
python-jsonpath-rw:
Installed: 1.4.0-2
Candidate: 1.4.0-2
Version table:
*** 1.4.0-2 500
500 http://ca.archive.ubuntu.com/ubuntu zesty/main amd64 Packages
500 http://ca.archive.ubuntu.com/ubuntu zesty/main i386 Packages
100 /var/lib/dpkg/status
$ dpkg -L python-jsonpath-rw
/.
/usr
/usr/bin
/usr/bin/python2-jsonpath
/usr/lib
/usr/lib/python2.7
/usr/lib/python2.7/dist-packages
/usr/lib/python2.7/dist-packages/jsonpath_rw
/usr/lib/python2.7/dist-packages/jsonpath_rw/__init__.py
/usr/lib/python2.7/dist-packages/jsonpath_rw/bin
/usr/lib/python2.7/dist-packages/jsonpath_rw/bin/__init__.py
/usr/lib/python2.7/dist-packages/jsonpath_rw/bin/jsonpath.py
/usr/lib/python2.7/dist-packages/jsonpath_rw/jsonpath.py
/usr/lib/python2.7/dist-packages/jsonpath_rw/lexer.py
/usr/lib/python2.7/dist-packages/jsonpath_rw/parser.py
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info/PKG-INFO
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info/dependency_links.txt
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info/entry_points.txt
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info/requires.txt
/usr/lib/python2.7/dist-packages/jsonpath_rw-1.4.0.egg-info/top_level.txt
/usr/share
/usr/share/doc
/usr/share/doc/python-jsonpath-rw
/usr/share/doc/python-jsonpath-rw/changelog.Debian.gz
/usr/share/doc/python-jsonpath-rw/copyright
$ python -c "import sys, pprint; pprint.pprint(sys.path)"
['',
'/usr/lib/python2.7',
'/usr/lib/python2.7/plat-x86_64-linux-gnu',
'/usr/lib/python2.7/lib-tk',
'/usr/lib/python2.7/lib-old',
'/usr/lib/python2.7/lib-dynload',
'/usr/local/lib/python2.7/dist-packages',
'/usr/local/lib/python2.7/dist-packages/jsonpath-0.75-py2.7.egg',
'/usr/lib/python2.7/dist-packages']

Python3.6 ImportError: cannot import name 'main' Linux RHEL6

My ultimate goal is to download and install awscli
http://docs.aws.amazon.com/cli/latest/userguide/awscli-install-linux.html
Seems python and pip are required in order to accomplish my goal.
Installing python via yum isn't working for me, so I downloaded and installed python3 manually as follows:
wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz
tar xf Python-3.6.1.tar.xz
cd Python-3.6.1.tar.xz
./configure
make
sudo make altinstall
When I do
pip3.6 --version
I get this:
Traceback (most recent call last):
File "/usr/local/bin/pip3.6", line 7, in <module>
from pip import main
ImportError: cannot import name 'main'
Environment:
RHEL 6.6
Accessed via ssh
What is going on? How to install pip and python3.6 correctly on my linux box?
Edit:
When trying to install via yum it fails so I am trying to avoid using yum:
[asemani#rcdc9pfda5r Python-3.6.1]$ sudo yum -y install python-pip
[sudo] password for asemeani:
Loaded plugins: security
Setting up Install Process
No package python-pip available.
Error: Nothing to do
Another Edit:
After importing sys and then called sys.path this is what I get:
[asemani#rcdc9pfda5r Python-3.6.1]$ python3.6
Python 3.6.1 (default, Apr 28 2017, 11:03:27)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-17)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/lib/python36.zip', '/usr/local/lib/python3.6', '/usr/local/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/site-packages']
>>> quit()
[asemani#rcdc9pfda5r Python-3.6.1]$ ls -l /usr/local/lib/python3.6/site-packages
total 32
-rw------- 1 root root 126 Apr 27 23:45 easy_install.py
drwx------ 11 root root 4096 Apr 27 23:45 pip
drwx------ 2 root root 4096 Apr 27 23:45 pip-9.0.1.dist-info
drwx------ 5 root root 4096 Apr 27 23:45 pkg_resources
drwx------ 2 root root 4096 Apr 27 23:45 __pycache__
-rw-r--r-- 1 root root 119 Apr 27 23:45 README.txt
drwx------ 5 root root 4096 Apr 27 23:45 setuptools
drwx------ 2 root root 4096 Apr 27 23:45 setuptools-28.8.0.dist-info

You need to edit the pip3 file like so (your path could be different):
nano /Library/Frameworks/Python.framework/Versions/3.6/bin/pip3
And change the import main line so it says:
from pip._internal import main
After this change, my pip3 started to work and I am on macOS. If you are using Linux, your path could be something like /usr/bin/pip3
I found this tip and more information from this thread:
https://github.com/pypa/pip/issues/5240

Don't invoke pip/pip3 directly, which are shortcuts at different locations for different user and they are not well maintained when you upgrade pip (check 'which pip' and 'sudo which pip').
$ sudo python -m pip install xxx #for python2
$ sudo python3 -m pip install xxx #for python3
These commands do not take shortcuts and directly invoke pip module, which is maintained very well (also works on Windows)

My OS is Linux Mint 18
sudo nano /usr/bin/pip3
Change
from pip import main
To
from pip._internal import main

sudo chmod -R a+rx /usr/local/lib/python3.6/site-packages
You can see the problem right there in your ls -l /usr/local/lib/python3.6/site-packages output that your pip directory is only readable by the owner, which is root.

Do a cd /usr/bin
Then sudo nano pip so as to edit the pip file
Change from pip import main to from pip._internal import main
This will resolve the issue

try the following as a way around the issue until it got solved
sudo python -m pip --version
>> pip 10.0.1 from /usr/local/lib/python2.7/site-packages/pip (python 2.7)
sudo python -m pip install numpy --upgrade
>> Requirement already up-to-date: numpy in /usr/local/lib/python2.7/site-packages (1.14.2)

I don't exactly know the answer, but: that error indicates that the script can find some package called pip, but not the right one. Probably, an old version of pip, from back before they created a main method (you can check pip.__version__ from the python shell).
I'm willing to bet that you still have another, older version of python installed which has its own version of pip. For some reason your pythonpath is loading that one instead of the one that goes with py3.6.
Some possibilities to look into:
I don't know anything about redhat, but is there some redhat-specific way of choosing the "default python" to be used?
Is the shebang line at the top of the pip script something like #!/usr/bin/env python instead of #!/usr/bin/python3.6 like it should be?
Is it possible to modify your shell's PATH so that the downloaded python is used?
Is it possible to change your PYTHONPATH (i think it gets added to the default value of sys.path inside python; look it up) so that it loads the new pip instead of the old pip?

My OS was Mac sierra and I had to change the following line
from pip import main
into
from pip._internal import main

Seems the get pip script was missing the fact that more than 1 version of python could reside on a machine. So I added the following lines:
PY2 = sys.version_info[0] == 2
PY3 = sys.version_info[0] == 3
For some reason it was missing this. Anyway this worked for me. Thanks for your help guys.

Do the following steps to solve the problem --(Ubuntu-Debian)
step 1 - Go to directory
cd usr/bin/
and open pip3 file using gedit or nano(via terminal) .
Step 2- change from pip import main into from pip._internal import main
P.S - If you donot have permission to change use command - sudo chmod 777 pip3 and save this file.
Hope this helps!!

'EntryPoint' object has no attribute 'resolve' when using Google Compute Engine

I have an issue related to Cryptography package in Python. Can you please help in resolving these, if possible ? (tried a lot, but couldnt figure out the exact solution)
The python code which initiates this error:
print("Salt: %s" % salt)
server_key = pyelliptic.ECC(curve="prime256v1") # ----->> Line2
print("Server_key: %s" % server_key) # ----->> Line3
server_key_id = base64.urlsafe_b64encode(server_key.get_pubkey()[1:])
http_ece.keys[server_key_id] = server_key
http_ece.labels[server_key_id] = "P-256"
encrypted = http_ece.encrypt(data, salt=salt, keyid=server_key_id,
dh=self.receiver_key, authSecret=self.auth_key) # ----->> Line8
Value of "Salt" is getting displayed in 100% of the cases.
If Line3 gets executed successfully, I see the the following EntryPoint Error because of http_ece.encrypt() call (Line8):
AttributeError("'EntryPoint' object has no attribute 'resolve'",)
(Ref. File Link: https://github.com/martinthomson/encrypted-content-encoding/blob/master/python/http_ece/init.py#L128 )
Requirements.txt(partial):
cryptography==1.5
pyelliptic==1.5.7
pyOpenSSL==16.1.0
On Running the command: sudo pip freeze --all |grep setuptools, I get:
setuptools==27.1.2
Please let me know if any more detail is required.
This problem seems to be basically due to some Old/Incompatible packages(related to PyElliptic, Cryptography, PyOpenSSL and/or setuptools) installed on the VM. For Reference: https://github.com/pyca/cryptography/issues/3149
Can someone please suggest a good solution to resolve this issue completely ?
Thanks,

The issue referenced in c66303382 has this traceback (you never gave your traceback so I have to assume yours ends the same way):
File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/__init__.py", line 35, in default_backend
_default_backend = MultiBackend(_available_backends())
File "/usr/local/lib/python2.7/dist-packages/cryptography/hazmat/backends/__init__.py", line 22, in _available_backends
"cryptography.backends"
The full line that triggers the error looks like this:
_available_backends_list = [
ep.resolve()
for ep in pkg_resources.iter_entry_points(
"cryptography.backends"
)
]
Searching the repository for EntryPoint definition, then blaming pkg_resources/__init__.py where it is reveals that pkg_resources.EntryPoint.resolve() was added in commit 92a553d3adeb431cdf92b136ac9ccc3f2ef98bf1 (2015-01-05) that went into setuptools v11.3.
Thus you'll see this error if you use an older version.

Ran Following Commands from the project path /opt/projects/myproject-google/myproject and it resolved the Attribute EntryPoint Error Issue:
(Assuming project virtual env path as: /opt/projects/myproject-google/venv)
Command: (from path: /opt/projects/myproject-google/myproject)
export PYTHONPATH= # [Blank]
sudo pip install --upgrade virtualenv setuptools
sudo rm -rf ../venv
sudo virtualenv ../venv
source ../venv/bin/activate
sudo pip install --upgrade -r requirements.txt
deactivate
Running the above commands upgraded the virtual environment & the setuptools version inside the virtual Env. located at path: /opt/projects/myproject-google/venv/lib/python2.7/site-packages. To test if setuptools have upgraded successfully, try some of these commands:
Command: sudo virtualenv --version
Output: 15.0.3
Command: echo $PYTHONPATH
Output: [blank]
Command: python -c 'import pkg_resources; print(pkg_resources.__file__)'
Output: ~/.local/lib/python2.7/site-packages/pkg_resources/__init__.pyc
Command: python -c 'import sys; print(sys.path)'
Output: ['', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-x86_64-linux-gnu', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '~/.local/lib/python2.7/site-packages', '/usr/local/lib/python2.7/dist-packages', '/opt/projects/myproject-google/myproject', '/usr/local/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages', '/usr/lib/python2.7/dist-packages/PILcompat']
Command: ls /opt/projects/myproject-google/venv/lib/python2.7/site-packages
Output:
easy_install.py pip pkg_resources setuptools-27.2.0.dist-info wheel-0.30.0a0.dist-info
easy_install.pyc pip-8.1.2.dist-info setuptools wheel
Command: python -c 'from cryptography.hazmat.backends import default_backend; print(default_backend())'
Output: <cryptography.hazmat.backends.multibackend.MultiBackend object at 0x7ff83a838d50>
Command /opt/projects/myproject-google/venv/bin/python -c 'from cryptography.hazmat.backends import default_backend; print(default_backend())'
Output
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named cryptography.hazmat.backends
Command: /opt/projects/myproject-google/venv/bin/python -c "import pkg_resources; print(pkg_resources.__file__)"
Output: /opt/projects/myproject-google/venv/local/lib/python2.7/site-packages/pkg_resources/__init__.pyc
Ref Link: https://github.com/pyca/cryptography/issues/3149
These Steps resolved the Attribute EntryPoint Issue completely with an updated version of cryptography package & the setuptools.
Update As on 15 September 2016, The Cryptography Team has again added the workaround for supporting old packages too.
(Ref. Link: https://github.com/pyca/cryptography/issues/3150 )

Locale.Error with building python based docker

I am new to docker and would appreciate if someone can help me get rid of this error while building the docker image. It is giving some kind of locale error. How can I get rid of this error ?
Collecting pip
Downloading pip-8.1.1-py2.py3-none-any.whl (1.2MB)
Collecting setuptools
Downloading setuptools-20.3.1-py2.py3-none-any.whl (508kB)
Collecting wheel
Downloading wheel-0.29.0-py2.py3-none-any.whl (66kB)
Installing collected packages: pip, setuptools, wheel
Successfully installed pip-8.1.1 setuptools-20.3.1 wheel-0.29.0
+ pip install --no-cache-dir --upgrade pip==8.0.2
Traceback (most recent call last):
File "/usr/bin/pip", line 11, in
sys.exit(main())
File "/usr/lib/python2.7/site-packages/pip/__init__.py", line 215, in main
locale.setlocale(locale.LC_ALL, '')
File "/usr/lib64/python2.7/locale.py", line 547, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
The command '/bin/sh -c set -ex && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY" && curl -fSL 'https://bootstrap.pypa.io/get-pip.py' | python2 && pip install --no-cache-dir --upgrade pip==$PYTHON_PIP_VERSION' returned a non-zero code: 1
This is my Dockerfile:
FROM mybase:1.0.7
RUN set -x \
&& yum install -y python-devel libffi-devel python-cffi \
&& yum clean all
ENV LANG C.UTF-8
ENV GPG_KEY C01E1CAD5EA2C4F0B8E3571504C367C218ADD4FF
ENV PYTHON_VERSION 2.7.11
ENV PYTHON_PIP_VERSION 8.0.2
RUN set -ex \
&& gpg --keyserver ha.pool.sks-keyservers.net --recv-keys "$GPG_KEY" \
&& curl -fSL 'https://bootstrap.pypa.io/get-pip.py' | python2 \
&& pip install --no-cache-dir --upgrade pip==$PYTHON_PIP_VERSION
RUN pip install --no-cache-dir virtualenv
CMD ["python2"]

The locale.setlocale docs say the locale should be valid. If an empty string is passed, the LANG variable is used to set the locale. This error is probably caused because your LANG is not a supported locale
In your docker script, you set LANG to C.UTF-8. It looks like C.UTF-8 is not a supported locale in glibc and I am guessing hence in Python (See this and this).
You can set your LANG to a supported type like en_US.UTF-8 (The default on my computer).
On Python-2.6, I get the following results
Python 2.6.6 (r266:84292, Jan 22 2014, 09:42:36)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.setlocale(locale.LC_ALL, "")
'en_US.utf8'
>>> locale.setlocale(locale.LC_ALL, "C.UTF-8")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.6/locale.py", line 513, in setlocale
return _setlocale(category, locale)
locale.Error: unsupported locale setting
>>> locale.setlocale(locale.LC_ALL, "en_US.UTF-8")
'en_US.UTF-8'
>>> locale.setlocale(locale.LC_ALL, "de_DE.UTF-8")
'de_DE.UTF-8'
You can view the locales available on your computer by running
$ locale -a

Migrating default python installation from 2.6 to 2.7: problems with easy_install

I'm using debian with a default python installation of 2.6
I wanted to migrate to python 2.7, including installing easy_install.
I followed someone else's instruction to delete my /usr/bin/python, then link
ln -s /usr/bin/python2.7 /usr/bin/python
I downloaded the most recent version of setuptools
and cd'ed to the file. The install help asked me to run it as a shell program, I did, with the following error:
sh setuptools-0.6c11-py2.7.egg
Traceback (most recent call last):
File "<string>", line 1 in <module>
ImportError: No Module named setuptools.command.easy_install
I have a feeling that my installation of easy_install is related to the version of python I'm running, but I can't quite figure it out. I tried also downloading the .tar.gz file, cd-ing into the directory, and running
python setup.py build; setup.py install
After I run that, I can use easy_install, with the following error:
Traceback (most recent call last):
File "/usr/local/bin/easy_install", line 5, in <module>
from pkg_resources import load_entry_point
ImportError: No Module named pkg_resources
Can anyone suggest to me a solution? Thanks for the help.

Replacing the system Python isn't a great idea. Moving up by one point release
probably won't break your system, but who wants to take the risk? Installing
libraries to your system Python using easy_install can also lead to problems
if they conflict with each other.
An alternative is to build Python from source, install it to your home
directory, and use virtualenv to create isolated environments into which you can
install whatever libraries you need for a given project using pip (which is
the more modern equivalent to easy_install).
For Python 2.7, if you want some of the 'optional' parts of the standard
library, that means building a couple of other things, too. Here's a script
(largely cobbled together from blog posts scattered across the interweb) that
works for me on Debian "Squeeze" (stable, at the time of writing):
#!/bin/bash -e
# Setup
sudo aptitude install build-essential
mkdir -p ${HOME}/.local
mkdir build-python
cd build-python
# Get sources
### Tcl/Tk <http://www.tcl.tk/software/tcltk/download.html>
wget "http://downloads.sourceforge.net/project/tcl/Tcl/8.5.13/tcl8.5.13-src.tar.gz"
wget "http://downloads.sourceforge.net/project/tcl/Tcl/8.5.13/tk8.5.13-src.tar.gz"
### Berkeley DB <http://www.oracle.com/technetwork/products/berkeleydb/downloads/index-082944.html>
wget "http://download.oracle.com/berkeley-db/db-4.8.30.tar.gz"
### Python <http://www.python.org/download/>
wget "http://www.python.org/ftp/python/2.7.3/Python-2.7.3.tgz"
# Build Tcl
tar xzf tcl8.5.13-src.tar.gz
cd tcl8.5.13/unix
./configure --prefix=${HOME}/.local
make
make install
cd ../..
# Build Tk
tar xzf tk8.5.13-src.tar.gz
cd tk8.5.13/unix
./configure --prefix=${HOME}/.local
make
make install
cd ../..
# Build Berkeley DB 4.8
tar xzf db-4.8.30.tar.gz
cd db-4.8.30/build_unix
../dist/configure --prefix=${HOME}/.local/opt/BerkeleyDB.4.8 --enable-tcl --with-tcl=${HOME}/.local/lib
make
make install
cd ../..
# Set compile flags
export LDFLAGS="-L${HOME}/.local/lib -L${HOME}/.local/opt/BerkeleyDB.4.8/lib"
export CPPFLAGS="-I${HOME}/.local/include -I${HOME}/.local/opt/BerkeleyDB.4.8/include"
export CXXFLAGS=${CPPFLAGS}
export CFLAGS=${CPPFLAGS}
export LD_LIBRARY_PATH=${HOME}/.local/lib:${HOME}/.local/opt/BerkeleyDB.4.8/lib
export LD_RUN_PATH=${LD_LIBRARY_PATH}
# Build Python 2.7
tar xzf Python-2.7.3.tgz
cd Python-2.7.3
./configure --prefix=${HOME}/.local
make
make altinstall
cd ..
# Install virtualenv, pip and virtualenvwrapper
curl http://python-distribute.org/distribute_setup.py | ${HOME}/.local/bin/python2.7
curl https://raw.github.com/pypa/pip/master/contrib/get-pip.py | ${HOME}/.local/bin/python2.7
${HOME}/.local/bin/pip install virtualenvwrapper
# Update ~/.bashrc
echo 'export PATH="${HOME}/.local/bin:${PATH}"' >> ${HOME}/.bashrc
echo 'export WORKON_HOME="${HOME}/.local/virtualenv"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_PYTHON="${HOME}/.local/bin/python2.7"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_VIRTUALENV="${HOME}/.local/bin/virtualenv"' >> ${HOME}/.bashrc
echo 'export VIRTUALENVWRAPPER_VIRTUALENV_ARGS="--python=python2.7"' >> ${HOME}/.bashrc
echo 'source ${HOME}/.local/bin/virtualenvwrapper.sh' >> ${HOME}/.bashrc
# Finish ...
cd ..
echo -e "\n\n ... Done!"
This script will ask for your password in order to install GCC etc. if it needs to, then take a while to build everything - about 25 minutes on my (ancient)
machine and (terrible) internet connection. If you're paying attention, at some
point you'll see the following message scroll past:
Python build finished, but the necessary bits to build these modules were not found:
bsddb185 dl imageop
sunaudiodev
To find the necessary bits, look in setup.py in detect_modules() for the module's name.
Those four modules are archaic and/or deprecated, so you don't need to worry
about them. If the message mentions any other modules, that means some
necessary library isn't installed on your system. You can still run Python if that's the case, but won't be able to import those modules. Shout in the comments if you're affected by this, and
I'll update the script accordingly.
Once that's finished successfully, you need to source your .bashrc:
$ source ~/.bashrc
... and you'll then be able to run your newly compiled Python ...
$ python2.7
Python 2.7.3 (default, Nov 17 2012, 02:00:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
... create a virtualenv to work in ...
$ mkvirtualenv my_env
$ python
Python 2.7.3 (default, Nov 17 2012, 02:00:26)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
... install libraries into it ...
$ pip install beautifulsoup4
... and so on.
To exit a virtualenv:
$ deactivate
To re-enter it at a later date:
$ workon my_env
For more, check out the documentation for pip and virtualenvwrapper.