I'm not sure if the default python installation is the one that I've been installing modules to, and if that may be the cause of a conflicting Unicode byte size compatibility error. In short, I've installed Numpy 1.7 using Python 2.7.3 and when I try to install this other program that uses Python and Numpy as dependencies, I get this error:
Traceback (most recent call last):
File "setup.py", line 20, in <module>
from weblogolib import __version__
File "/home/chris/Documents/IS/Bioinformatics-Software/weblogo-3.3/weblogolib/__init__.py", line 108, in <module>
from numpy import array, asarray, float64, ones, zeros, int32,all,any, shape
File "/usr/lib/python2.7/dist-packages/numpy/__init__.py", line 137, in <module>
import add_newdocs
File "/usr/lib/python2.7/dist-packages/numpy/add_newdocs.py", line 9, in <module>
from numpy.lib import add_newdoc
File "/usr/lib/python2.7/dist-packages/numpy/lib/__init__.py", line 4, in <module>
from type_check import *
File "/usr/lib/python2.7/dist-packages/numpy/lib/type_check.py", line 8, in <module>
import numpy.core.numeric as _nx
File "/usr/lib/python2.7/dist-packages/numpy/core/__init__.py", line 5, in <module>
import multiarray
ImportError: /usr/lib/python2.7/dist-packages/numpy/core/multiarray.so: undefined symbol: PyUnicodeUCS4_AsUnicodeEscapeString
So I guess I have a conflicting unicode byte size (2-byte vs. 4-byte). I went to check to see if I had conflicting versions of Python that could be messing this up.
python --version
Python 2.7.3
But this seems at odds with
which python
/usr/local/bin/python
When I go to /usr/local/bin I find these files (relevant to python):
python
python2
python2.7
python-config
python2-config
python2.7-config
Now I've installed numpy into the dist-packages directory of /usr/lib/python2.7/dist-packages which corresponds to what I get for python --version. But the fact that when I try which python and get a directory for python and not python2.7 concerns me that this might be conflicting when I try to install the program that uses python and numpy as dependencies.
So I guess to clarify my question(s): Are these normal files to find for a python installation or have I somehow installed three different versions? Could they be causing my error with the unrecognized symbol? Is there a way to uninstall if they are indeed extraneous versions?
Thanks for any help you can provide!
Oh and here is a link to a previous question I had, where I edited the PYTHONPATH while trying to fix an ImportError I was getting, if that might be affecting things....ImportError: No module named numpy
Here are the results of trying virtualenv:
chris#ubuntu:~/Documents/IS/Bioinformatics-Software$ virtualenv weblogo-3.3
New python executable in weblogo-3.3/bin/python
Installing setuptools.............done.
Installing pip...............done.
chris#ubuntu:~/Documents/IS/Bioinformatics-Software$ cd weblogo-3.3
chris#ubuntu:~/Documents/IS/Bioinformatics-Software/weblogo-3.3$ source bin/activate
(weblogo-3.3)chris#ubuntu:~/Documents/IS/Bioinformatics-Software/weblogo-3.3$ pip install numpy
Requirement already satisfied (use --upgrade to upgrade): numpy in /usr/lib/python2.7/dist-packages
Cleaning up...
The problem indeed seems to be a mismatch of Python and Numpy compile settings.
/usr/local/bin is where custom Python is installed, you should try to run using /usr/bin/python instead.
Another solution is to use a virtualenv. Try this:
virtualenv myproject
cd myproject
source bin/activate
pip install numpy
Basically virtualenv sets up a different Python installation with its own packages in the "myproject" directory. Running the "activate" command tells the system that you want to use this installation instead of the default system. This lets you have a different Python environment for different projects. Using virtualenv, each project can have its own versions of Python packages even if they're incompatible with other projects or system packages.
Note you will have to repeat the "source" command each time you open a new shell and want to use that virtual environment. Also you might have to install the virtualenv command by using your OS package manager. If this isn't possible (e.g. you don't have root access) or your OS version is too old for some reason, you can also download it manually from https://pypi.python.org/packages/source/v/virtualenv/
If you do ls -l /usr/local/bin/python* you should see that python and python2 are actually symlinks to python2.7, and likewise python-config and python2-config are symlinks to python2.7-config.
What OS are you on? This is more a question for superuser, but try something like this. Ditch easy_install and use pip if you haven't already.
On Ubuntu:
sudo apt-get install python-setuptools
sudo easy_install pip
pip install --user numpy
Related
The default Python version for MSYS2 seems to be 3.8. I need to use 3.7 at the moment because I have to use PyInstaller and it is not currently compatible with 3.8 in MSYS2. I can download the earlier version of Python from http://repo.msys2.org/ and install it using pacman. With a fresh install of MSYS2 I run the following commands:
pacman -S glib2-devel
pacman -U python-3.7.4-1-x86_64.pkg.tar.xz
pacman -S python-pip
pacman -S python-setuptools
If I try to run a python script I am met with an error:
File "setup.py", line 15, in <module>
from setuptools import setup
ModuleNotFoundError: No module named 'setuptools'
This is due to the fact that everything installed after Python is actually installing in the default Python 3.8 location rather than 3.7:
C:\msys64\usr\lib\python3.8\site-packages
If I copy and paste the contents of site-packages into Python 3.7 and then try running a script I get the error:
File "setup.py", line 15, in <module>
from setuptools import setup
File "/usr/lib/python3.7/site-packages/setuptools/__init__.py", line 19, in <module>
from setuptools.dist import Distribution
File "/usr/lib/python3.7/site-packages/setuptools/dist.py", line 34, in <module>
from setuptools import windows_support
File "/usr/lib/python3.7/site-packages/setuptools/windows_support.py", line 2, in <module>
import ctypes
File "/usr/lib/python3.7/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: No such file or directory
Which is supposed to be resolved via libffi, which was installed prior to Python, but likely does not go to a location that 3.7 can recognize?
Is there a way to set a specific version of Python as the default in MSYS2? Perhaps a path that can be set in the .bashrc file? I tried to set PYTHONPATH in there to Python 3.7 but it didn't make a difference as to where the packages ended up being installed to.
Go to the following URL...
https://repo.msys2.org/msys/x86_64/
Look for the version of the package you need and download it. Use
pacman -U pkgname
To install it...
The reason I know this is because gcc11 on MSYS is non-functional... it produces programs which crash and say
"During startup your program exited with code..." mentioned here:
During startup program exited with code 0xc0000139
The only workaround is to downgrade to the previous compiler which does work or use clang, which was not an option for me.
Trying to use an old version of Python is probably possible, but I think it will be very difficult. You would need to make sure all the other installed packages are compatible.
I am using PyInstaller with Python 3.8 in MSYS2 without issues. I would recommend trying to work through any issues with that, instead of trying to use older versions of packages.
I'm running VPS with ubuntu:
Distributor ID: Ubuntu
Description: Ubuntu 14.04.5 LTS
Release: 14.04
Codename: trusty
I tried to upgrade Python to 2.7.16 from 2.7.6.
Just after upgrade Gunicorn library failed to start with
Traceback (most recent call last):
File "/home/user/bin/python-2.7/bin/gunicorn", line 7, in <module>
from gunicorn.app.wsgiapp import run
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 9, in <module>
from gunicorn.app.base import Application
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/gunicorn/app/base.py", line 12, in <module>
from gunicorn import util
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/gunicorn/util.py", line 12, in <module>
import pkg_resources
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/pkg_resources/__init__.py", line 82, in <module>
__import__('pkg_resources.extern.packaging.requirements')
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/pkg_resources/extern/__init__.py", line 61, in load_module
"distribution.".format(**locals())
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
I was told that it could be some setuptools issue, so I renewed them with
python -m ensurepip
Looking in links: /tmp/tmp98U8zf
Requirement already satisfied: setuptools in ./bin/python-2.7/lib/python2.7/site-packages (41.1.0)
Requirement already satisfied: pip in ./bin/python-2.7/lib/python2.7/site-packages (19.2.2)
(no changes)
I tried to do
pip install packaging
(no changes)
or downgrade setuptools to 19.2 (as suggested in Python 2.7 The 'packaging' package is required; normally this is bundled with this package)
and got
File "/home/user/bin/python-2.7/lib/python2.7/site-packages/gunicorn/config.py", line 8, in <module>
import copy
File "/home/user/bin/python-2.7/lib/python2.7/copy.py", line 52, in <module>
import weakref
File "/home/user/bin/python-2.7/lib/python2.7/weakref.py", line 14, in <module>
from _weakref import (
ImportError: cannot import name _remove_dead_weakref
Is there any working way to upgrade to 2.7.16 python in my case?
UPD:
I upgraded python with it with
wget https://www.python.org/ftp/python/2.7.16/Python-2.7.16.tgz
make clean && ./configure --enable-unicode=ucs4 --enable-shared --prefix=/home/user/bin/python-2.7/ --with-ensurepip=install && make && make install
pip install -r ~/django/django_projects/requirements.txt
pip freeze: https://www.pastiebin.com/5d592ea701503
Thanks.
Scenario
First: When trying to execute a Python script, the following error message appears:
ImportError: The 'packaging.requirements' package is required; normally this is bundled with this package so if you get this warning, consult the packager of your distribution.
Second: When trying to check the version of setuptools with pip show setuptools (or when using any other pip command), the next error message appears:
ImportError: cannot import name _remove_dead_weakref
Reason
This is because the current Python installation is somehow screwed up. In my case the mess was caused by an upgrade from Ubuntu 16.04 to Ubuntu 18.04.
Solution
With Virtual Environment
In the best case you were already using a virtual environment (this was my case). The solution here would be to recreate/setup your venv again (step-by-step):
$ cd /path/to/your/venv
# remove your old venv
$ rm -rf ./*
# create a new one
$ /usr/bin/virtualenv . --python=YOUR-PYTHON-VERSION
# activate venv
$ source bin/activate
# verify the correct python version is installed
$ python --version
For example replace YOUR-PYTHON-VERSION with python2.7 or python3.7.
After recreating your venv the problem should be fixed and you should be able to use pip again.
Without Virtual Environment
I think the best way to fix the problem would be to completely uninstall all non-default Python versions (for example Ubuntu 18.04 comes with Python 3.6+) and then make a clean reinstall of all other Python versions needed.
Try upgrading pip by typing:
pip install --upgrade pip
I need to use netcdf but do not have install permission for python modules. I have downloaded netcdf-0.1.2.tar.gz from here: https://pypi.python.org/simple/netcdf/ and unzipped the tar ball. I have been following this stack overflow post in an attempt to use the module but have had no luck so far:
(Python) Use a library locally instead of installing it
here is what I have tried:
Installing virtualenv:
I do not have permission to do this
python setup.py install -- user:
again, I don't have permission
running my script with netcdf as my current working directory:
I tried this as well, here are the issues I have run into:
first I went into netcdf-0.1.2 and made a new file called asdf.py
which contains the following:
import netcdf
print("testing")
running python asdf.py gives the following error:
Traceback (most recent call last):
File "asdf.py", line 1, in <module>
import netcdf
File "/.../Downloads/netcdf-0.1.2/netcdf/__init__.py", line 1, in <module>
from netcdf import *
File "/.../Downloads/netcdf-0.1.2/netcdf/netcdf.py", line 1, in <module>
from netCDF4 import Dataset, numpy
ImportError: No module named netCDF4
I'm not sure how to fix this error, any help would be greatly appreciated
in case this is somehow relevant, the version of Linux I am using is 3.2.0-23-generic
also I have numpy installed already
Easest would be to install Anaconda or Miniconda with your user rights.
Anaconda already as netCDF4installed. In case of Miniconda install with:
conda install netcdf4
If you have Python 3 installed, then you will have the venv package in the standard library, so you do not need "virtualenv" to be installed for you separately (as would be the case with Python 2). Instead use python3 -mvenv , in a similar way to how you would use virtualenv, for example:
python3 -mvenv /path/to/my_venv
or to include any non-standard packages already installed on the system:
python3 -mvenv --system-site-packages /path/to/my_venv
After that, you should be able to activate the environment and pip install packages, e.g.
source /path/to/my_venv/bin/activate # for csh use activate.csh instead
pip install netCDF4
Remember to source the activate script at run time as well as installation time:
source /path/to/my_venv/bin/activate
python
and you should then find that in your python session you have the netCDF4 package available, e.g.
import netCDF4
my_dataset = netCDF4.Dataset('myfile.nc')
Of course, substitute the actual path in place of /path/to/my_venv above.
None of this requires any root privileges.
(And as someone else has suggested, another option for you is to use conda.)
I'd also like to highlight that the package is imported using capitals
import netCDF4 as nc
This might not matter on a mac, but for Windows it is key.
I wrote myself a handy bash script, which solves the task of creating a virtualenv with its own compiled virtualenv and python. It aims at creating a mostly self contained virtualenv, with maybe only native libraries installed in system level if necessary, but installing all python packages and virtualenv and pip and such things inside the virtualenv.
The script can be found here.
I invoke the script as follows:
self_contained_venv.sh \
-n udacity_model_building_and_validation \
-p https://www.python.org/ftp/python/3.4.4/Python-3.4.4.tar.xz \
-v https://pypi.python.org/packages/c8/82/7c1eb879dea5725fae239070b48187de74a8eb06b63d9087cd0a60436353/virtualenv-15.0.1.tar.gz#md5=28d76a0d9cbd5dc42046dd14e76a6ecc \
-d pandas scikit-learn seaborn
Given the required packages for compiling python and virtualenv are installed on the system, the script creates a nice virtualenv. However, when I try to access any installed modules/packages from within the virtualenv, python is not able to find them. To demonstrate this, I'll put some output of commands and code here:
First of all of course I have to activate the virtualenv:
. bin/activate
output: None, works without problem.
Then I print the pythonpath python is aware of:
import sys
for i in sys.path:
print(i)
output:
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python34.zip
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/lib-dynload
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/plat-linux
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So far so good. Then I try to import a module / package I installed during usage of my bash script: pandas:
python
(IDLE is running)
import pandas as pd
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pandas'
Another try:
import numpy as np
output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'numpy'
Huh? So none of the packages is available? Lets check pip again:
which pip
output:
alias pip='localpython/bin/pip3.4'
./localpython/bin/pip3.4
Ok, so it's using my local pip.
Check packages:
pip list
output:
numpy (1.11.0)
pandas (0.18.1)
pip (8.1.2)
psutil (4.1.0)
Python-contrib-nbextensions (alpha)
python-dateutil (2.5.3)
pytz (2016.4)
PyYAML (3.11)
setuptools (18.2)
six (1.10.0)
virtualenv (15.0.1)
Hm the packages are there, so why can't python find them? Let's see where those packages are located, simply by trying to remove one:
pip uninstall pandas
output (shortened, because it fills many pages):
Uninstalling pandas-0.18.1:
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas-0.18.1-py3.4.egg-info
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__init__.py
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/__init__.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/_version.cpython-34.pyc
/home/xiaolong/development/Python/udacity_model_building_and_validation/localpython/lib/python3.4/site-packages/pandas/__pycache__/info.cpython-34.pyc
Aha, so the packages are in the path.
Another attempt on finding out if python looks in the right places:
>>> from distutils.sysconfig import get_python_lib
>>> print(get_python_lib())
/home/xiaolong/development/Python/udacity_model_building_and_validation/lib/python3.4/site-packages
So this one indicates it does not look in the right place, if I understand correctly.
Note:
The script does the following things, which might not be obvious:
compile python with a prefix (local python)
compile virtualenv for the local python
it aliases pip to the local pip of the version of the installed python
it aliases virtualenv to the locally installed one
it installs packages from pypi if specified
it updates the local pip if there is a newer version available
I am a beginner still at writing bash scripts, so I think the structure and logic of the script is fairly easy to understand. It also prints information about success of its operations in the terminal.
Further Notes:
I did not use su or sudo to run the script.
My OS is a Fedora 22, although I think in this case any major distro would work the same way.
OS has been updated recently.
Question: So why can't Python find them / its own packages? (What do I need to change?)
can you try to compare path to pyhton and pip in your script and manually in your system:
which python and which pip?
After that check pip freeze | grep pandas it should return you package ...if no you need to add this package to site-packages folder of you python.
I've used pip to install the module numpy (sudo pip install numpy).This works fine.
When importing numpy for use in my own module i get several the following errors.
Traceback (most recent call last):
File "<pyshell#65>", line 1, in <module>
import numpy
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/__init__.py", line 180, in <module>
from . import add_newdocs
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/add_newdocs.py", line 13, in <module>
from numpy.lib import add_newdoc
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/lib/__init__.py", line 8, in <module>
from .type_check import *
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/lib/type_check.py", line 11, in <module>
import numpy.core.numeric as _nx
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/core/__init__.py", line 14, in <module>
from . import multiarray
ImportError: dlopen(/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/core/multiarray.so, 2): Symbol not found: _PyBuffer_Type
Referenced from: /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/core/multiarray.so
Expected in: flat namespace
in /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/numpy/core/multiarray.so
I'm assuming the error has something to do with the multiarray.so file. I've tried turning it off and on (uninstall re install) and adding the where pip initially saves the module to python export path
(export PYTHONPATH="usr/local/lib/python2.7/site-packages")
doesn't seem to work.
Using which python in terminal gives me Python 2.7.6**, however I downloaded python 3.5 off of the site.
Issue resolved.
The reason i was having installation issues with pip, was down to the fact (thanks pv) that i was using the command:
"pip install moduleName"
rather than
"pip3.5 install moduleName"
this caused all sorts of complications, because i'm running python 3.5 and the "pip" command is for 2.7.
Thanks again.
FWIW -- and for people in the future who are googling for this error -- I ran into this issue when installing numpy separately via Homebrew. It caused all my installations of Python (via pyenv) to refer to the /usr/local/lib installation of numpy (which is where Homebrew installs it). Uninstalling it via brew uninstall numpy fixed the error.
(I don't remember why I had a homebrew-installed numpy, but oh well)
I encountered the same problem.
I tried many ways like:
$pip uninstall numpy
But that did not fix the problem.
I assume a major reason is that I have anaconda multi-Python environments (I have created both py27 and py35). As a result the PYTHONPATH includes paths for both py27 and py35.
My default python env is py35. So using
$conda install numpy
will install numpy under the anaconda python3.5 path.
When uninstalling the numpy, we are not sure which numpy path is exporting, maybe it is only for my case.
Hence I refer to #Dan Nhuyen's solution.
I uninstall numpy via:
$brew uninstall numpy.
As a result, the extra ambiguous paths is removed. Finally, it works.
What worked for me was to request pip to build numpy from source, instead of using the pre-compiled wheel binary:
pip install --no-binary numpy -r requirements.txt