During the past years, I have installed many Python libraries with various Python versions. To make them ready to work immediately, I installed them blindly without control. Currently they're causing problems when I tried to install pynest which invokes numpy, scipy and matplotlib. After struggling, I am going to clean and reinstall Python and the libraries.
After investigation, I found Python 2.5/2.6/2.7/3.2 on my system, and each of them has some copies or other things at: (my OS == Mac OS X 10.7.5 Lion)
/Library/Frameworks/
/opt/local/Library/Frameworks/
/opt/local/bin/
/Applications/
/usr/local/bin/
/usr/bin/
/System/Library/Frameworks/
I know I'm crazy to have these. Now I have removed all these except the things in /System/Libarary/Frameworks (I never remove any thing from /System/Library/). After the clean work, which python now gives /usr/bin/python which links to /System/Library/Frameworks.
Now, is it a clear environment for me to reinstall python? How to double check that there's no other versions existing? How should I reinstall them to guarantee that they and their libraries won't be everywhere and have many copies again?
I want to install a clean Python 2.7 onto a proper location, and make my system know exactly where it is and never install any libraries somewhere else. Please give me some advice that how to manage it like in a professional way.
For your information, here is my current $PATH, I think it should be modified:
/opt/local/bin:/opt/local/sbin:/opt/nest/lib/python2.7/site-packages:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/usr/texbin:/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/site-packages/django/bin:/usr/X11/bin:/opt/local/bin:/opt/local/sbin:/usr/local/lib/python2.7/site-packages:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.2/bin
Please let me know If you need more information. Thank you!
UPDATE:
I'm rethinking profoudly why it becomes so crazy. I believe it's because I installed things via:
easy_install / macports / homebrew / fink / pip sometimes;
.dmg sometimes;
.pkg sometimes;
compile source code sometimes;
and they made things at different locations. I wonder what's the mechanism behind these ways? How do they choose target location? How to prevent them from messing things up?
Why did it get messed up?
There're a couples of different way to install Python, as the update of OP says, and they locate files in different locations. For example, macports puts things into /opt/local/, while homebrew puts things into /usr/local/. Also, Mac OS X brings a few python versions with itself. So, if you install python many times via different ways, you will get many python versions existing independently on your system.
What problem does it cause?
I don't know exactly. I guess the problem is that if you have many versions of python, then which one to use and where to find packages will be determined by the path order in your system PATH and the PYTHONPATH respectively. So you may lose control of where to install python modules. Consider that if you run sudo python setup.py install to install a module (it finds python by the root's PATH) and then try to import the module by python -c "import it" (this time it finds python by your PATH), maybe something will go wrong. This is my guess, I didn't validate it. But in my own case, something did go wrong.
How to avoid this?
I think the principle would be that be aware of that different ways and tools install things independently to different locations, so use them mindfully.
Unless you intend to, don't install the same thing twice via different
ways. (If you intend to do it for python, you might want to check out virtualenv)
Keep an eye on the path order in your PATH and consider if it's
correct.
When installing modules, be clear which python (or pip) is
running and where the module is installed.
So, how did I solve my own case?
Since it had been messing up already and seemed to be very hard to cure, so finally I solved this question by a full OS re-installation, and started to follow the DOs-and-DONTs above. For the installation of the scientific environment with python (numpy/scipy/matplotlib, which had shown problems to make me ask this question), I found this tutorial was extremely helpful. So, problem solved finally.
Here is what was confusing me and how I solved it.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ ls /usr/local/bin/python
ls: /usr/local/bin/python: No such file or directory
So notice I didn't have a HomeBrew installation of python2.7, but did have the python3 installation. The version under /usr/bin/python is using the system default. You can tell based on the module search path:
$ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
`enter code here`Type "help", "copyright", "credits" or "license" for
more information.
>>> import sys
>>> sys.path
['', '/Library/Python/2.7/...
Notice the '/Library/Python'... that's Mac OS's version of python. But I want to stay strictly on a user installed version (i.e. HomeBrew).
So here's what I did to fix this:
$ brew install python
...
Warning: python 2.7.13 is already installed, it's just not linked.
You can use `brew link python` to link this version.
$ brew link --overwrite python
$ which python
/usr/local/bin/python
$ python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/usr/local/Cellar/python/2.7.13...
Its no longer /Library/.. but /usr/local.
Now its finding all of my pip installed modules! Problem solved!
UPDATE:
After updating brew to version 1.5.4, it seems the symbolic links were removed. And now you have to add this to your path:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Read the Caveats section in 'brew info python':
==> Caveats
This formula installs a python2 executable to /usr/local/bin.
If you wish to have this formula's python executable in your PATH then add
the following to ~/.bash_profile:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
Pip and setuptools have been installed. To update them
pip2 install --upgrade pip setuptools
You can install Python packages with
pip2 install <package>
They will install into the site-package directory
/usr/local/lib/python2.7/site-packages
See: https://docs.brew.sh/Homebrew-and-Python.html
In order to install a python distributions into specific folder, you can use the --prefix scheme during python installation. Using the prefix scheme, you can for example install Python 2.7 into the folder /opt/py27. Now, in order to use the new installed Python distribution you have to: cleanup you PATH and LD_LIBRARY_PATH:
Remove all 'old' Python paths and
configure (according to my example) the environment variables like this:
PATH: Add /opt/py27/bin
LD_LIBRARY_PATH: Add /opt/py27/lib
That's it.
(In case you need multiple environments of Python installed at the same time, I'd suggest to have a look at virtualenv)
tl;dr
brew install python
Symptoms
I had similar issues with python programs not finding dependencies.
My python3 version was a broken symlink.
My pip was pointing to a python 3.8
And my pip3 was pointing to 3.9
python -V was outputting some python 2.7 version
python3 -V was outputting some python3.8 version
Solution
I ran brew install python and it fixed all my problems.
Related
I have installed python on my macbook with the python installer from Python.org. Subsequently, I went and installed pip, ipython, and numpy. Everything seemed fine. However, now I am getting the following problem. I can import numpy when I run ipython, but not when I run regular python.
E.g.
Logister-MacBook-Pro:~ Logister$ ipython
Python 2.7.6 (default, Sep 9 2014, 15:04:36)
Type "copyright", "credits" or "license" for more information.
IPython 3.1.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
In [1]: import numpy
In [2]: import site; site.getsitepackages()
Out[2]:
['/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python',
'/Library/Python/2.7/site-packages']
However, when I try to do the same thing in python 2.7.9:
Logister-MacBook-Pro:~ Logister$ python
Python 2.7.9 (v2.7.9:648dcafa7e5f, Dec 10 2014, 10:10:46)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named numpy
>>> import site; site.getsitepackages()
['/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages',
'/Library/Frameworks/Python.framework/Versions/2.7/lib/site-python',
'/Library/Python/2.7/site-packages']
When I try to install numpy via pip it gives me the following response:
Logister-MacBook-Pro:~ Logister$ sudo -H pip install numpy
Requirement already satisfied (use --upgrade to upgrade):
numpy in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
It seems like python 2.7.6 understands where to find numpy, but 2.7.9 does not. Either, how can I point 2.7.9 to the right place, or how can I install numpy so 2.7.9 sees it as well?
Edit: I can run:
site.addsitedir('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python')
In 2.7.9 and then I am able to import numpy. But I dont want to do this every time I launch 2.7.9. Is there a permanent fix? Also, how do I get ipython to run 2.7.9 instead of 2.7.6?
I had a similar problem.
There is two differences versions of python installed on your computer.
Then you´ve installed numpy on python 2.7.6. but the folder of python 2.7.9 haven't numpy. Do you understand?
Other thing your macOS have a groups of variables called "Environment Variables" there it's identifying the command python with python 2.7.9.
Then if you want to use numpy with the interpreter of python do you need change the Enviroment Variables for the command "python" take the version 2.7.6 or install numpy at python 2.7.9 and work with both versions.
Good luck!
With 2 different version you must have 2 python interpreters on your machine. They are installed on different locations with different libraries. So its just right that you cannot find the module installed on interpreter A while using the interpreter B. The which command can be useful to you to figure out where those interpreter are. You can make a symbolic link manually from a folder to another but it would mess up your mind, and you probably gonna get lost later.
I recommed you to install python and ipython via Homebrew, so it would automatically do all the hard work for you.
Install homebrew
ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"
Since you said numpy, you are probably looking for scientific stuff so:
# set up some taps and update brew
brew tap homebrew/science # a lot of cool formulae for scientific tools
brew tap homebrew/python # numpy, scipy, matplotlib, ...
brew update && brew upgrade
# install a brewed python
brew install python
Later you can run, but I recommend you to follow this tutorial:
brew install zmq
pip install ipython[all]
I was able to solve the problem by adding the following line to my .bash_profile:
export PYTHONPATH=${PYTHONPATH}:/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python
I then changed the Ipython version by following this stackoverflow question.
I'm having trouble importing matplotlib.pylab as I get the following:
In [2]: import matplotlib.pylab
Abort trap: 6
I have installed EPD
MacBook-Pro-2:bin$ which python
/Library/Frameworks/EPD64.framework/Versions/Current/bin/python
MacBook-Pro-2:bin$ which ipython
/Library/Frameworks/EPD64.framework/Versions/Current/bin/ipython
And when I run it it works as usual:
MacBook-Pro-2:bin$ ipython
Enthought Python Distribution -- www.enthought.com
Python 2.7.3 |EPD 7.3-2 (64-bit)| (default, Apr 12 2012, 11:14:05)
Type "copyright", "credits" or "license" for more information.
I also cloned matplotlib from github and installed it via the usual python setup.py install but I think my system can't work out which matplotlib to use?
MacBook-Pro-2:bin$ echo $PATH
/Library/Frameworks/EPD64.framework/Versions/Current/bin:
/Users/user/anaconda/bin:/opt/local/bin:/opt/local/sbin:
/opt/local/bin:/opt/local/sbin:/usr/local/share/python:
/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:
/opt/X11/bin:/usr/local/git/bin:/usr/texbin
I cut it down to:
/Library/Frameworks/EPD64.framework/Versions/Current/bin:
/usr/local/bin:
/usr/bin:/bin:
/opt/X11/bin:
/usr/local/git/bin:
/usr/texbin
but I still get the same error. How could I strip the above down so it just uses the necessary packages within EPD so it doesn't cause a conflict with other python/matplotlib installs? I've looked elsewhere on the web relating to the Abort Trap: 6 error but there isn't anything helpful so far.
Also when I install matplotlib from source It seems to want to write into:
/Library/Frameworks/EPD64.framework/Versions/7.3/lib/python2.7/site-packages/
I think this is the cause of the issue. I have manually written into the EPD distro the source matplotlib files.
One final thing which might be related:
I had to use my brew installed python to install wxWidgets (brew install wxWidgets) which installed the following:
/Library/Caches/Homebrew/wxmac-3.0.0.0.tar.bz2
Then I changed $PATH to link back to EPD since it did not have wx installed. I could not install wxWidgets using EPD ipython/python active for some reason. Would not using EPD for this install break anything?
I am using Ubuntu 12.04 64-bit, and started learning python today.(I tried to install a pirate version of MATLAB but failed...)
I have a linear programming problem to solve, and I want to use lp_solve module for Python.
I tried for 1~2 hours to find the download file and install the module.
I am not sure if I downloaded a right thing, and I could not install it until now.
How can I install this?
There is no download link in http://lpsolve.sourceforge.net/, and it tells me to run a command
python setup.py install
but there is no setup.py file in anywhere, including the lpsolve source file I downloaded somewhere.
If you know where to download it, and install it, could you teach me how to do them, step by step?
I am not sure about the version of my Python.
Thank you.
Adding few more details to the answer provided by dnozay.
Download the following two files from http://sourceforge.net/projects/lpsolve/files/lpsolve/
lp_solve_5.5.2.0_dev_ux64.tar.gz - contains the .so files
lp_solve_5.5.2.0_Python2.5_exe_ux64.tar.gz - contains the python wrapper scripts for lpsolver, which helps to invoke the native library from .so files.
Unzip the above downloaded files, where each directory formed by unzip will have an lpsolve55.so file, though at different locations.
Specify the paths to lpsolve55.so file in each directory by setting the following two environment variables:
export LD_LIBRARY_PATH=/usr/local/lib:/home/xxx/lp_solve_dev/
export PYTHONPATH=/home/xxx/usr/lib/python2.5/site-packages
To test if lpsolver is configured as expected :
[xx-xxxx#ip-xx-x-x-xx ~]$ python
>>>Python 2.7.9 (default, Apr 1 2015, 18:18:03)
[GCC 4.8.2 20140120 (Red Hat 4.8.2-16)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>from lpsolve55 import *
>>>lpsolve()
lpsolve Python Interface version 5.5.0.9
using lpsolve version 5.5.2.0
Usage: ret = lpsolve('functionname', arg1, arg2, ...)
P.S.: make sure you have installed python-dev (if not, type sudo apt-get install python-dev at the command line) before you do this all.
The download link is:
http://sourceforge.net/projects/lpsolve/, or
http://sourceforge.net/projects/lpsolve/files/lpsolve/ for the files tab.
Once you have it installed, you may need to tweak your PYTHONPATH.
You also may want to look into cvexp:
http://pypi.python.org/pypi/cvexp
I'm new to Linux/CentOS and I'm not having the most fun using it. I was developing something locally on my Mac using python 2.7 when I ran it on the server I ran into issues which I put down to being the outdated version of Python (2.4.3) I then updated it, which wasn't a very good idea. I built it from source and then ran into issues, Yum and other modules disappeared.
I'm not back using Python 2.4.3 but I still don't have Yum back.
[root#issues ~]# yum Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback There was a problem importing one of the
Python modules required to run yum. The error leading to this problem
was:
No module named yum
Please install a package which provides this module, or verify that
the module is installed correctly.
It's possible that the above module doesn't match the current version
of Python, which is:
2.4.3 (#1, Jun 18 2012, 08:55:31) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)]
If you cannot solve this problem yourself, please go to the yum faq
at: http://wiki.linux.duke.edu/YumFaq
Searching loads of forums everyone says that it's because of a different version of Python, this is no longer the case for me, I think it's something to do with Yum being mapped to a different directory.
I found this command somewhere else and ran this:
[root#issues ~]# rpm -V python
missing /usr/bin/python2
missing /usr/bin/python2.4
Does anyone know what to do or where to go for help? I really need to get this fixed as soon as possible!
Thanks
Actually just to add, If I run python I get this output.
[root#issues ~]# python
Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
'import site' failed; use -v for traceback
Python 2.4.3 (#1, Jun 18 2012, 08:55:31)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>
Yes on Linux systems, it is not a good idea to upgrade default python which is 2.4.x version and Linux package management and many other tools require it to function correctly. It is always better to install other python versions separately and use it directly (like: /usr/bin/python2.7)
Python executable itself is not working correctly as it is not able to load default module 'site' during start-up.
The python installation is broken.
Since you can not use "Yum" or "RPM" to bootstrap your packages, I suggest that you use your "OS" installation CD or ISO image to install the rpm/yum packages directly. It will fix the python installation as dependency.
I had gone through this issue and after various trial, this was only way I could fix it.
[Edit:]
See: Python 2.x multiple version issues regarding PYTHONPATH
Also the error is that it is not able to find the package directory.
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
See the paths that I have. Check out the path of packages and can you set "PYTHONHOME".
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /usr/lib64/python2.4/site.pyc matches /usr/lib64/python2.4/site.py
import site # precompiled from /usr/lib64/python2.4/site.pyc
# /usr/lib64/python2.4/os.pyc matches /usr/lib64/python2.4/os.py
import os # precompiled from /usr/lib64/python2.4/os.pyc
If you are not able to locate similar package location, can you use RPM to directly install python2.4 again
rpm --force -hUv url_for_python_rpm
I console access to a computer where I do not have root nor sudo rights.
Python version is 2.5.2 and numpy is not available. I cannot use python setup.py install --user nor there are any compilers available on the machine.
Can I somehow use the compiled packages available https://edge.launchpad.net/~scipy/+archive/ppa/+packages without installing them? I tried importing the numpy module directly but it complains:
Python 2.5.2 (r252:60911, Jan 4 2009, 21:59:32)
[GCC 4.3.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/home/XXX/temp/python-numpy-1.2.1/numpy/__init__.py", line 121,
in <module>
raise ImportError(msg)
ImportError: Error importing numpy: you should not try to import numpy from
its source directory; please exit the numpy source tree, and relaunch
your python intepreter from there.
>>>
Thanks!
Update:
The sysadmin will not install any kind of software in the machine (it's a VPS and my guess is that they have a standard image for deployment). They are crazy paranoic, they won't even tell me what flavor of unix they are running, and even the Apache service has the version number obfuscated! This is all the information I get upon login:
Linux server1 2.4.22 #4 SMP Wed Nov 5 17:44:16 CET 2003 i686 unknown
And for compiling:
python setup.py install --home=~
[...]
RuntimeError: Broken toolchain: cannot link a simple C program
cat /proc/version
Linux version 2.6.32.25-grsec-dh (root#dl345.dinaserver.com) (gcc version 4.3.2
(Debian 4.3.2-1.1) ) #2 SMP Wed Nov 3 13:21:01 CET 2010
If you can resolve all the dependencies, you might be able to install it in your $HOME using dpkg. dpkg doesn't resolve dependencies automatically, so you might have to figure out the right order to install the packages in. Download the .deb files that you're interested in and run the following command for each package:
$ dpkg -i --force-not-root --root=$HOME mypackagename.deb
If you then add the directory with the newly installed Numpy to your $PYTHONPATH, or to sys.path, Numpy might just work.
Alternatively, you might be able to extract the files you need from one of the other binary distributions of Numpy around (such as Sage).
Numpy is quite fussy about what versions of its dependencies it requires though, so you're probably best off downloading the packages for the specific version of Linux that you're using.
Finally, consider asking your administrator whether s/he'll install Numpy for you. You'd be surprised how often a simple request can solve all your problems, especially since it's just one apt-get command.
EDIT: Just as an alternative, if you can get access to another machine running the same version/architecture of Ubuntu/Debian, you might be able to download the numpy source tarball, compile with python setup.py build and then just copy everything in directory_where_you_extracted_the_tarball/build/numpy/lib.OS-arch-PythonVersion (on my system, it is lib.linux-x86_64-2.6/) to a directory of your choice on the target machine. Then, just add that directory to your $PYTHONPATH and you're done. Remember to copy the contents, not the whole directory (tar -jcf np.tar.bz2 /path/to/numpy/build/numpy/lib.OS-arch-PythonVersion/numpy then get the tar.bz2 to the remote machine and extract it in a directory of your choice).
There is some documentation on how to use setuptools here: http://docs.python.org/install/index.html#how-installation-works
Building Numpy by hand is not for the faint of heart though, so this might lead to a lot of head-banging and hair-tearing.
I'm not 100% this will work, but Enthought has a free version of the EPD that has numpy and scipy included, that may not require a compiler to install (since it's just installing binaries as far as I can tell), and doesn't need root access:
http://www.enthought.com/products/epd_free.php
You could try setting up a virtualenv environment on a similar machine with similar architecture. Then install virtualenv locally on the VPS machine and try copying the environment there.
You can use python's distutils (which is what python setup.py runs) install to a local directory, which must be added to your PYTHONPATH. E.G.,
python setup.py install --prefix=~/local
which uses a directory hierarchy ~/local/lib/python2.x. (Or you can use --home=<dir> to avoid the python2.x part)