If I install several packages with Python 2.6 (e.g. using easy_install) and then I want to upgrade to Python 2.7, is there a way to upgrade Python and then automatically "import" all those installed packages along with it? Or do they have to be reinstalled?
Two related questions: (1) if a package is installed in a Python 2.6 packages directory, is it legitimate to import it into the PYTHONPATH of a newer Python, like Python 2.7, or must all the packages be reinstalled with Python 2.7? (2) if I use easy_install, how can I tell it to use the newer Python? E.g. 2.7 instead of 2.6? Or should I just reinstall easy_install using Python 2.7 to do this? thanks.
First, this is one of the many reasons you want to use pip instead of easy_install. (You still need easy_install to get pip itself, but beyond that, don't touch it ever again.) If you'd used pip, you could just do this:
pip freeze > modules.dump
That gives you a list of all of the modules you have installed, and their version numbers. Most of the time, you can just take the list of modules (line.split('==')[0] for line in f) and pass it to pip install.
But that's for future reference. For today, you have to piece it together yourself by looking through your site-packages directory. Fortunately, many things will end up as foo_bar-1.2.3.4-blah-blah.egg, so all you have to do is guess whether the package is named foo-bar or foo_bar at PyPI, and usually even if you guess wrong, easy_install or pip will get the right thing anyway. So, you can't quite automate it, but you can get close.
But yes, however you do it, you do need to reinstall. Anything that requires C extension code has to be recompiled. Pure-Python packages may not need to be changed, but they may, and you're better safe than sorry. Also, if you try to copy some things over but not others, you're going to make a big mess of your dependencies.
(1) if a package is installed in a Python 2.6 packages directory, is it legitimate to import it into the PYTHONPATH of a newer Python, like Python 2.7, or must all the packages be reinstalled with Python 2.7?
Don't do that; reinstall them, as explained above.
(2) if I use easy_install, how can I tell it to use the newer Python? E.g. 2.7 instead of 2.6? Or should I just reinstall easy_install using Python 2.7 to do this? thanks.
You need the 2.7 easy_install. You can usually use a 2.7 easy_install with 2.6 by running, e.g., python2.6 $(which easy_install), but the other way around isn't guaranteed to work.
And you don't want to do that anyway. If you want two versions of Python in parallel, you want two versions of easy_install—normally you want to end up with easy_install-2.6 and easy_install-2.7, with easy_install as a symlink to whichever one you consider your "primary" python.
Related
Background
Currently, I have Python 2.7.17 and Python 3.8.1 installed on my Mac (v 10.14.6). Both Python versions were downloaded directly from Python.org using a "macOS 64-bit installer" .pkg file, in late 2019. Both were installed here:
python2: /Library/Frameworks/Python.framework/Versions/2.7/bin/python2
python3: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3
Currently, I use only pip (either pip2 or pip3, depending) to install python packages, which are saved here:
python2: /Library/Frameworks/Python.framework/Versions/X.X/lib/python2.7/site-packages
python3: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
I am about to begin an online course which requires that I install miniconda in order to install the packages that I need for the course. But after having a very bad experience having Anaconda installed on my machine years ago, and it taking me months to remove it and re-create a clean Python install, I am nervous. I don't want my machine to become a mess again, rife with error messages that it can't find this or that python version or this or that python library.
Question(s)
Assuming there will now be multiple installations of Python on my machine (in addition to the versions that I already have, since I believe yet another version of python is installed along with conda), and there will be new places where python packages will be saved, what can I expect to change about how I write python after I install miniconda?
Is there anything I will need to do differently from what I am doing now when I write python code?
How can I ensure that everything remains working and compatible and I avoid the fate I suffered many years ago?
Happy to edit/clarify this question per user suggestions. Thanks!
Use conda environments and install Python into those environments. This ignores, by design, the versions of Python that are installed at the system level. Also, what's installed into environments (other than the one you're in) is ignored. As somebody who uses conda for all Python development, I don't even know what I have installed in /usr/local/bin/ or other places. Everything is in /path/to/miniconda3/envs/env_name_n. You don't have to uninstall other versions of Python you have elsewhere; the point of environments is to keep everything isolated.
What can I expect to change about how I write python after I install miniconda?
Is there anything I will need to do differently from what I am doing now when I write python code?
Use conda environments and use conda as your package manager. For almost every major project, it nearly a drop-in replacement for pip. Outside of managing dependencies, not much changes.
Recently I've installed Python 3 after using 2 for quite a while, so I have many libraries for Python that I've installed using pip. I've already started using the symlink python3, but what else do I need to do to change over?
For example, while I have made the alias python=python3, I don't know about the modules I've installed with pip. I was using a version of pip for Python 2, but does that mean modules were written for Python 2 and not 3? Do I now need to start using pip3 (or make a corresponding alias pip=pip3 or pip=pip3.5)? If modules were installed using the command pip, does that mean I have to reinstall them for Python 3 using pip3? If I do need to reinstall them for 3, should I start removing them from my Python 2 env as cleanup?
And another question that seems like it may entail a lot of work: How should I change all my virtual environments for Python projects? They were all Python 2 envs, and now I'm not sure whether it's necessary, useful, or possible to reinstall or upgrade the modules within them to the corresponding ones for Python 3.
Sorry if this seems like a duplicate question - I've searched and only found resources regarding "how to upgrade" rather than "what to do after upgrading". Thanks for your help!
It is strongly recommended not to symlink python to Python 3, at least on Debian-based Linux distros. Debian utility scripts use both Python 3 and 2.7 to work (which means you already had a copy of Python 3 somewhere...), and as a consequence Debian-based distributions ship both versions. Symlinking can cause unexpected breakages. This may or may not be true on other systems, however, but with such ambiguity it is probably useful not to try.
Instead, live with symlinking python3 to Python 3. Leave python alone.
pip is usually symlinked to the corresponding pip program for 2.7. Use pip3 to specify you want to use the pip program for Python 3. If you use pip, you'll find modules for Python 3 being installed to Python 2.7 site directories and nothing will work as you expect it to. There should be no reason why you can't symlink pip to pip3, but I'd still not recommend it - it's better to tread cautiously in this regard.
You don't need to change your current virtual environments. Just create new ones with virtualenv, pointing to the correct Python program to use. This is the whole point of virtual environments: to sandbox different Python versions, so that neither of them adversely interact with each other.
If you have projects you really want to ship to Python 3, I'd recommend using a version control system to back up your current files, and then reproducing them in a new virtual environment configured for Python 3. This is quick, simple and painless.
I want to completely reinstall Python 2 but none of the guides I have found allow me to uninstall it. No matter what I do, python --version still returns 2.7.10, even after I run the Python 2.7.11 installer. All the other guides on StackOverflow tell me to remove a bunch of files, but python is still there.
This may be a bit late, but for future searchers I'll post anyway:
I was looking to do the same. But I came across this paragraph at the Foundation (Getting and uninstalling MacPython) which convinced me to leave well alone and not uninstall it.
The Apple-provided build of Python is installed in /System/Library/Frameworks/Python.framework and /usr/bin/python, respectively. You should never modify or delete these, as they are Apple-controlled and are used by Apple- or third-party software. Remember that if you choose to install a newer Python version from python.org, you will have two different but functional Python installations on your computer, so it will be important that your paths and usages are consistent with what you want to do.
Set your an alias to use the python version that you want to use from inside your .bashrc (or zsh if you use it).
Like:
alias python='/usr/bin/python3.4'
Agree with the accepted answer that uninstalling is a bad idea, but for those of you using HomeBrew to install your own Python, you don't need an alias as in #Mat Marsiglio's answer. Rather you can do what the HomeBrew installation suggestions:
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
This gives brew's python precedence over the built-in one at /usr/bin/python
I use python 2.7 and have the following packages installed:
distribute
decorator
*matplotlib
memory-profiler
networkx
*numpy
Pillow
pip
py2exe
PyAudio
*PyBluez
*pygame
pyglet
*PyInstaller
pyparsing
*pyserial
python-dateutil
pytz
pywin32
requests
scikit-learn
*scipy
setuptools
six
The ones marked with a * are critical.
I want to move to python 3.5, abandoning 2.7 completely (don't want to have both).
Can I do that while still keeping my old packages or do I have to install everything from scratch?
Are there any reasons I shouldn't move to py 3.5 from 2.7?
Should I move first to 3.3/3.4 or just straight to 3.5?
I'm not using a virtualenv because I'm not very familiar with that. Should I?
UPDATE: Follow-up.
I can just make a list of the packages I have and then manually install all of them.
How do I go about installing 3.5 and uninstalling 2.7?
Do I remove 2.7 first then install 3.5 or have them both simultaneously (will this create any issue?) then remove 2.7?
I mainly use python for numpy, bluetooth, serial, OpenCV, OpenGL. I intend to foray into a bit of web too later. I don't have any issue with syntax or method change. I was previously holding out for pyinstaller but that too has been recently ported to 3.5.
Just realized after switching.
Don't install python 3.5 if you want OpenCV to work. It's not yet built for that and some functions don't work.
You will need to reinstall all your packages. You should check that all those packages are available for Python 3. (From your question it sounds like you may have already done this, but not totally sure.)
It would be wise to test how things go with Python 3 before completely abandoning Python 2. You can do this by installing the two side-by-side, or by installing Python 3 in a virtual machine or some such thing. If you have existing code you want to keep using, you definitely want to test it to make sure everything runs smoothly on Python 3.
There is likely no reason to go to 3.3/3.4. Just go straight to 3.5.
Virtualenv can definitely be useful for setting up different Python environments on the same computer. However, you'll still need to install Python 3 before you can use it in a virtualenv.
Moving packages is bad idea, it is safier to reinstall them using pip:
Use pip freeze > requirements.txt. It will store all packages and versions in file
Install Python 3.5
Run pip install -r requirements.txt. It will install same packages on your python3.5
Run all tests against this Python to make sure your app still works as expected
The only reason to stay with 2.7 is incompatible code: if your code or one of your packages does not work on Py3K.
Virtualenv is useful tool, any python developer should know how to use it.
I have, in /usr/local/lib/python2.7/site-packages multiple versions of the same package.
E.g. I have django-angular-0.7.13-py2.6.egg, django-angular-0.7.13-py2.6.egg-info, and django-angular-0.7.13-py2.7.egg.
Is it safe to delete the two files that are, ostensibly, the wrong version?
When I got into the python interpreter, import the package/module, it tells me it's run from <module 'django_angular' from '/usr/local/lib/python2.7/site-packages/django-angular-0.7.13-py2.6.egg/django_angular/__init__.py'>...
I'm concern I'll irreparably damage my python packages, as is so easy when you mess with these things.
All I could find on the topic is this article, but that's windows specific, and doesn't address having multiple versions of the same thing.
You shouldn't go and delete the files manually, although it is probably safe. It is quite unlikely that your system depends on Django and Python always uses only one version of the installed packages anyway.
However, I would strongly recommend keeping your development environment separated from your system packages. If you haven't already, take a look at pip and virtualenv. Here is one tutorial for them.
With pip you can also uninstall your system libraries if you really want to do that. However, pip only sees one version at a time, so if you want to uninstall the above packages you have to run pip uninstall multiple times. But after all the versions are gone you can install the version you really want.
Also, a better option for displaying the installed versions is to use yolk. With that you don't have to browse the site-packages manually.