I have read a bunch of posts here and on Google, but my question is far more basic than the answers: If Python(2.7) came pre-installed on my MacBook Pro (High Sierra), can I just do sudo easy_install pip (as suggested) from the command line--withOUT causing issues? I have a vague understanding of global/local installations, and my understanding is certain Python installations aren't compatible with local/global kernel installations. I hope I am getting the terminology right, but I saw several warnings about installing pip for "a homebrew based python installation", but I am not sure whether Python on my laptop is installed via homebrew (nor how to find out).
My question came about because I wanted to install the Hydrogen package to use in Atom, the text editor (to help me learn Python). I finally succeeded in installing Hydrogen, but got stumped by the missing kernels (not sure which ones I need, so I am willing to install them all). But I can't seem to install the kernels without pip. So here I am.
My apologies for asking such a basic question--and thanks!
The rule of thumb is: If your operating system has a package manager, use it.
Unfortunately, MacOS is the only UNIX-like operating system that does not come with a decent package managment system.
(There is the app store, but that is useless for a lot open source software for several different reasons. It's also a walled garden.)
You have several choices (in descreasing suitability):
Use one of the package managers available for MacOS. Which one is the best choice for you depends on all the packages you need being available.
Use a Python distribution. I've used Anaconda on ms-windows, and that has saved me a lot of hassle. A good choice if you are only looking for Python and related libraries.
Build everything yourself. This can be very time-consuming and is a duplication of effort. You will learn a lot though.
I would second Piinthesky's comment that you install Python 3.6. Python 2.7 is now a legacy version.
Well although I am no Mac expert I've given it a shot anyway:
Yes you could but do you really want to risk it (or even do it)?
Mac-OS must rely on Python to fulfill something in the OS otherwise it would not come inbuilt. This means two things:
The Python installation will be minimal. By that I mean it will have things missing (any large library for a start). They will do this mainly to cut down on the OS size. Therefore you will not have the full Python library and in the long term you may end up missing out.
Second if anything went wrong (IE you broke your installation or even modified it -yep I've done this in Linux and have ended up factory resetting) then you may cause something to stop working and may need to factory reset or perform some other drastic action on your OS. A separate installation would prevent your from risking this. This is very useful because there comes a time when you may decide to update certain modules with pip and find it can't or it updates something that you shouldn't be messing with.
Yes it's possible you may run into compatibility problems but I think it's most widely accepted that you do not use the inbuilt one as it needs to remain unchanged if the OS is to use it correctly. Messing with it increases the chances of it breaking.
Conclusion: So even though installing modules with pip (and getting pip) can be done with the inbuilt Python it comes down whether you want to risk harming your OS. I strongly suggest you get a separate installation and leave the inbuilt one as it is. Second as you mentioned you will find that the inbuilt versions are never up-to-date or are built were they are not really compatible with standard libraries (expect things like the missing runtime libraries all the time) , just another reason to stay clear of them.
This is how I solved this problem-for those newbies who just want Hydrogen to work:
Installed Python 3 (instead of messing around with Python 2.7 and pip).
Followed instructions here (https://packaging.python.org/tutorials/installing-packages/#ensure-you-can-run-pip-from-the-command-line) for 'get-pip.py'.
In Atom, cmd+shift+p to bring up the packages menu, clicked on 'Hydrogen Run', which gave me the errors again.
Copied the code from the warnings and installed the kernels needed (via the command line).
Hydrogen is now working.
Thanks for all the tips!
Related
The development environment, we use, is FreeBSD. We are evaluating Python for developing some tools/utilities. I am trying to figure out if all/most python packages are available for FreeBSD.
I tried using a CentOS/Ubuntu and it was fairly easy to install python as well as packages (using pip). On FreeBSD, it was not as easy but may be I'm not using the correct steps or am missing something.
We've some tools/utilities on FreeBSD that run locally and I want Python to interact with them - hence, FreeBSD.
Any inputs/pointers would be really appreciated.
Regards
Sharad
The assumption that powerful and high-profile existing python tools use a lot of different python packages almost always holds true. We use FreeBSD in our company for quite some time together with a lot of python based tools (web frameworks, py-supervisor, etc.) and we never ran into the issue that a certain tool would not run on freeBSD or not be available for freeBSD.
So to answer your question:
Yes, all/most python packages are available on FreeBSD
One caveat:
The freeBSD ports system is really great and will manage all compatibility and dependency issues for you. If you are using it (you probably should), then you might want to avoid pip. We had a problem in the past where the package manager for ruby did not really play well with the ports database and installed a lot of incompatible gems. This was a temporary issue with rubygems but gave us a real headache. We tend to install everything from ports since then and try to avoid 3rd party package managers like composer, pip, gems, etc. Often the ports invoke the package managers but with some additional arguments so they ensure not to break dependencies.
Is Python support for FreeBSD as good as for say CentOS/Ubuntu/other linux flavors?
It is probably better than on other OSes, but I'm a FreeBSD-bigot.
However! As Freitags says, you do not want to use pip (nor gem, I might add). All of these language-specific packaging systems were born out of developers' frustration with the various inadequacies of OS-specific packagers.
Had the world been using BSD, pip (nor gem) would've been unnecessary.
Why am singing this pean here? To warn you, that you might not find some obscure Python package already ported -- despite being available via pip. Packages of any prominence are ported (here is the current list), but something less known might not be.
Do not despair -- create a port yourself using any of the existing examples and FreeBSD Handbook. It is very easy to do and, if you submit it to FreeBSD, it will already be there the next time you need it.
Best of luck.
I'm at a bit of despair.
I've spent the last two days trying to set-up a proper Python (2.7x) Windows enviornment.
I tried a bunch of different "bundles", and it boils down to this:
It's either I can't install the packages I want;
or I can't use the IDE I want.
I tried a simple install of "python" with no additions.
Very early, I come to a stop while using pip/easy_install, or exe files to install additional packages.
I then tried "Portable Python" which weathered much better.
It has a nice (though limited) bunch of packages.
I can use it with any IDE I want (usually PyDev/Eclipse).
Here too, It's practically impossible (for me, following what I read) to install most packages.
I then went with Enthought Canopy (formerly EPD).
This has a superb tool for package managing. I did not find any problem, with package handling.
However, it comes with a very limited (almost insulting) IDE.
All my actions to point eclipse interpreter to the Canopy directories, went awry.
Lastly, I tried WinPython.
This is somewhere in-between "Portable" and "Canopy".
It has more packages pre-bundled, than "portable" has, but less than canopy.
It also has a package management GUI, but unlike canopy's, this one fails.
Please, tell me how you set-up a Win/Python extendable enviornment.
And please, do not flag this as a duplicate - I have read dozens of related questions here on SO, and elsewhere. Non actually helped me.
I would not be wasting your times, unless I tried all paths before (obviously using SO and Google to solve my problems before going through another lengthy install).
[EDIT (SOLVED) ]:
I uninstalled everything, and then installed "plain" python.
I also manually removed all references to orphaned "pythons" of past from the PATH and PYTHONHOME env vars.
That seems to work.
I finally managed to install numpy, but it seems to only work in python2.6 . I don't know how to install it in the 2.7 folder (been trying for hours, but I'm just a beginner developer in my first months). Anyway, if I use Python 2.7 and append the absolute path to sys.path, could there be problems?
Any suggestions?
Thank you.
It could partially work but this is a bad idea. Just don't do it. Even if it seems to work, it may not. And if it really does, then it will fail randomly in the future.
These are the potential problems that come into my mind:
Extensions (those written in C, C++ etc.) are specific to a particular Python version; and numpy has a few extensions, AFAICS. It will work only if you don't use any of them (i.e. use pure Python modules);
Python compiles modules into bytecode. The bytecode is specific to a particular Python version. If you use modules from python2.6 directory in python2.7, the compiled files will collide. I doubt this will cause major problems except for the fact that they will be recompiled every time Python version is switched;
Python code can be version-specific. It's unlikely for minor versions (but for example Python 2/3 could have serious differences) but still can happen. In other words, the modules installed for Python2.6 can be actually a bit different than those for Python2.7;
If you change the loading order, Python2.7 may start loading some standard modules from Python2.6. it could work, it could cause random breakages;
It will make all modules installed for Python2.6 visible. It can cause a few random switches somewhere with unpredictable result. I doubt there's something specific for that version but some modules may actually decide to use some kind of deprecated interface finding it visible.
There could be more. You may actually try but be prepared that you may waste a lot of time trying to find out why something does not work as expected later.
And unless I'm missing something, I think installing numpy for Python2.7 involves mostly running the setup using Python 2.7; like:
python2.7 setup.py clean
python2.7 setup.py build
python2.7 setup.py install
Depending on your particular install/system, it may be preferable to use the package manager, binary bundle or a tool like pip instead. If you'd like more details on how to enforce Python2.7 with those, you'd have to tell us which one is of your choice.
So, id like to start serious python development, and its proven to be a big pain. Im not worried at all about the language itself; I like it well enough and I will have no problems picking it up. But the ecosystem is driving me crazy.
First I tried to get up and running under windows. I gave up on that after a few days, as 90% of packages dont include windows support / install instructions. So I switched to macosx, which people said was good for mac development.
More frustration ensues. Id like to use python as a matlab replacement and tool development platform, so spyderlib seems like an excellent tool. But now ive been busy trying to build pyqt on my mac for two days, to no avail, and im starting to question the wisdom of it all. Obviously, following several guides literally invariantly ends in cryptic errors. For which platform was this dependency built? What arcane compiler flags need to be set? I dont know and I dont care; why doesnt the installer figure it out? Oh wait, there isnt any... I want to USE these tools, not first completely reverse engineer them to find out how to build them.
There is a vast amount of implied knowledge in all the documentation I can find on these matters, both with regard to unix and pythonic quirks. Is there any way to scale this mountain, in a place with a managable learning curve? Right now I have no idea what im doing. Or should I go back to windows and try to coerce the unix packages I need into cooperation?
On Mac OS X, you can get spyder with macports. This should build everything needed.
If you prefer Windows, take a look at Python(x,y). It has a bunch of scientific tools pre-built, including spyder.
Finally, the Enthought Python distribution is worth considering for scientific work.
Have you tried ActivePython?
Why battle with compiling the modules yourself when you can get the pre-built packages from PyPM?
pypm install pyqt4 matplotlib scipy numpy
From my experience the best platform for kind of project you're describing is Linux. There you just install the libs you need from package manager and that's it. Binary packages, so compiling is not required.
If you want to stick with MacOS X, you should install either MacPorts or Fink. It's usually easy to use. Problem is, that things like Qt take forever to compile. But you won't be doing that very often.
As for installing Python modules, the best is PIP, which is very nice replacement for easy_install did does much more. Especially useful if you want virtualenv setup.
This is nearly the exact opposite of my experience with Python on Windows. Python itself installs with a binary installer, most add-on packages support easy_install, others provide binary installers of their own. The IDE I use is SciTE, which uses the old DOS install model - copy the files to a directory and run the SciTE.exe file. If you get a source distribution of a Python package, go to the directory containing setup.py and run python setup.py install. Maybe that's the implied knowledge you're talking about.
You can also find many unofficial Windows binaries at http://www.lfd.uci.edu/~gohlke/pythonlibs/.
I switched to Mac a few years ago and found that it took me quite a while of googling to properly install all the packages I needed for Python development. While I installed everything I made a list of the steps required to setup a functional system that may be appropriate for you as well. I usually use NetCDF4, HDF5, Numpy, Matplotlib, f2py, and Fortran in combination with Python. I published my list of 22 setup-steps for installing from source on my website. Installing from source is somewhat more time-consuming than using macports and fink, but enables you to have a working environment that is optimized for your system.
short version: how can I get rid of the multiple-versions-of-python nightmare ?
long version: over the years, I've used several versions of python, and what is worse, several extensions to python (e.g. pygame, pylab, wxPython...). Each time it was on a different setup, with different OSes, sometimes different architectures (like my old PowerPC mac).
Nowadays I'm using a mac (OSX 10.6 on x86-64) and it's a dependency nightmare each time I want to revive script older than a few months. Python itself already comes in three different flavours in /usr/bin (2.5, 2.6, 3.1), but I had to install 2.4 from macports for pygame, something else (cannot remember what) forced me to install all three others from macports as well, so at the end of the day I'm the happy owner of seven (!) instances of python on my system.
But that's not the problem, the problem is, none of them has the right (i.e. same set of) libraries installed, some of them are 32bits, some 64bits, and now I'm pretty much lost.
For example right now I'm trying to run a three-year-old script (not written by me) which used to use matplotlib/numpy to draw a real-time plot within a rectangle of a wxwidgets window. But I'm failing miserably: py26-wxpython from macports won't install, stock python has wxwidgets included but also has some conflict between 32 bits and 64 bits, and it doesn't have numpy... what a mess !
Obviously, I'm doing things the wrong way. How do you usally cope with all that chaos ?
I solve this using virtualenv. I sympathise with wanting to avoid further layers of nightmare abstraction, but virtualenv is actually amazingly clean and simple to use. You literally do this (command line, Linux):
virtualenv my_env
This creates a new python binary and library location, and symlinks to your existing system libraries by default. Then, to switch paths to use the new environment, you do this:
source my_env/bin/activate
That's it. Now if you install modules (e.g. with easy_install), they get installed to the lib directory of the my_env directory. They don't interfere with existing libraries, you don't get weird conflicts, stuff doesn't stop working in your old environment. They're completely isolated.
To exit the environment, just do
deactivate
If you decide you made a mistake with an installation, or you don't want that environment anymore, just delete the directory:
rm -rf my_env
And you're done. It's really that simple.
virtualenv is great. ;)
Some tips:
on Mac OS X, use only the python installation in /Library/Frameworks/Python.framework.
whenever you use numpy/scipy/matplotlib, install the enthought python distribution
use virtualenv and virtualenvwrapper to keep those "system" installations pristine; ideally use one virtual environment per project, so each project's dependencies are fulfilled. And, yes, that means potentially a lot of code will be replicated in the various virtual envs.
That seems like a bigger mess indeed, but at least things work that way. Basically, if one of the projects works in a virtualenv, it will keep working no matter what upgrades you perform, since you never change the "system" installs.
Take a look at virtualenv.
What I usually do is trying to (progressively) keep up with the Python versions as they come along (and once all of the external dependencies have correct versions available).
Most of the time the Python code itself can be transferred as-is with only minor needed modifications.
My biggest Python project # work (15.000+ LOC) is now on Python 2.6 a few months (upgrading everything from Python 2.5 did take most of a day due to installing / checking 10+ dependencies...)
In general I think this is the best strategy with most of the interdependent components in the free software stack (think the dependencies in the linux software repositories): keep your versions (semi)-current (or at least: progressing at the same pace).
install the python versions you need, better if from sources
when you write a script, include the full python version into it (such as #!/usr/local/bin/python2.6)
I can't see what could go wrong.
If something does, it's probably macports fault anyway, not yours (one of the reasons I don't use macports anymore).
I know I'm probably missing something and this will get downvoted, but please leave at least a little comment in that case, thanks :)
I use the MacPorts version for everything, but as you note a lot of the default versions are bizarrely old. For example vim omnicomplete in Snow Leopard has python25 as a dependency. A lot of python related ports have old dependencies but you can usually flag the newer version at build time, for example port install vim +python26 instead of port install vim +python. Do a dry run before installing anything to see if you are pulling, for example, the whole of python24 when it isn't necessary. Check portfiles often because the naming convention as Darwin ports was getting off the ground left something to be desired. In practice I just leave everything in the default /opt... folders of MacPorts, including a copy of the entire framework with duplicates of PyObjC, etc., and just stick with one version at a time, retaining the option to return to the system default if things break unexpectedly. Which is all perhaps a bit too much work to avoid using virtualenv, which I've been meaning to get around to using.
I've had good luck using Buildout. You set up a list of which eggs and which versions you want. Buildout then downloads and installs private versions of each for you. It makes a private "python" binary with all the eggs already installed. A local "nosetests" makes things easy to debug. You can extend the build with your own functions.
On the down side, Buildout can be quite mysterious. Do "buildout -vvvv" for a while to see exactly what it's doing and why.
http://www.buildout.org/docs/tutorial.html
At least under Linux, multiple pythons can co-exist fairly happily. I use Python 2.6 on a CentOS system that needs Python 2.4 to be the default for various system things. I simply compiled and installed python 2.6 into a separate directory tree (and added the appropriate bin directory to my path) which was fairly painless. It's then invoked by typing "python2.6".
Once you have separate pythons up and running, installing libraries for a specific version is straightforward. If you invoke the setup.py script with the python you want, it will be installed in directories appropriate to that python, and scripts will be installed in the same directory as the python executable itself and will automagically use the correct python when invoked.
I also try to avoid using too many libraries. When I only need one or two functions from a library (eg scipy), I'll often see if I can just copy them to my own project.