I'm running MacOS Sierra 10.12.6
By default the system came with Python 2.7.10
I installed Python 3.6.3 (with IDLE) so I can learn Python (3). I understand that this is normal as MacOS may rely on Python 2.x for some programs. Either way, Python3 runs just fine if I run python3 from the command line/terminal, or if I use IDLE (which defaults to Python 3).
Now I want to install some libraries like Beautiful Soup.
And I believe I can install it as follows:
pip3 install beautifulsoup4
which should automatically install it. However, I read that it's recommended to use virtualenv on Mac BEFORE I run the above command. As a newbie, I don't want to mess anything up on my PC, so can anyone point me out how I can do this correctly?
For example, I can follow this link: http://sourabhbajaj.com/mac-setup/Python/virtualenv.html
But I just want to write here to make sure I'm following the right article/commands before I do it. Just being super careful!
Also, can I make a folder with my "virtual environment" and then add sub-folders inside that for each project? Meaning, I don't need to do this everytime, I have one virtual environment and any project that I do just is a subfolder within that space so I can use any libraries that I installed. Just trying to grasp the concept.
Thanks!
Sorry to add confusion.. this can be a tough subject for someone starting out.
The official docs recommend venv, which is similar to, but slightly different than virtualenv.
I would strongly recommend pycharm. It will create your venv for you as part of your project, which you might find helpful.
[Edit: Some other virtual environment features of pycharm that will help you].
If you type in an import statement for a package that isn't installed, it will offer to install it for you.
typing alt-F12 will bring you up a console with your virtual environment active
It syncs up your requirements.txt document for you
It manages your virtual environment path for you (as long as you are running inside pycharm), helping avoid import problems that many newcomers have with virtual environments.
I am not affiliated with pycharm, btw -- I just think it is a great tool for python developers, especially for newcomers, and its treatment of virtual environments is especially helpful.
You create one virtualenv for each project as a way of keeping track of the specific dependencies to keep them minimal which then makes it easier when you want to share projects with other people.
But this is not something you need. No harm comes from installing packages in your real environment as well. So you can safely run
pip3 install beautifulsoup4
Related
I’m trying to figure out how to install a second python environment alongside anaconda.
On windows I can just install python in a different folder stand reference the desired python environment using env variables. I’d like to do the same on Mac.
A virtual env won’t do the trick as it does not copy the standard library and other things. It needs to be a complete stand alone environment. I guess I could compile it, but is there an easier way?
Thank you very much for any input.
You can do that using pyenv.
It allows you to have several python versions, and even different distributions.
It works, mostly on user space. So, no additional requirements are needed (apart from compilation tools)
Being a "normal" user (i.e. not overly familiar with code), I find myself in a bit of a python conundrum and would very much like to try to muster some kind of assistance with the following:
I use svtplay-dl and youtube-dl via homebrew in Mac OSX 10.14.6, which also requires python (among many other extensions such as ffmpeg et al). However, I now seem to have three different python versions installed, and since python3.5 is no longer supported, I thought I would delete this and make sure that homebrew/svtplay-dl/youtube-dl uses a path to python3.9.
When I type "which" followed by the following in Terminal:
python
python3.5
python3.9
...I get the following respective results:
/usr/local/bin/python
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
/usr/local/bin/python3.9
Based upon this short background info, can someone advise how I best uninstall/remove python3.5 and get homebrew et al to use python3.9 every time python is required?
Any help with this would be very much appreciated!
/Mark
The normal answer is to use something like virtualenv (https://virtualenv.pypa.io/en/latest/) to isolate your python environment and dependencies locally.
You need to install virtualenv on your system, then you tell it what python version to install. Once activated, any dependencies are installed and isolated in that environment. Also see: https://gist.github.com/pandafulmanda/730a9355e088a9970b18275cb9eadef3
If you want to go a step further you can use docker (https://docs.docker.com/language/python/build-images/) to isolate everything, including the operating system, in that case you don't need virtualenv, but to start with I'd play around with virtualenv.
thanks for reading this. I am using macOS High Sierra. I am not very familiar with terminal or environment variables, but am trying to learn more. From reading other threads and google, it seems like I either have multiple pythons installed, or have pythons running from different paths. However I am not able to find a solution to resolving this, either by re-pathing my IDLE or deleting it entirely.
I do have python, python launcher, and anaconda (not very sure how anaconda works, have it installed a few years back and didn't touch it) installed. I am trying to install pandas (pip install pandas), which tells me that I have it installed, but when I run it on IDLE, it says module not found. Though if i run python3 on terminal and type my code in, it works (so pandas has indeed been installed).
When i run which python on terminal, it returns
/Users/myname/anaconda3/bin/python
(when i enter into this directory from terminal, it shows that in the bin folder, I have python, python.app, python3, python3-config, python3.7, python3.7-config, python3.7m, python3.7m-config)
When i run which idle on terminal, it returns
/usr/bin/idle (im not even sure how to find this directory from the terminal)
When i run import os; print(os.path) on IDLE, it returns module 'posixpath' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py'
Would really appreciate some help to figure out how to ensure that when i install modules from terminal, it would be installed into the same python as the one IDLE is using. Also, I would like to know whether it is possible for me to work on VSCode instead of IDLE. I cant seem to find suitable extensions for data science and its related modules (like statsmodels, pandas etc). Thanks a lot!
First of all, a quick description of anaconda:
Anaconda is meant to help you manage multiple python "environments", each one potentially having its own python version and installed packages (with their own respective versions). This is really useful in cases where you would like multiple python versions for different tasks or when there is some conflict in versions of packages, required by other ones. By default, anaconda creates a "base" environment with a specific python version, IDLE and pip. Also, anaconda provides an improved way (with respect to pip) of installing and managing packages via the command conda install <package-name>.
For the rest, I will be using the word "vanilla" to refer to the python/installation that you manually set up, independent of anaconda.
Explanation of the problem:
Now, the problem arises since you also installed python independently. The details of the problem depend on how exactly you set up both python and anaconda, so I cannot tell you exactly what went wrong. Also, I am not an OSX user, so I have no idea how python is installed and what it downloads/sets alongside.
By your description however, it seems that the "vanilla" python installation did not overwrite neither your anaconda python nor anaconda's pip, but it did install IDLE and set it up to use this new python.
So right now, when you are downloading something via pip, only the python from anaconda is able to see that and not IDLE's python.
Possible solutions:
1. Quick fix:
Just run IDLE via /Users/myname/anaconda3/bin/idle3 every time. This one uses anaconda's python and should be able to see all packages installed via conda install of pip install (*). I get this is tiresome, but you don't have to delete anything. You can also set an "alias" in your ~/.bashrc file to make the command idle specifically linking you there. Let me know with a comment if you would like me to explain how to do that, as this answer will get too long and redundant.
2. Remove conda altogether (not recommended)
You can search google on how to uninstall anaconda along with everything that it has installed. What I do not know at this point is whether your "vanilla" python will become the default, whether you will need to also manually install pip again and whether there is the need to reinstall python in order for everything to work properly.
3. Remove your python "vanilla" installation and only use anaconda
Again, I do not know how python installation works in OSX, but it should be reasonably straightforward to uninstall it. The problem now is that probably you will not have a launcher for IDLE (since I am guessing anaconda doesn't provide one on OSX) but you will be able to use it via the terminal as described in 1..
4. Last resort:
If everything fails, simply uninstall both your vanilla python (which I presume will also uninstall IDLE) and anaconda which will uninstall its own python, pip and idle versions. The relevant documentation should not be difficult to follow. Then, reinstall whichever you want anew.
Finally:
When you solve your problems, any IDE you choose, being VScode (I haven't use that either), pycharm or something else, will probably be able to integrate with your installed python. There is no need to install a new python "bundle" with every IDE.
(*): Since you said that after typing pip install pandas your anaconda's python can import pandas while IDLE cannot, I am implying in my answer that pip is also the one that comes with anaconda. You can make sure this is the case by typing which pip which should point to an anaconda directory, probably /Users/myname/anaconda3/bin/pip
First: This would be a comment if I had enough reputation.
Second: I would just delete python. Everything. And reinstall it.
To repeat and summarized what has been said on various other question answers:
1a. 3rd party packages are installed for a particular python(3).exe binary.
1b. To install multiple packages to multiple binaries, see the option from python -m pip -h.
To find out which python binary is running, execute import sys; print(sys.executable).
3a. For 3rd party package xyz usually installed in some_python/Lib/site-packages, IDLE itself has nothing to do with whether import xyz works. It only matters whether xyz is installed for 'somepython' (see 1a).
3b. To run IDLE with 'somepython', run somepython -m idlelib in a terminal or console.
somepython can be a name recognized by the OS or a path to a python executable.
I've installed anaconda, but I still want to use python as normal within a Terminal window (zsh). At the moment, running which python shows /Users/USERNAME/opt/anaconda3/bin/python.
I don't want this command to point to my anaconda installation. I want to use python as I have in the past, and for it to point to my python 2.7 installation (/usr/bin/python).
The same issue is present with pip and with python3. How do I change these paths back to their default locations?
I want to start out by saying I personally have never used Anaconda, but I have many friends who have and they always seem to wind up getting into hot water as far as configurations go. If you don't need the data-science specific tools that Anaconda offers (most of which I imagine you can get just as easily by using pip install <package_name>), then I STRONGLY urge you to not use Anaconda. I would suggest you instead use homebrew to install python 3.7 and leave the system version of python 2.7 alone... Anyways...
I recently fixed an issue similar to the one you're describing for a friend. I ended up deleted his anacaonda build and reinstalling fresh from homebrew (link: https://brew.sh/), so this is how I helped him and hopefully this will help you too.
Firstly, I would check the configuration of your ~/.bash_profile (or potentially ~/.bashrc) file. Comment out any lines you find that have to do with Anaconda by preceding with the # character. I believe when anaconda installs it puts the location of the anaconda-version-of-python higher in the PATH hierarchy, therefore you computer sees it first before it has the chance to look in /usr/bin...
Secondly, you need to move the anaconda-python to the trash... I would start by trashing the whole '/Users/USERNAME/opt/anaconda3' directory. If memory serves me correctly anaconda stores other python related things in the /MacitoshHD/Library/Frameworks directory, but don't quote me on that. I would advise looking there and trashing whatever seems to be related to the anaconda install.
After that, in a new terminal (so that your PATH refreshes), I installed homebrew, and then ran the command brew install python3 to install python3. I think perhaps I ran the command recommended in the installer output, something like "brew link python3" to symlink python3 where it needs to be linked after the installation.
Finally, everything is done python2 and python3 are installed and working with no issues. python2 is the system default version and python3 is the homebrew installed version. Feel free to ask any questions and I will try to help more, hopefully those instructions will get you started though.
I'm a Java/Scala dev transitioning to Python for a work project. To dust off the cobwebs on the Python side of my brain, I wrote a webapp that acts as a front-end for Docker when doing local Docker work. I'm now working on packaging it up and, as such, am learning about setup.py and virtualenv. Coming from the JVM world, where dependencies aren't "installed" so much as downloaded to a repository and referenced when needed, the way pip handles things is a bit foreign. It seems like best practice for production Python work is to first create a virtual environment for your project, do your coding work, then package it up with setup.py.
My question is, what happens on the other end when someone needs to install what I've written? They too will have to create a virtual environment for the package but won't know how to set it up without inspecting the setup.py file to figure out what version of Python to use, etc. Is there a way for me to create a setup.py file that also creates the appropriate virtual environment as part of the install process? If not — or if that's considered a "no" as this respondent stated to this SO post — what is considered "best practice" in this situation?
You can think of virtualenv as an isolation for every package you install using pip. It is a simple way to handle different versions of python and packages. For instance you have two projects which use same packages but different versions of them. So, by using virtualenv you can isolate those two projects and install different version of packages separately, not on your working system.
Now, let's say, you want work on a project with your friend. In order to have the same packages installed you have to share somehow what versions and which packages your project depends on. If you are delivering a reusable package (a library) then you need to distribute it and here where setup.py helps. You can learn more in Quick Start
However, if you work on a web site, all you need is to put libraries versions into a separate file. Best practice is to create separate requirements for tests, development and production. In order to see the format of the file - write pip freeze. You will be presented with a list of packages installed on the system (or in the virtualenv) right now. Put it into the file and you can install it later on another pc, with completely clear virtualenv using pip install -r development.txt
And one more thing, please do not put strict versions of packages like pip freeze shows, most of time you want >= at least X.X version. And good news here is that pip handles dependencies by its own. It means you do not have to put dependent packages there, pip will sort it out.
Talking about deploy, you may want to check tox, a tool for managing virtualenvs. It helps a lot with deploy.
Python default package path always point to system environment, that need Administrator access to install. Virtualenv able to localised the installation to an isolated environment.
For deployment/distribution of package, you can choose to
Distribute by source code. User need to run python setup.py --install, or
Pack your python package and upload to Pypi or custom Devpi. So the user can simply use pip install <yourpackage>
However, as you notice the issue on top : without virtualenv, they user need administrator access to install any python package.
In addition, the Pypi package worlds contains a certain amount of badly tested package that doesn't work out of the box.
Note : virtualenv itself is actually a hack to achieve isolation.