I'm new to coding and I was wondering if someone could explain what exactly python interpreters/environments do and the how that relates to the python versions and packages that can be used in certain projects.
I was trying to code a twitter bot with python and I use VS code as my IDE. While trying to import tweepy into my python code, I noticed I kept getting an error that it couldn't import. After some googling I realized it was because I had to set the python interpreter of the python file to the one that had tweepy installed to. So I changed it and the error with tweepy was solved. But I noticed that I have a bunch of python interpreters in seemingly different locations (screenshot of the interpreter options I'm given). The interpreter that fixed the issue was the one in the pyenv path.
I had installed pyenv some months back because I wanted my terminal (I'm on macOS) to automatically launch python3 when I typed python into the terminal. However, I don't really know what it does beyond that.
So my questions are:
Why do I have multiple python interpreters? Is there a way to get rid of the ones and just keep the one from pyenv (like clean up the ones in /usr/local/bin/python3, /opt/homebrew/bin/python or /usr/bin/python3 since I'm not using them)? or should I not do that?
What exactly does pyenv do? Is it okay that my python libraries are getting installed to pyenv by default? Or should I change it so that it's getting installed to homebrew or one of the usr/bin paths?
Sorry, this is my first time asking a question here so I might not sound that cohesive.
The interpreter is the program which executes a python source file. This is a program like any other, and you can have various versions of it at once (indeed python is quite good at being self contained, and multiple versions will live alongside one another quite happily).
Macs ship with an outdated python 2, used for something in the system. If you remove or update it, things may break. So you install your own python 3 somewhere. If you want to load the new python when you type python in a shell, that shell's $PATH needs to point to the right python. You generally set this in something like ~/.profile.
Pyenv is a tool for managing multiple python interpreters. It's usually used on a per-project basis, to test code against multiple pythons. Further to complicate things, it's often used alongside a virtual environment tool like pipenv. Using it to avoid manually fiddling with $PATH is fine, but a slightly orthogonal use case. All pyenv does is to put little scripts with the same name as python executables somewhere in your $PATH where they supercede anything else. These scripts then figure out which python should get called in this case. So pyenv isn't installing anything, it's just working out which python is going to do it.
The solution to all your problems turns out to be quite simple:
python -m pip install abc
Get into the habit of installing stuff with the python you want to use already set up in the shell, and calling pip like that. That way whatever you use it will always be installed in the right env.
Related
Yesterday I've been trying to install Python 3.8.5. I got the installer from the python website. My problem is that whenever I run python --version in the command line, it keeps saying, like in the image above, Python 2.7.17.
Today I also installed the Microsoft store package, and it's the same.
But running py apparently gives me the correct version 3.8.5. Does this mean that every python program I'll write will use the current version? That's what I currently want to make sure of.
As a side note also, I uninstalled all other python versions except 3.8.5 that I saw in my settings>Apps list. So, that adds to my confusion why running python --version would still show 2.7.17.
Maybe not as relevant, but I also have WSL2 enabled on my machine, I would like to install python to both. At the moment I'm focusing on the Windows10 part, but if there's a solution that will help both, that'll be awesome.
Thanks in advance!
[update]: adding python3 --version screenshot
[update]: yet another confusing part for me:
should I just let this be?
Is this something I need to straighten up first before moving forward?
are my python programs going to be ran in version 3.8.5 and not
2.7.17?
I know my answer is very late but i see a lot of people facing this problem. So if you want to interact with the latest Python version in your command prompt, you just need to set newer (or which one you want) Python path variable above/before other Python version's path variables. This solution also helped me a lot with other path problems.
This is a problem with your PATH environment variable being set incorrectly. I'm guessing that you installed Python 3 globally, after you had Python 2.7 installed. Whenever you do this and have the option Add to PATH checked, it will overwrite the default python location to the new install, thus causing Python 3 to now open instead. You can change this back manually by changing the value in PATH.
I'd suggest using something like pipenv to manage your python versions and respective libraries, that will create a sandbox for your application to run in and will always point to the correct dependencies.
Ok, so hopefully this fixes every question I had. Hopefully comments will assure me that is so.
What I did was, delete the environment variables relating to Python 27. Found the Python3.exe path and added that to the path variable. Not exactly sure how that works, but it did give me the outcome I wanted to see.
thanks for reading this. I am using macOS High Sierra. I am not very familiar with terminal or environment variables, but am trying to learn more. From reading other threads and google, it seems like I either have multiple pythons installed, or have pythons running from different paths. However I am not able to find a solution to resolving this, either by re-pathing my IDLE or deleting it entirely.
I do have python, python launcher, and anaconda (not very sure how anaconda works, have it installed a few years back and didn't touch it) installed. I am trying to install pandas (pip install pandas), which tells me that I have it installed, but when I run it on IDLE, it says module not found. Though if i run python3 on terminal and type my code in, it works (so pandas has indeed been installed).
When i run which python on terminal, it returns
/Users/myname/anaconda3/bin/python
(when i enter into this directory from terminal, it shows that in the bin folder, I have python, python.app, python3, python3-config, python3.7, python3.7-config, python3.7m, python3.7m-config)
When i run which idle on terminal, it returns
/usr/bin/idle (im not even sure how to find this directory from the terminal)
When i run import os; print(os.path) on IDLE, it returns module 'posixpath' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py'
Would really appreciate some help to figure out how to ensure that when i install modules from terminal, it would be installed into the same python as the one IDLE is using. Also, I would like to know whether it is possible for me to work on VSCode instead of IDLE. I cant seem to find suitable extensions for data science and its related modules (like statsmodels, pandas etc). Thanks a lot!
First of all, a quick description of anaconda:
Anaconda is meant to help you manage multiple python "environments", each one potentially having its own python version and installed packages (with their own respective versions). This is really useful in cases where you would like multiple python versions for different tasks or when there is some conflict in versions of packages, required by other ones. By default, anaconda creates a "base" environment with a specific python version, IDLE and pip. Also, anaconda provides an improved way (with respect to pip) of installing and managing packages via the command conda install <package-name>.
For the rest, I will be using the word "vanilla" to refer to the python/installation that you manually set up, independent of anaconda.
Explanation of the problem:
Now, the problem arises since you also installed python independently. The details of the problem depend on how exactly you set up both python and anaconda, so I cannot tell you exactly what went wrong. Also, I am not an OSX user, so I have no idea how python is installed and what it downloads/sets alongside.
By your description however, it seems that the "vanilla" python installation did not overwrite neither your anaconda python nor anaconda's pip, but it did install IDLE and set it up to use this new python.
So right now, when you are downloading something via pip, only the python from anaconda is able to see that and not IDLE's python.
Possible solutions:
1. Quick fix:
Just run IDLE via /Users/myname/anaconda3/bin/idle3 every time. This one uses anaconda's python and should be able to see all packages installed via conda install of pip install (*). I get this is tiresome, but you don't have to delete anything. You can also set an "alias" in your ~/.bashrc file to make the command idle specifically linking you there. Let me know with a comment if you would like me to explain how to do that, as this answer will get too long and redundant.
2. Remove conda altogether (not recommended)
You can search google on how to uninstall anaconda along with everything that it has installed. What I do not know at this point is whether your "vanilla" python will become the default, whether you will need to also manually install pip again and whether there is the need to reinstall python in order for everything to work properly.
3. Remove your python "vanilla" installation and only use anaconda
Again, I do not know how python installation works in OSX, but it should be reasonably straightforward to uninstall it. The problem now is that probably you will not have a launcher for IDLE (since I am guessing anaconda doesn't provide one on OSX) but you will be able to use it via the terminal as described in 1..
4. Last resort:
If everything fails, simply uninstall both your vanilla python (which I presume will also uninstall IDLE) and anaconda which will uninstall its own python, pip and idle versions. The relevant documentation should not be difficult to follow. Then, reinstall whichever you want anew.
Finally:
When you solve your problems, any IDE you choose, being VScode (I haven't use that either), pycharm or something else, will probably be able to integrate with your installed python. There is no need to install a new python "bundle" with every IDE.
(*): Since you said that after typing pip install pandas your anaconda's python can import pandas while IDLE cannot, I am implying in my answer that pip is also the one that comes with anaconda. You can make sure this is the case by typing which pip which should point to an anaconda directory, probably /Users/myname/anaconda3/bin/pip
First: This would be a comment if I had enough reputation.
Second: I would just delete python. Everything. And reinstall it.
To repeat and summarized what has been said on various other question answers:
1a. 3rd party packages are installed for a particular python(3).exe binary.
1b. To install multiple packages to multiple binaries, see the option from python -m pip -h.
To find out which python binary is running, execute import sys; print(sys.executable).
3a. For 3rd party package xyz usually installed in some_python/Lib/site-packages, IDLE itself has nothing to do with whether import xyz works. It only matters whether xyz is installed for 'somepython' (see 1a).
3b. To run IDLE with 'somepython', run somepython -m idlelib in a terminal or console.
somepython can be a name recognized by the OS or a path to a python executable.
We are using Python scripts to process satellite images using different toolboxes or libraries (like GDAL etc.). We now have run into the problem that some 3rd-party Python applications we use (where we have no influence) require different versions of these python packages (for example, the Sentinel2 SEN2COR atmospheric correction requires GDAL 1.x, where we also want to use GDAL 2.x to be able to modify JPEG2000 files).
What's the best way to get this thing set up? I would prefer a way where I can have multiple installations of the same python version (e.g. 2.7, but that does not REALLY matter), and install packages and versions separately for each of them. Like that I could make sure that the SEN2COR script runs in his own python installation, where I install the packages needed and never touch it again, and work in another python installation with my other scripts.
I think a thing like virtualenv would be perfect for that, but there is one important point: All our scripts are command line scripts and are started from a variety of sources, like MATLAB scripts, R scripts or even cron jobs sometimes. Is there a way in virtualenv where I can do something like /usr/bin/python-version-only-for-sen2cor process_data.py arguments and /location/of/other/python-version reload_table.py from the shell or from inside other programs? Whats the best way for our setup? I probably can just install python multiple times and always modify the environment vars to use a different python version while installing the packages, but I guess this is prone to errors. Any suggestions?
You could have a look at pythonbrew or pyenv.
I just used pythonbrew recently. I just run my program with a bash script (to avoid typing the whole path):
#!/bin/bash
~/.pythonbrew/pythons/Python-2.7.7/bin/python my_program.py
Is it possible to use system python in homebrew?
I have python 2.7.5 installed on my mac, but when I try to install any homebrew package with python in dependencies, it starts loading python 2.7.9.
It is important for me to use system python because of lots of installed python packages.
What version of python homebrew uses depends on two things:
Where the python alias points to.
What the python path is in the formula.
In most cases, homebrew forumlae simply use the python command, and does not specify a python path. If it does, then it is usually /usr/bin/python.
So, you can fix the problem in two ways:
Change the homebrew formulate to point to the right python interpreter of your choice.
Change where the python command points to.
The first option is problematic, since it means that you need to go into your Cellar folder, and alter things. This is not nice, and usually not required.
A better alternative is to point to the right python command. This is where pyenv comes in handy. pyenv is a command line tool that helps you manage different versions of python. Follow the instructions here:
https://github.com/yyuu/pyenv-installer
to install it.
Once that is done, change your python command to whatever version of python you'd like it to point to.
I new to Python and to programming in general. I'm a novice, and do not work in programming, just trying to teach myself how to program as a hobby. Prior to Python, I worked with Ruby for a bit and I learned that one of the biggest challenges was actually properly setting up my computer.
Background: I'm on a Macbook with OSX 10.7.
With Ruby, you have to (or rather, you should), edit your ./profile and add PATH info. When you install and use RVM, there are additional items you need to add to your bash_profile.
Do you have to make similar changes with Python? What are the best practices as I'm installing/getting started to ensure I can install modules and packages correctly?
python works out of the box on OS X (as does ruby, for that matter). The only changes I would recommend for a beginner are:
1) Python likes to be reassured that the terminal can handle UTF-8 before it will print Unicode strings. Add export LANG=en_US.UTF-8 to .profile. (It may be that the .UTF-8 part is already present by default on Lion - I haven't checked since Snow Leopard.) Of course, this is something that will help you in debugging, but you shouldn't rely on it being set this way on other machines.
2) Install pip by doing easy_install pip (add sudo if necessary). After that, install Python packages using pip install; this way, you can easily remove them using pip uninstall.
Take a loot at Python on the Macintosh page first. Like it says, Python comes pre-installed on Mac OS X. It means that you don't have to do anything special in order to use it.
To get started, you can run a Terminal.app, type python and that will get you Python interactive shell up and running.
However, Python on OS X might be of a slightly older version. For example, OS X 10.7.3 comes with Python 2.7.1, whereas latest release version of the Python is 3.2.3. If you want to use other versions, then you will have to install them. Then it all depends on what, where and how you install. If you want to have multiple versions alongside, you may need to set some environment variables like PATH to have binaries you installed found by the bash etc. You can do it through bash ~/.profile if needed.
But until you get to that point - don't worry about it use a version shipped with OS X. Once you want a newer one - download and install it. Then, if it doesn't work out of the box or you have any other problems or concerns, feel free to ask a more specific question.