We Keep Coding

In need of a little assistance in configuring my python installation

Being a "normal" user (i.e. not overly familiar with code), I find myself in a bit of a python conundrum and would very much like to try to muster some kind of assistance with the following:
I use svtplay-dl and youtube-dl via homebrew in Mac OSX 10.14.6, which also requires python (among many other extensions such as ffmpeg et al). However, I now seem to have three different python versions installed, and since python3.5 is no longer supported, I thought I would delete this and make sure that homebrew/svtplay-dl/youtube-dl uses a path to python3.9.
When I type "which" followed by the following in Terminal:
python
python3.5
python3.9
...I get the following respective results:
/usr/local/bin/python
/Library/Frameworks/Python.framework/Versions/3.5/bin/python3.5
/usr/local/bin/python3.9
Based upon this short background info, can someone advise how I best uninstall/remove python3.5 and get homebrew et al to use python3.9 every time python is required?
Any help with this would be very much appreciated!
/Mark

The normal answer is to use something like virtualenv (https://virtualenv.pypa.io/en/latest/) to isolate your python environment and dependencies locally.
You need to install virtualenv on your system, then you tell it what python version to install. Once activated, any dependencies are installed and isolated in that environment. Also see: https://gist.github.com/pandafulmanda/730a9355e088a9970b18275cb9eadef3
If you want to go a step further you can use docker (https://docs.docker.com/language/python/build-images/) to isolate everything, including the operating system, in that case you don't need virtualenv, but to start with I'd play around with virtualenv.

How do I resolve #error: architecture not supported during pip install psutil?

I've run into some problems while trying to download packages with pip. Namely, when I run
pip install "dask[complete]"
on MacOS Catalina 10.15.7. Specifically, the package that throws this error is psutil.
I run into an #error: architecture not supported message. I believe this is because the installer is looking for the MacOS 10.14.6 SDKs, but I am on 10.15.7. One user on this post stated that installing a new Python 3 fixed the issue. Maybe this is because the newer Python comes with a cached version of the package that uses the MacOS 10.15.7 SDKs.
I'm guessing that there might be two solutions to this issue:
Download a non-cached version of psutil, which might be compatible with the version of Python I'm using as well as MacOS Catalina.
Download a newer version of Python 3, namely 3.8.5+, and replace the Xcode Python 3 with this version, enabling me to stay up-to-date with current Python and packages.
I'm wary of the second method because of any possible dependencies I will lose while updating, and I've also been made very aware through research that generally speaking, you don't want to mess with system Python installations.
I'm not opposed to being walked through the first solution, if it would indeed solve the problem, but am very curious about the second solution, again, if it is possible. If neither approach would work out as intended, has anyone encountered this issue and resolved it without using a fresh, SEPARATE install of Python? I don't really want to start messing with virtual environments, but if I must, I supposed I will just download PyCharm and restart there.
EDIT: After continuing to try to work around the issue, I've been led to believe that it is not being caused by an outdated version of psutil; could it be an outdated version of setup.py? In this case, would it indeed be an issue with an old Python version? I'm very new to configuring Python in depth like this, any help would be greatly appreciated.

Try setting the environment variable ARCHFLAGS="-arch x86_64" so Xcode 12 doesn't try to build an ARM64 universal binary. It worked for me on a different python package that had the same error.
ARCHFLAGS="-arch x86_64" pip install "dask[complete]"
(Thanks to Keith Smiley's comment at on Bitbucket for pointing this out.)

IDLE and python is different, not able to install modules properly

thanks for reading this. I am using macOS High Sierra. I am not very familiar with terminal or environment variables, but am trying to learn more. From reading other threads and google, it seems like I either have multiple pythons installed, or have pythons running from different paths. However I am not able to find a solution to resolving this, either by re-pathing my IDLE or deleting it entirely.
I do have python, python launcher, and anaconda (not very sure how anaconda works, have it installed a few years back and didn't touch it) installed. I am trying to install pandas (pip install pandas), which tells me that I have it installed, but when I run it on IDLE, it says module not found. Though if i run python3 on terminal and type my code in, it works (so pandas has indeed been installed).
When i run which python on terminal, it returns
/Users/myname/anaconda3/bin/python
(when i enter into this directory from terminal, it shows that in the bin folder, I have python, python.app, python3, python3-config, python3.7, python3.7-config, python3.7m, python3.7m-config)
When i run which idle on terminal, it returns
/usr/bin/idle (im not even sure how to find this directory from the terminal)
When i run import os; print(os.path) on IDLE, it returns module 'posixpath' from '/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/posixpath.py'
Would really appreciate some help to figure out how to ensure that when i install modules from terminal, it would be installed into the same python as the one IDLE is using. Also, I would like to know whether it is possible for me to work on VSCode instead of IDLE. I cant seem to find suitable extensions for data science and its related modules (like statsmodels, pandas etc). Thanks a lot!

First of all, a quick description of anaconda:
Anaconda is meant to help you manage multiple python "environments", each one potentially having its own python version and installed packages (with their own respective versions). This is really useful in cases where you would like multiple python versions for different tasks or when there is some conflict in versions of packages, required by other ones. By default, anaconda creates a "base" environment with a specific python version, IDLE and pip. Also, anaconda provides an improved way (with respect to pip) of installing and managing packages via the command conda install <package-name>.
For the rest, I will be using the word "vanilla" to refer to the python/installation that you manually set up, independent of anaconda.
Explanation of the problem:
Now, the problem arises since you also installed python independently. The details of the problem depend on how exactly you set up both python and anaconda, so I cannot tell you exactly what went wrong. Also, I am not an OSX user, so I have no idea how python is installed and what it downloads/sets alongside.
By your description however, it seems that the "vanilla" python installation did not overwrite neither your anaconda python nor anaconda's pip, but it did install IDLE and set it up to use this new python.
So right now, when you are downloading something via pip, only the python from anaconda is able to see that and not IDLE's python.
Possible solutions:
1. Quick fix:
Just run IDLE via /Users/myname/anaconda3/bin/idle3 every time. This one uses anaconda's python and should be able to see all packages installed via conda install of pip install (*). I get this is tiresome, but you don't have to delete anything. You can also set an "alias" in your ~/.bashrc file to make the command idle specifically linking you there. Let me know with a comment if you would like me to explain how to do that, as this answer will get too long and redundant.
2. Remove conda altogether (not recommended)
You can search google on how to uninstall anaconda along with everything that it has installed. What I do not know at this point is whether your "vanilla" python will become the default, whether you will need to also manually install pip again and whether there is the need to reinstall python in order for everything to work properly.
3. Remove your python "vanilla" installation and only use anaconda
Again, I do not know how python installation works in OSX, but it should be reasonably straightforward to uninstall it. The problem now is that probably you will not have a launcher for IDLE (since I am guessing anaconda doesn't provide one on OSX) but you will be able to use it via the terminal as described in 1..
4. Last resort:
If everything fails, simply uninstall both your vanilla python (which I presume will also uninstall IDLE) and anaconda which will uninstall its own python, pip and idle versions. The relevant documentation should not be difficult to follow. Then, reinstall whichever you want anew.
Finally:
When you solve your problems, any IDE you choose, being VScode (I haven't use that either), pycharm or something else, will probably be able to integrate with your installed python. There is no need to install a new python "bundle" with every IDE.
(*): Since you said that after typing pip install pandas your anaconda's python can import pandas while IDLE cannot, I am implying in my answer that pip is also the one that comes with anaconda. You can make sure this is the case by typing which pip which should point to an anaconda directory, probably /Users/myname/anaconda3/bin/pip

First: This would be a comment if I had enough reputation.
Second: I would just delete python. Everything. And reinstall it.

To repeat and summarized what has been said on various other question answers:
1a. 3rd party packages are installed for a particular python(3).exe binary.
1b. To install multiple packages to multiple binaries, see the option from python -m pip -h.
To find out which python binary is running, execute import sys; print(sys.executable).
3a. For 3rd party package xyz usually installed in some_python/Lib/site-packages, IDLE itself has nothing to do with whether import xyz works. It only matters whether xyz is installed for 'somepython' (see 1a).
3b. To run IDLE with 'somepython', run somepython -m idlelib in a terminal or console.
somepython can be a name recognized by the OS or a path to a python executable.

'python' points to anaconda installation

I've installed anaconda, but I still want to use python as normal within a Terminal window (zsh). At the moment, running which python shows /Users/USERNAME/opt/anaconda3/bin/python.
I don't want this command to point to my anaconda installation. I want to use python as I have in the past, and for it to point to my python 2.7 installation (/usr/bin/python).
The same issue is present with pip and with python3. How do I change these paths back to their default locations?

I want to start out by saying I personally have never used Anaconda, but I have many friends who have and they always seem to wind up getting into hot water as far as configurations go. If you don't need the data-science specific tools that Anaconda offers (most of which I imagine you can get just as easily by using pip install <package_name>), then I STRONGLY urge you to not use Anaconda. I would suggest you instead use homebrew to install python 3.7 and leave the system version of python 2.7 alone... Anyways...
I recently fixed an issue similar to the one you're describing for a friend. I ended up deleted his anacaonda build and reinstalling fresh from homebrew (link: https://brew.sh/), so this is how I helped him and hopefully this will help you too.
Firstly, I would check the configuration of your ~/.bash_profile (or potentially ~/.bashrc) file. Comment out any lines you find that have to do with Anaconda by preceding with the # character. I believe when anaconda installs it puts the location of the anaconda-version-of-python higher in the PATH hierarchy, therefore you computer sees it first before it has the chance to look in /usr/bin...
Secondly, you need to move the anaconda-python to the trash... I would start by trashing the whole '/Users/USERNAME/opt/anaconda3' directory. If memory serves me correctly anaconda stores other python related things in the /MacitoshHD/Library/Frameworks directory, but don't quote me on that. I would advise looking there and trashing whatever seems to be related to the anaconda install.
After that, in a new terminal (so that your PATH refreshes), I installed homebrew, and then ran the command brew install python3 to install python3. I think perhaps I ran the command recommended in the installer output, something like "brew link python3" to symlink python3 where it needs to be linked after the installation.
Finally, everything is done python2 and python3 are installed and working with no issues. python2 is the system default version and python3 is the homebrew installed version. Feel free to ask any questions and I will try to help more, hopefully those instructions will get you started though.

Does Python have a package/module management system?

Does Python have a package/module management system, similar to how Ruby has rubygems where you can do gem install packagename?
On Installing Python Modules, I only see references to python setup.py install, but that requires you to find the package first.

Recent progress
March 2014: Good news! Python 3.4 ships with Pip. Pip has long been Python's de-facto standard package manager. You can install a package like this:
pip install httpie
Wahey! This is the best feature of any Python release. It makes the community's wealth of libraries accessible to everyone. Newbies are no longer excluded from using community libraries by the prohibitive difficulty of setup.
However, there remains a number of outstanding frustrations with the Python packaging experience. Cumulatively, they make Python very unwelcoming for newbies. Also, the long history of neglect (ie. not shipping with a package manager for 14 years from Python 2.0 to Python 3.3) did damage to the community. I describe both below.
Outstanding frustrations
It's important to understand that while experienced users are able to work around these frustrations, they are significant barriers to people new to Python. In fact, the difficulty and general user-unfriendliness is likely to deter many of them.
PyPI website is counter-helpful
Every language with a package manager has an official (or quasi-official) repository for the community to download and publish packages. Python has the Python Package Index, PyPI. https://pypi.python.org/pypi
Let's compare its pages with those of RubyGems and Npm (the Node package manager).
https://rubygems.org/gems/rails RubyGems page for the package rails
https://www.npmjs.org/package/express Npm page for the package express
https://pypi.python.org/pypi/simplejson/ PyPI page for the package simplejson
You'll see the RubyGems and Npm pages both begin with a one-line description of the package, then large friendly instructions how to install it.
Meanwhile, woe to any hapless Python user who naively browses to PyPI. On https://pypi.python.org/pypi/simplejson/ , they'll find no such helpful instructions. There is however, a large green 'Download' link. It's not unreasonable to follow it. Aha, they click! Their browser downloads a .tar.gz file. Many Windows users can't even open it, but if they persevere they may eventually extract it, then run setup.py and eventually with the help of Google setup.py install. Some will give up and reinvent the wheel..
Of course, all of this is wrong. The easiest way to install a package is with a Pip command. But PyPI didn't even mention Pip. Instead, it led them down an archaic and tedious path.
Error: Unable to find vcvarsall.bat
Numpy is one of Python's most popular libraries. Try to install it with Pip, you get this cryptic error message:
Error: Unable to find vcvarsall.bat
Trying to fix that is one of the most popular questions on Stack Overflow: "error: Unable to find vcvarsall.bat"
Few people succeed.
For comparison, in the same situation, Ruby prints this message, which explains what's going on and how to fix it:
Please update your PATH to include build tools or download the DevKit from http://rubyinstaller.org/downloads and follow the instructions at http://github.com/oneclick/rubyinstaller/wiki/Development-Kit
Publishing packages is hard
Ruby and Nodejs ship with full-featured package managers, Gem (since 2007) and Npm (since 2011), and have nurtured sharing communities centred around GitHub. Npm makes publishing packages as easy as installing them, it already has 64k packages. RubyGems lists 72k packages. The venerable Python package index lists only 41k.
History
Flying in the face of its "batteries included" motto, Python shipped without a package manager until 2014.
Until Pip, the de facto standard was a command easy_install. It was woefully inadequate. The was no command to uninstall packages.
Pip was a massive improvement. It had most the features of Ruby's Gem. Unfortunately, Pip was--until recently--ironically difficult to install. In fact, the problem remains a top Python question on Stack Overflow: "How do I install pip on Windows?"

And just to provide a contrast, there's also pip.

The Python Package Index (PyPI) seems to be standard:
To install a package:
pip install MyProject
To update a package
pip install --upgrade MyProject
To fix a version of a package pip install MyProject==1.0
You can install the package manager as follows:
curl -O http://python-distribute.org/distribute_setup.py
python distribute_setup.py
easy_install pip
References:
http://guide.python-distribute.org/
http://pypi.python.org/pypi/distribute

As a Ruby and Perl developer and learning-Python guy, I haven't found easy_install or pip to be the equivalent to RubyGems or CPAN.
I tend to keep my development systems running the latest versions of modules as the developers update them, and freeze my production systems at set versions. Both RubyGems and CPAN make it easy to find modules by listing what's available, then install and later update them individually or in bulk if desired.
easy_install and pip make it easy to install a module ONCE I located it via a browser search or learned about it by some other means, but they won't tell me what is available. I can explicitly name the module to be updated, but the apps won't tell me what has been updated nor will they update everything in bulk if I want.
So, the basic functionality is there in pip and easy_install but there are features missing that I'd like to see that would make them friendlier and easier to use and on par with CPAN and RubyGems.

There are at least two, easy_install and its successor pip.

As of at least late 2014, Continuum Analytics' Anaconda Python distribution with the conda package manager should be considered. It solves most of the serious issues people run into with Python in general (managing different Python versions, updating Python versions, package management, virtual environments, Windows/Mac compatibility) in one cohesive download.
It enables you to do pretty much everything you could want to with Python without having to change the system at all. My next preferred solution is pip + virtualenv, but you either have to install virtualenv into your system Python (and your system Python may not be the version you want), or build from source. Anaconda makes this whole process the click of a button, as well as adding a bunch of other features.

That'd be easy_install.

It's called setuptools. You run it with the "easy_install" command.
You can find the directory at http://pypi.python.org/

I don't see either MacPorts or Homebrew mentioned in other answers here, but since I do see them mentioned elsewhere on Stack Overflow for related questions, I'll add my own US$0.02 that many folks seem to consider MacPorts as not only a package manager for packages in general (as of today they list 16311 packages/ports, 2931 matching "python", albeit only for Macs), but also as a decent (maybe better) package manager for Python packages/modules:
Question
"...what is the method that Mac python developers use to manage their modules?"
Answers
"MacPorts is perfect for Python on the Mac."
"The best way is to use MacPorts."
"I prefer MacPorts..."
"With my MacPorts setup..."
"I use MacPorts to install ... third-party modules tracked by MacPorts"
SciPy
"Macs (unlike Linux) don’t come with a package manager, but there are a couple of popular package managers you can install.
Macports..."
I'm still debating on whether or not to use MacPorts myself, but at the moment I'm leaning in that direction.

On Windows install http://chocolatey.org/ then
choco install python
Open a new cmd-window with the updated PATH. Next, do
choco install pip
After that you can
pip install pyside
pip install ipython
...

Since no one has mentioned pipenv here, I would like to describe my views why everyone should use it for managing python packages.
As #ColonelPanic mentioned there are several issues with the Python Package Index and with pip and virtualenv also.
Pipenv solves most of the issues with pip and provides additional features also.
Pipenv features
Pipenv is intended to replace pip and virtualenv, which means pipenv will automatically create a separate virtual environment for every project thus avoiding conflicts between different python versions/package versions for different projects.
Enables truly deterministic builds, while easily specifying only what you want.
Generates and checks file hashes for locked dependencies.
Automatically install required Pythons, if pyenv is available.
Automatically finds your project home, recursively, by looking for a Pipfile.
Automatically generates a Pipfile, if one doesn’t exist.
Automatically creates a virtualenv in a standard location.
Automatically adds/removes packages to a Pipfile when they are un/installed.
Automatically loads .env files, if they exist.
If you have worked on python projects before, you would realize these features make managing packages way easier.
Other Commands
check checks for security vulnerabilities and asserts that PEP 508 requirements are being met by the current environment. (which I think is a great feature especially after this - Malicious packages on PyPi)
graph will show you a dependency graph, of your installed dependencies.
You can read more about it here - Pipenv.
Installation
You can find the installation documentation here
P.S.: If you liked working with the Python Package requests , you would be pleased to know that pipenv is by the same developer Kenneth Reitz

In 2019 poetry is the package and dependency manager you are looking for.
https://github.com/sdispater/poetry#why
It's modern, simple and reliable.

Poetry is what you're looking for. It takes care of dependency management, virtual environments, running.