We Keep Coding

Tensorflow metal plugin already registered error

I have installed Tensorflow and Metal plugin by using pip on Mac Mini 2020 M1,
$ pip3 install tensorflow-macos tensorflow-metal
$ pip3 uninstall numpy # related to https://stackoverflow.com/q/66060487/2395656
$ pip3 install numpy==1.20.3
then I tried to list devices to see Mac GPU,
import tensorflow
d = tensorflow.config.list_physical_devices()
print(d)
It produces the error,
Init Plugin
Init Graph Optimizer
Init Kernel
Init Plugin
2021-06-10 02:20:21.128021: F tensorflow/c/experimental/stream_executor/stream_executor.cc:823] Non-OK-status: stream_executor::MultiPlatformManager::RegisterPlatform( std::move(cplatform)) status: Internal: platform is already registered with name: "METAL"
I think that Metal plugin is trying to register itself multiple times.
Please help, thanks!

On my MacBook Air M1 under Monterey 12.3 and the brew-installed pythons python#3.8 and python#3.9 in their standard locations at /opt/homebrew, together with Xcode 13.3 at its standard location, after looking for hours I finally noticed the following and have this theory:
First of all I don't like to use conda or miniconda etc. (only non-python dependency I seemed to lack was hdf5 anyway). Also, I confirm that creating a venv mentioned above by tensorflow support works just fine. But if I just want to pip install tensorflow-macos and tensorflow-metal in the system locations for brewed pythons:
brew install hdf5
HDF5_DIR=/opt/homeware python3 -m pip install tensorflow-macos
If at the last stage the plugin tensorflow-metal gets installed with:
python3 -m pip install tensorflow-metal
while using a brew-ed python#3.9 from /opt/homebrew/bin/python3 at the brew system default dir:
/opt/homebrew/lib/python3.9/site-packages/tensorflow-plugins
then I get the error about the METAL plugin having been already registered.
If however I install in my user directory library instead ~/Library/Python/3.9/lib/python/site-packages via
python3 -m pip install --user tensorflow-metal
then everything works. Note this concerns just the plugin, i.e. the tensorflow-macos package can still be at the brew system location.
I noticed all this, because from past Xcode Python (3.8) usage I had a user packages directory (Python Library) at ~/Library/Python/3.8/lib/Python/site-packages (only place you can add packages when using the system Python) and since it existed, the brewed python#3.8 installed everything there without me supplying --user to pip.
So I banged my head against the wall as to why brewed python#3.8 works and python#3.9 doesn't. I cleaned out all my packages from python#3.9 system directory via pip uninstall, even purged the entire system site-packages directory and still with a clean setup using the two simple pip steps it kept failing. That's because there was no user packages directory for 3.9 and everything kept going back to the brew system directory.
I am quite convinced that is not a case of some configuration or package mix clashing between various pythons as I have been very careful to clean everything before I reproduce. Furthermore, moving the installation of tensorflow-metal under python#3.8 from my user package directory to the brew system directory reproduces the error under python#3.8 that was previously working.
Long story short: There is a bug with tensorflow plugin initialization that somehow triggers when tensorflow-metal plugin is installed under /opt/homebrew causing its registration to be called twice. This is avoided when installed in local --user directory.
I have wasted enough time to make this work and I am happy with my theory for now so will drop further looking into it. I just thought I might shed some light for anyone else down the road in case they come across this.

I have solved the problem after quite some trying.
export OPENBLAS=$(/opt/homebrew/bin/brew --prefix openblas)
export CFLAGS="-falign-functions=8 ${CFLAGS}"
python3 -m venv ~/tensorflow-metal
source ~/tensorflow-metal/bin/activate
python -m pip install -U pip
pip install Cython
pip install --no-use-pep517 numpy==1.19.3
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
I believe Tensorflow can use both CPU and GPU now.
import tensorflow
tensorflow.config.list_physical_devices()
>>> [PhysicalDevice(name='/physical_device:CPU:0', device_type='CPU'), PhysicalDevice(name='/physical_device:GPU:0', device_type='GPU')]

Please try the same after creating a virtual environment.
python3 -m venv ~/tensorflow-metal
source ~/tensorflow-metal/bin/activate
python -m pip install -U pip
python -m pip install tensorflow-macos
python -m pip install tensorflow-metal
Reference- https://developer.apple.com/forums/thread/684889

How to force install package in virtualenv?

Trying to install django with different version that in system, it shows me:
Installing collected packages: Django
Found existing installation: Django 1.7.11
Not uninstalling django at /home/user/lib/python2.7, outside environment /home/user/webapps/v2_dev/venv
Successfully installed Django-1.8.19
But in fact there is old version
tried with different commands:
./venv/bin/pip install Django==1.8.11
pip install Django==1.8.11
UPDATED:
When I install my packages it shows:
The required version of setuptools (>=16.0) is not available,
and can't be installed while this script is running. Please
install a more recent version first, using
'easy_install -U setuptools'.
(Currently using setuptools 3.1 (/home/user/lib/python2.7/setuptools-3.1-py2.7.egg))
When I do the upgrade:
venv/bin/pip install --upgrade setuptools
Requirement already up-to-date: setuptools in ./venv/lib/python2.7/site-packages (40.5.0)

I arrived at this post while looking for how to force install something in a virtualenv despite it being already installed in the global python. This happens when the virtual env was created with --system-site-packages.
In this situation, for certain packages it may be important to have a local version within the virtualenv, even if for many other packages we can share the global versions. This is the case of pytest, for example. However, pip will refuse to install a package in the virtualenv if it can already find the most recent version in the system site.
The solution is to use pip install --ignore-installed mypackage.

Instead of installing setuptools and Django like ./venv/bin/pip install ..., try to activate your virtual environment first and install the stuff you need afterwards.
Activating virtual environment:
Go to the folder where your virtual environment is located (typically the root folder of your project) and type one of the two:
source venv/bin/activate (Unix-based systems)
venv\Scripts\activate (Windows)
This will ensure that you are not mixing packages installed in different environments.
Forcing reinstall of the packages:
Simple upgrade can be done by adding: --upgrade or -U
Forcing reinstall of the packages can be done by adding: --force-reinstall
In your case (once the environment is activated):
python -m pip install -U --force-reinstall setuptools Django
Step by step:
Deactivate and delete the old virtual environment
Create new environment using python -m virtualenv venv (python 2) or python -m venv venv (python 3)
python above is the interpreter which you want to use in your project. That's the only point where you might want to use for example python3 or some absolute path instead. Later use the code as is.
source venv/bin/activate
Activating the virtual environment
python -m pip install -U pip
If you have issue with ImportError: No module named _internal than probably you are using an old version of pip. Issue is described here
python -m pip install -U --force-reinstall -r requirements.txt
-U --force-reinstall is a bit of an overkill in case of fresh environment, but it will do no harm
Go to the place where your manage.py is located and start the server using python manage.py runserver

The problem was in Webfaction VPS
Need an empty file named sitecustomize.py in the /home/username/webapps/appName/env/lib/python2.
That empty file overrides their python customizations, one of which is to include any packages in the ~/lib/python2.7 directory.
You might need to deactivate your virtual env and activate it again for changes to take effect.

workaround but it works!
in your virtualenv directory change the properties of the pyvenv.cfg file
include-system-site-packages = True
this will cause the packages installed on the main to be used

Can't install tensorflow with pip or anaconda

Does anyone know how to properly install tensorflow on Windows?
I'm currently using Python 3.7 (also tried with 3.6) and every time I get the same "Could not find a version that satisfies the requirement tensorflow-gpu (from versions: )
No matching distribution found for tensorflow-gpu" error
I tried installing using pip and anaconda, both don't work for me.
Found a solution, seems like Tensorflow doesn't support versions of python after 3.6.4. This is the version I'm currently using and it works.

Tensorflow or Tensorflow-gpu is supported only for 3.5.X versions of Python. Try installing with any Python 3.5.X version. This should fix your problem.

Here is what i did to get tensorflow working with windows.
Download python 3.7.0 (64 bit from Python Releases for Windows) Install it and check python version by running below command in cmd:
python --version
Python 3.7.0
Then run below command to upgrade pip to latest
python -m pip install --upgrade pip
Now install tensorflow using pip
pip install tensorflow
That's it you have installed tensorflow on windows. Below image shows what happens when you type above commands
link for tensorflow for python 3x
https://pypi.org/project/tensorflow/#files
tensorflow-1.14.0-cp37-cp37m-win_amd64.whl (68.3 MB)

Unfortunately, tensorflow can't installed correctly on python 3.7 and last version of anaconda: so, the best and effective way to do this is to downgrade your python to python 3.6.7 use the next steps:
1- download the latest version of Anaconda
use Anaconda prompt with administrator privilege
2- conda install python=3.6.7 (need a long time)
3-conda install tensorflow
4- conda install keras
5- conda install numpy

Check whether you have a CPU or GPU, if your system doesn't have GPU, then it will generate error.
If you are going to install tensorflow using Windows command prompt (assuming python is already installed), then just run the following command. Go to the root directory, preferably 'C:' drive then run
For GPU- pip3 install --upgrade tensorflow-gpu
For CPU- pip3 install --upgrade tensorflow
If you are using Anaconda, then open Anaconda Navigator->Environments->Select 'All' from the drop down menu and then search TensorFlow. If you are using CPU, then select 'tensorflow', else for GPU select 'tensorflow-gpu'. Then click Apply.
Screenshot of Anaconda Navigator-In case you are not familiar where to look.

Download the latest version of anaconda from here
I'm using Anaconda 2019.03 for Windows Installer, Python 3.7, 64-bit
Upgrade pip as:
python -m pip install --upgrade pip
(a). Install tensorflow using pip3 as:
pip3 install --user --upgrade tensorflow
(b). If you installed anaconda with all users mode. The above
command will become:
pip3 install --upgrade tensorflow
Check installation success as:
python -c "import tensorflow as tf; print(tf.reduce_sum(tf.random.normal([1000, 1000])))"

As of Feb 2020, Tensorflow is not supported for Python 3.8+
To make it work install a virtualenv w/ Python 3.7 from here: https://www.python.org/downloads/windows/

Tensorflow pip installation for python version 3.5-3.8 requires pip 19.0 or later, as mentioned in the official tensorflow documentation.
Here is a part of this documentation:
System requirements
Python 3.5–3.8
Python 3.8 support requires TensorFlow 2.2 or later.
pip 19.0 or later (requires manylinux2010 support)
Try running pip install --upgrade pip inside your python3 virtualenv.

You mentioned Anaconda. Do you run your python through there?
If so check in Anaconda Navigator --> Environments, if your current environment have got tensorflow installed.
If not, install tensorflow and run from that environment.
Should work.

I had the same problem that yours, happened that I had python at version 3.7. So for installing the Tensorflow with the GPU support I used:
py -3.6 -m pip install tensorflow-gpu
at a Virtualenv ambient. You can see the documentation behind on this website:
https://docs.python.org/3/installing/index.html
I think that this solution is better than downgrade the Python version.

Actually the easiest way to install tensorflow is:
install python 3.5 (not 3.6 or 3.7) you can check wich version you have by typing "python" in the cmd.
When you install it check in the options that you install pip with it and you add it to variables environnement.
When its done just go into the cmd and tipe "pip install tensorflow"
It will download tensorflow automatically.
If you want to check that it's been installed type "python" in the cmd then some that ">>>" will appear, then you write "import tensorflow" and if there's no error, you've done it!

As of November 2018, Tensorflow is not supported for Python 3.6.4+
What to do:
Downgrade Anaconda's Python from 3.7 or 3.6.5+ to 3.6.4
a. Open Command Prompt, find your Anaconda environment, and change your directory
where anaconda or where python
Example path and cd:
cd C:\ProgramData\Anaconda3\bin
b. While in anaconda3/bin in your Command Prompt, source activate your base Anaconda environment
source activate
c. In your Command Prompt, downgrade your base Anaconda environment
conda install python=3.6.4
d. Install Tensorflow
pip install tensorflow
Further reading:
http://docs.anaconda.com/anaconda/user-guide/faq/#how-do-i-get-the-latest-anaconda-with-python-3-5

As of July 2019, I have installed it on python 3.7.3 using py -3 -m pip install tensorflow-gpu
py -3 in my installation selects the version 3.7.3.
The installation can also fail if the python installation is not 64 bit. Install a 64 bit version first.

This is Manas working as a intern in COE-AI lab(CET,BBSR) under tech machindra.
We faced same error. After a little research, we found that there is a glitch in anaconda python 3.7. It does not install tensorflow through pip or conda install command, even if it does..produces same error..
Here is the solution, install conda environment for python=3.6:
go to the directory where conda is installed
cd anaconda3
conda create -n tensorflow python=3.6
conda activate tensorflow
conda install tensorflow python=3.6
python
Import tensorflow.
steps
Do it twice:
enter image description here
enter image description here

-> Not able to install tensorflow , Here I have a solution that worked for me
Step 1 :
- Check version of python(3.7.3)
- Python must be 64 bit Otherwise tensorflow never ever going to be installed it defiitely gives Error.
Step 2 :
pip install tensorflow==2.0.0
Step 3 :
pip install sklearn==0.0
pip install Pillow==8.0.1
pip install h5py==2.10.0
Step 4 :
pip install keras==2.3.1
Now its ready to use.

In Windows 10 with Python 3.8.5, first i tried directly then it was giving following error:
ERROR: Could not find a version that satisfies the requirement tensorflow==1.15 (from versions: none)
Then i installed successfully in virtual environment shown below:
PS E:\udemy\full_stack_web_ai\deeplearning> .\myenv\Scripts\activate
(myenv) PS E:\udemy\full_stack_web_ai\deeplearning> pip install tensorflow
Collecting tensorflow ###
Downloading tensorflow-2.4.1-cp38-cp38-win_amd64.whl (370.7 MB)
Use virtual environment using:
python -m venv myenv
.\myenv\Scripts\activate

Not Enabling the Long Paths can be the potential problem.To solve that,
Steps include:
Go to Registry Editor on the Windows Laptop
Find the key "HKEY_LOCAL_MACHINE"->"SYSTEM"->"CurrentControlSet"->
"File System"->"LongPathsEnabled" then double click on that option and change the value from 0 to 1.
3.Now try to install the tensorflow it will work.

Cannot upgrade packages using pip inside virtualenv

I wanted to be able to access all of my site packages from another installation of Python, so I created a virtual environment in this way:
venv my_project --system-site-packages
I noticed that my version of Keras was outdated, so from within my virtualenv, I executed:
pip install keras
which worked without an issue. I'm using pip version 9.0.1
I'm trying to run a python program that uses TensorFlow, but when I run it, I get an error:
ImportError: No module named tensorboard.plugins
I googled around and found that I needed to upgrade TensorFlow. I tried several commands:
(my_project/) user#GPU5:~/spatial/zero_padded/powerlaw$ pip install tensorflow
The above gives me a 'requirement already satisfied' error.
$ pip install --target=~/spatial/zero_padded/powerlaw/my_project/ --upgrade tensorflow
Collecting tensorflow
Could not find a version that satisfies the requirement tensorflow (from versions: )
No matching distribution found for tensorflow
The output of which python:
/user/spatial/zero_padded/powerlaw/my_project/bin/python
I think my PYTHONPATH is the first line in this:
(my_project/) user#GPU5:~/spatial/zero_padded/powerlaw/my_project$ python -c "import sys; print '\n'.join(sys.path)"
/user/spatial/zero_padded/powerlaw/my_project
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python27.zip
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7/plat-linux2
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7/lib-tk
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7/lib-old
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7/lib-dynload
/user/spatial/zero_padded/powerlaw/my_project/lib/python2.7/site-packages
/user/pkgs/enthought/canopy-1.5.1/lib/python2.7/site-packages
/user/pkgs/enthought/canopy-1.5.1/lib/python2.7/site-packages/PIL
/opt/enthought/canopy-1.5.1/appdata/canopy-1.5.1.2730.rh5-x86_64/lib/python2.7/site-packages
How do I upgrade TensorFlow inside my virtualenv?

Pretty sure that all you need to do is run pip install with -U to upgrade the package inside the virtualenv:
(my_project/) user#GPU5:~/spatial/zero_padded/powerlaw$ pip install -U tensorflow
-U is just shorthand for --upgrade. But, you should really go ahead and create a dependencies file for yourself called requirements.txt that lives in the project root and specify version numbers there.
e.g.,
tensorflow==1.2.0
And that makes it easier to install all requirements
pip install -r requirements.txt

The best way to do it is install the dependencies outside de vm, and create a new one I'm afraid to say that. Because doing upgrades is different than installing

Control the pip version in virtualenv

How do I control the version of pip which is used in a freshly created venv?
By default, it uses a vendored pip distribution which may be out of date or unsuitable for whatever other reason. I want to be able to create a venv with a user-specified version of pip installed initially, as opposed to creating one and then upgrading the pip installation from within the env.

For me, I just upgraded pip/virtualenv/virtualenvwrapper on my machine (not inside the virtualenv). Subsequently created virtualenvs had the updated version.
deactivate
pip install --upgrade pip virtualenv virtualenvwrapper
mkvirtualenv ...

From reading the source of virtualenv, it looks like pip is installed from a source tarfile included with virtualenv. In virtualenv 1.10.1, it is pip-1.4.1.tar.gz in the site-packages/virtualenv_support directory (it gets setuptools from the same place). You could feasibly replace that archive to control the version; virtualenv.py, at least the version I have, doesn't care which version of pip is there:
if not no_pip:
install_sdist('Pip', 'pip-*.tar.gz', py_executable, search_dirs)
You could also pass the --no-pip option and then install the version you want from source.
In virtualenv 1.11, it looks for a wheel file (e.g. pip-*.whl) instead of a tar.gz, but other than that it acts the same way (thanks #wim for the update).

You cannot downgrade pip using pip, the solution is to install a specific version in your virtual environment:
virtualenv env -p python3.6 --no-pip
source env/bin/activate
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py pip==18.1
This will allow you to keep using --process-dependency-links that was removed in pip 19.

It's easy enough to replace the pip that gets installed in your virtual environment. Within your virtual environment active, simply execute the following command:
pip install pip==1.4.1

Since Python 3.9 the stdlib venv module has EnvBuilder.upgrade_dependencies. Unfortunately, it has two shortcomings:
Won't really help users to install a specific pip version, only the latest.
It still installs the vendored pip and setuptools versions first, and then uninstall them if they're outdated, which they almost always will be in practice.
It would be ideal to install the latest versions directly! The venv CLI provides a --without-pip argument that is useful here. You can use this to opt-out of the vendored pip, and then actually use the vendored pip wheel to install your desired pip version instead (along with any other packages you might want in a freshly created virtual environment).
It's best to put it into a function - this goes into your shell profile or rc file:
function ve() {
local py="python3"
if [ ! -d ./.venv ]; then
echo "creating venv..."
if ! $py -m venv .venv --prompt=$(basename $PWD) --without-pip; then
echo "ERROR: Problem creating venv" >&2
return 1
else
local whl=$($py -c "import pathlib, ensurepip; [whl] = pathlib.Path(ensurepip.__path__[0]).glob('_bundled/pip*.whl'); print(whl)")
echo "boostrapping pip using $whl"
.venv/bin/python $whl/pip install --upgrade pip setuptools wheel
source .venv/bin/activate
fi
else
source .venv/bin/activate
fi
}
As written, this function just pulls latest pip, setuptools, and wheel from index. To force specific versions you can just change this line of the shell script:
.venv/bin/python $whl/pip install --upgrade pip setuptools wheel
Into this, for example:
.venv/bin/python $whl/pip install pip==19.3.1
For Python 2.7 users, you may do a similar trick because virtualenv provides similar command-line options in --no-pip, --no-setuptools, and --no-wheel, and there is still a vendored pip wheel available to bootstrap since Python 2.7.9. Pathlib will not be available, so you'll need to change the pathlib usage into os.path + glob.

While creating virtual environment using venv module, use optional argument --upgrade-deps.
That will upgrade pip + setuptools to the latest on PyPI.
Example : python3 -m venv --upgrade-deps .venv
Reference link :
venv module documentation
It indicates "Changed in version 3.9: Add --upgrade-deps option to upgrade pip + setuptools to the latest on PyPI"
Note : I tried this using Python 3.10.4

Solved the same issue today on my windows machine with python 3.10.2 installed.
download required pip wheel from history to path\to\python\lib\ensurepip\bundled
in path\to\python\lib\ensurepip\__init__.py change _PIP_VERSION = your version
create environment as usual python -m venv path\to\env

I had issues with pip 22.3.1, so I wanted to downgrade it to 22.3, while pip 22.3.1 produces errors and not letting me downgrade as the other solutions suggest.
I solved the issue by creating a new venv with the specific pip version, as follows:
virtualenv env -p python3.10 --pip 22.3

TLDR
python -m pip install --upgrade pip==<target version number>
Example
Downgrading from pip 20.3 to pip 19.3 from within a virtual environment.
(env) $ pip --version
pip 20.3.1
(env) $ python -m pip install --upgrade pip==19.3 # downgrading
Collecting pip==19.3
Using cached pip-19.3-py2.py3-none-any.whl (1.4 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.3.1
Uninstalling pip-20.3.1:
Successfully uninstalled pip-20.3.1
Successfully installed pip-19.3
(env) $pip --version trex#Tobiahs-MacBook-Pro
pip 19.3