Machine: MacBook Air M1 2020
OS: macOs BigSur 11.4
Python version of venv: Python 3.8.6
Tensorflow version: ATF Apple Tensorflow 0.1a3
Pip version: 21.2.4
I have installed Tensorflow from github using this guide.
Now, my pip list is this.
Package Version
----------------------- ---------
absl-py 0.13.0
appnope 0.1.2
astunparse 1.6.3
backcall 0.2.0
cached-property 1.5.2
cachetools 4.2.2
certifi 2021.5.30
charset-normalizer 2.0.4
cycler 0.10.0
Cython 0.29.24
debugpy 1.4.1
decorator 5.0.9
entrypoints 0.3
flatbuffers 2.0
gast 0.5.2
google-auth 1.35.0
google-auth-oauthlib 0.4.5
google-pasta 0.2.0
grpcio 1.33.2
h5py 2.10.0
idna 3.2
ipykernel 6.2.0
ipython 7.26.0
ipython-genutils 0.2.0
jedi 0.18.0
jupyter-client 7.0.1
jupyter-core 4.7.1
Keras-Preprocessing 1.1.2
kiwisolver 1.3.1
Markdown 3.3.4
matplotlib 3.4.3
matplotlib-inline 0.1.2
nest-asyncio 1.5.1
numpy 1.18.5
oauthlib 3.1.1
opt-einsum 3.3.0
packaging 21.0
parso 0.8.2
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.3.1
pip 21.2.4
prompt-toolkit 3.0.20
protobuf 3.17.3
ptyprocess 0.7.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
Pygments 2.10.0
pyparsing 2.4.7
python-dateutil 2.8.2
pyzmq 22.2.1
requests 2.26.0
requests-oauthlib 1.3.0
rsa 4.7.2
setuptools 57.4.0
six 1.16.0
tensorboard 2.6.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.0
tensorflow-addons 0.1a3
tensorflow-estimator 2.6.0
tensorflow-hub 0.12.0
tensorflow 0.1a3
termcolor 1.1.0
tornado 6.1
traitlets 5.0.5
typeguard 2.12.1
typing-extensions 3.10.0.0
urllib3 1.26.6
wcwidth 0.2.5
Werkzeug 2.0.1
wheel 0.37.0
wrapt 1.12.1
I want install Object Detection Api from Tensorflow in that link.
I cloned the repo and them I follow the guide. (Python Package Installation)
When I execute this command
python -m pip install --use-feature=2020-resolver .
It starts to download, and start a print very long errors.
At the end of the operations, it gives me this error.
Using cached scipy-1.2.3.tar.gz (23.3 MB)
Collecting pandas
Using cached pandas-1.3.2-cp38-cp38-macosx_11_0_arm64.whl
Collecting tf-models-official>=2.5.1
Using cached tf_models_official-2.6.0-py2.py3-none-any.whl (1.8 MB)
Collecting kaggle>=1.3.9
Using cached kaggle-1.5.12-py3-none-any.whl
Collecting py-cpuinfo>=3.3.0
Using cached py_cpuinfo-8.0.0-py3-none-any.whl
Requirement already satisfied: numpy>=1.15.4 in /Users/stefan/Desktop/Studio/TFOD/tf-m1/lib/python3.8/site-packages (from tf-models-official>=2.5.1->object-detection==0.1) (1.18.5)
Collecting opencv-python-headless
Using cached opencv_python_headless-4.5.3.56-cp38-cp38-macosx_11_0_arm64.whl (10.7 MB)
Collecting tf-models-official>=2.5.1
Using cached tf_models_official-2.5.1-py2.py3-none-any.whl (1.6 MB)
Collecting tensorflow-datasets
Using cached tensorflow_datasets-4.4.0-py3-none-any.whl (4.0 MB)
Collecting google-api-python-client>=1.6.7
Downloading google_api_python_client-2.18.0-py2.py3-none-any.whl (7.4 MB)
|████████████████████████████████| 7.4 MB 3.4 MB/s
Collecting oauth2client
Using cached oauth2client-4.1.3-py2.py3-none-any.whl (98 kB)
Collecting tensorflow-model-optimization>=0.4.1
Using cached tensorflow_model_optimization-0.6.0-py2.py3-none-any.whl (211 kB)
Collecting pyyaml>=5.1
Downloading PyYAML-5.4.1.tar.gz (175 kB)
|████████████████████████████████| 175 kB 31.3 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Collecting gin-config
Using cached gin_config-0.4.0-py2.py3-none-any.whl (46 kB)
Collecting sacrebleu
Using cached sacrebleu-2.0.0-py3-none-any.whl (90 kB)
INFO: pip is looking at multiple versions of <Python from Requires-Python> to determine which version is compatible with other requirements. This could take a while.
INFO: pip is looking at multiple versions of object-detection to determine which version is compatible with other requirements. This could take a while.
ERROR: Cannot install object-detection because these package versions have conflicting dependencies.
The conflict is caused by:
tf-models-official 2.6.0 depends on tensorflow-text>=2.5.0
tf-models-official 2.5.1 depends on tensorflow-addons
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/user_guide/#fixing-conflicting-dependencies
I have the same issue installing the Object Detection API for Tensorflow 2 (OD API) from sources on my MacBook Air M1 2020. It starts to lookup/download all available dependencies with very long errors and after several hours the process drains all available RAM and forces the laptop to reboot. I think the problem is with incompatible dependencies for arm64. I tried to build/install OD API for Tensorflow 1 instead and it worked! I successfully trained a model with TensorFlow 2 and GPU enabled.
Use the tf1 folder when you installing the OD API instead of tf2:
cd models/research
# Compile protos.
protoc object_detection/protos/*.proto --python_out=.
# Install TensorFlow Object Detection API.
cp object_detection/packages/tf1/setup.py .
python -m pip install --use-feature=2020-resolver .
or just use this guide for installing OD API: https://github.com/tensorflow/models/blob/master/research/object_detection/g3doc/tf1.md
By the way,
here is a working Tensorflow setup on Apple M1 silicon with the latest TensorFlow versions and Metal GPU acceleration: https://github.com/ctrahey/m1-tensorflow-config
the best guide for object detection: https://neptune.ai/blog/how-to-train-your-own-object-detector-using-tensorflow-object-detection-api
I successfully install it with.
python -m pip install --force --no-dependencies .
My list of commands for install correctly tf2.0 for m1
conda create —-name=tf-m1
conda activate tf-m1
conda install python=3.8.6 -y
sh Desktop/PATH TO GITHUB DIR OF TENSORFLOW MAC(i used 0.1a3)/install_venv.sh /Users/stefan/miniforge3/envs/tf-m1
python -m pip install --upgrade pip
pip install ipykernel jupyter
python -m ipykernel install --user --name=tensorflow-m1.0
Tensorflow Test : ok (import tensorflow as tf; print(tf.__version__))
NOW USE CONDA INSTALL
conda install -c conda-forge matplotlib -y
conda install -c conda-forge scikit-learn -y
conda install -c conda-forge opencv -y
conda install -c conda-forge pandas -y
Tensorflow Test : ok
cd Desktop/PATH/
mkdir -p Tensorflow/models
git clone https://github.com/tensorflow/models Tensorflow/models
cd Tensorflow/models/research && protoc object_detection/protos/*.proto --python_out=. && cp object_detection/packages/tf2/setup.py . && python -m pip install --force --no-dependencies .
Object detection api have some dependencies i had installed.
(Pyarrow and apache-beam are not supported at the moment, but I think this isn't essential for general working of api)
pip install tf-slim
pip install pycocotools
pip install lxml
pip install lvis
pip install contextlib2
pip install --no-dependencies tf-models-official
pip install avro-python3
pip install pyyaml
Pip install gin-config
I don't know if is it the perfect installation of Tensorflow and TensorFlow object-detection-api, but at the moment this worked for me.
Things should work better if you upgrade to OS Monterey and install conda from miniforge and the packages listed below.
As of Oct. 25, 2021 macOS 12 Monterey is generally available.
Upgrade your machine to Monterey.
If you have conda installed, uninstall it.
Then follow the instructions from Apple here.
Cleaned up below:
Download and install Conda from Miniforge:
chmod +x ~/Downloads/Miniforge3-MacOSX-arm64.sh
sh ~/Downloads/Miniforge3-MacOSX-arm64.sh
source ~/miniforge3/bin/activate
In a conda environment, install the TensorFlow dependencies, base TensorFlow, and TensorFlow metal:
conda install -c apple tensorflow-deps
pip install tensorflow-macos
pip install tensorflow-metal
You should be good to go.
Related
I have below installed packages
asn1crypto 0.24.0
asyncio 3.4.3
beautifulsoup4 4.7.1
boto 2.49.0
boto3 1.9.199
botocore 1.12.199
certifi 2019.6.16
cffi 1.12.3
chardet 3.0.4
configparser 5.0.1
cryptography 2.7
cx-Oracle 6.4.1
docutils 0.14
gensim 3.5.0
idna 2.8
jmespath 0.9.4
joblib 0.13.2
lxml 4.3.4
nltk 3.3
pid 3.0.4
pip 20.0.2
psutil 5.7.2
pycparser 2.19
python-dateutil 2.8.0
python-pidfile 3.0.0
requests 2.22.0
s3transfer 0.2.1
setuptools 40.6.2
six 1.12.0
smart-open 1.8.4
sortedcontainers 2.1.0
soupsieve 1.9.3
statistics 1.0.3.5
urllib3 1.25.3
When I tried to install pandas 1.1.0 for python 3.6.9
python3.6 -m pip install --proxy=https://abcd.com --trusted-host pypi.python.org --trusted-host files.pythonhosted.org pandas==1.1.0 -i https://pypi.python.org/simple/
Looking in indexes: https://pypi.python.org/simple/
ERROR: Could not find a version that satisfies the requirement pandas==1.1.0 (from versions: none)
ERROR: No matching distribution found for pandas==1.1.0
WARNING: You are using pip version 20.0.2; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python3.6 -m pip install --upgrade pip' command.
I have tried without any version of pandas but no luck still same error
--install-option.
cmdoptions.check_install_build_global(options)
Looking in indexes: https://pypi.python.org/simple/
ERROR: Could not find a version that satisfies the requirement pandas (from versions: none)
ERROR: No matching distribution found for pandas
WARNING: You are using pip version 20.0.2; however, version 21.3.1 is available.
You should consider upgrading via the '/usr/local/bin/python3.6 -m pip install --upgrade pip' command.
Please help me
Solved using sudo pip install -U <package>
So when I do pip list --outdated I get:
Package Version Latest Type
----------------- ----------- ------ -----
anyio 3.3.3 3.3.4 wheel
cffi 1.14.6 1.15.0 wheel
contextlib2 0.6.0.post1 21.6.0 wheel
idna 3.2 3.3 wheel
ipywidgets 7.6.3 7.6.5 wheel
jsonschema 3.2.0 4.1.0 wheel
jupyter-client 6.1.12 7.0.6 wheel
keyring 23.0.1 23.2.1 wheel
matplotlib-inline 0.1.2 0.1.3 wheel
nbconvert 6.1.0 6.2.0 wheel
packaging 20.9 21.0 wheel
pandocfilters 1.4.3 1.5.0 wheel
pip 20.3.4 21.3 wheel
PyGObject 3.40.1 3.42.0 sdist
pyOpenSSL 20.0.1 21.0.0 wheel
pytz 2021.1 2021.3 wheel
pyzmq 22.2.1 22.3.0 wheel
resolvelib 0.5.5 0.8.1 wheel
setuptools 57.4.0 58.2.0 wheel
sip 4.19.25 6.3.1 wheel
terminado 0.11.1 0.12.1 wheel
trimesh 3.9.31 3.9.32 wheel
websocket-client 0.59.0 1.2.1 wheel
then when I do pip install anyio -U to update the first package I get:
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: anyio in ./.local/lib/python3.9/site-packages (3.3.4)
Requirement already satisfied: idna>=2.8 in ./.local/lib/python3.9/site-packages (from anyio) (3.3)
Requirement already satisfied: sniffio>=1.1 in /usr/lib/python3.9/site-packages (from anyio) (1.2.0)
As I understand the package is already updated, but pip still lists the package as outdated and also doesn't list it in the updated package list.
What is going on?
See this, and as you can guess the problem here is Defaulting to user installation because normal site-packages is not writeable which relates to some permission related issues which I am not sure of.
You can use here python3 -m pip install -U <package>.
A stable solution could be to reinstall python from scratch in your system.
This question already has answers here:
How can I install packages using pip according to the requirements.txt file from a local directory?
(19 answers)
Closed 6 months ago.
I have to install python packages from requirements files that's provided to me. However, when I use pip install -r requirements.txt command I get an error saying ERROR: Invalid requirement (from line 3 in requirements.txt. And when I comment the third line the error just continues to be there for the next lines. What does that mean and how can I install packages from the file?
Here's how the file contents look like:
# Name Version Build Channel
alabaster 0.7.12 py36_0
altgraph 0.17 pypi_0 pypi
appdirs 1.4.4 py_0
argh 0.26.2 py36_0
astroid 2.4.2 py36_0
async_generator 1.10 py36h28b3542_0
atomicwrites 1.4.0 py_0
attrs 20.3.0 pyhd3eb1b0_0
auto-py-to-exe 2.7.11 pypi_0 pypi
autopep8 1.5.4 py_0
babel 2.9.0 pyhd3eb1b0_0
backcall 0.2.0 py_0
bcrypt 3.2.0 py36he774522_0
black 19.10b0 py_0
bleach 3.2.2 pyhd3eb1b0_0
bottle 0.12.19 pypi_0 pypi
... So on
I am using new environment in Anaconda with python version 3.6.12.
First, freeze all of your pip packages in the requirements.txt file using the command
pip freeze > requirements.txt
This should create the requirements.txt file in the correct format. Then try installing using the command
pip install -r requirements.txt
Make sure you're in the same folder as the file when running this command.
If you get some path name instead of the version number in the requirements.txt file, use this pip command to work around it.
pip list --format=freeze > requirements.txt
Change your requirements.txt content as below and try pip install -r requirements.txt again.
alabaster==0.7.12
altgraph==0.17
appdirs== 1.4.4
argh==0.26.2
astroid== 2.4.2
async_generator==1.10
atomicwrites==1.4.0
attrs==20.3.0
auto-py-to-exe==2.7.11
autopep8==1.5.4
babel==2.9.0
backcall==0.2.0
bcrypt==3.2.0
black==19.10b0
bleach==3.2.2
bottle==0.12.19
If you use Anaconda for environment management you most likely created requirements.txt file via:
conda list --explicit > requirements.txt
To recreate the environment with all your listed packages use:
conda env create --file requirements.txt
See CONDA CHEAT SHEET.
I installed python3 and robot framework from command line. When I run robot test cases. I keep getting an error, "robot: command not found". Below is pip list and pip3 list.
Sindoo:~ XXXXXXXXX$ pip list
Package Version
------------------------------ ----------
appdirs 1.4.3
certifi 2020.4.5.1
distlib 0.3.0
filelock 3.0.12
importlib-metadata 1.6.0
pip 20.1
pipenv 2018.11.26
robotframework 3.2
robotframework-seleniumlibrary 3.3.1
selenium 3.141.0
setuptools 46.1.3
six 1.14.0
urllib3 1.25.9
virtualenv 20.0.20
virtualenv-clone 0.5.4
wheel 0.34.2
zipp 3.1.0
Sindoo:~ XXXXXXXXX$ pip3 list
Package Version
------------------------------ ----------
appdirs 1.4.3
certifi 2020.4.5.1
distlib 0.3.0
filelock 3.0.12
importlib-metadata 1.6.0
pip 20.1
pipenv 2018.11.26
robotframework 3.2
robotframework-seleniumlibrary 3.3.1
selenium 3.141.0
setuptools 46.1.3
six 1.14.0
urllib3 1.25.9
virtualenv 20.0.20
virtualenv-clone 0.5.4
wheel 0.34.2
zipp 3.1.0
When I try to check robot --version. I get an error.
robot --version
-bash: robot: command not found
May I know Why robot is not working?
Try running python -m robot or python3 -m robot instead.
I am running a Cookiecutter django project in a docker environment and I would like to add new packages via pip. Specifically I want to add: djangorestframework-jwt
When I do:
docker-compose -f local.yml run --rm django pip install
it seems like it would be perfectly working because I get:
Successfully installed PyJWT-1.7.1 djangorestframework-jwt-1.11.0
Now the problem is that it doesn't install it. It doesn't appear when I run pip freeze, and also not in pip list
Then I tried to put it into my requirements.txt file and run it with:
docker-compose -f local.yml run --rm django pip install -r requirements/base.txt
Same result. It says that it is successfully installed but it is not. I thought it might be a problem with my django version and the package, but the same happens when I try to update my pip. It says it updated, but when I run pip install -upgrade pip I get again:
You should consider upgrading via the 'pip install --upgrade pip' command.
I'm running out of options.
My requirements:
-r ./base.txt
Werkzeug==0.14.1 # https://github.com/pallets/werkzeug
ipdb==0.11 # https://github.com/gotcha/ipdb
Sphinx==1.7.5 # https://github.com/sphinx-doc/sphinx
psycopg2==2.7.4 --no-binary psycopg2 # https://github.com/psycopg/psycopg2
# Testing
# ------------------------------------------------------------------------------
pytest==3.6.3 # https://github.com/pytest-dev/pytest
pytest-sugar==0.9.1 # https://github.com/Frozenball/pytest-sugar
# Code quality
# ------------------------------------------------------------------------------
flake8==3.5.0 # https://github.com/PyCQA/flake8
coverage==4.5.1 # https://github.com/nedbat/coveragepy
# Django
# ------------------------------------------------------------------------------
factory-boy==2.11.1 # https://github.com/FactoryBoy/factory_boy
django-debug-toolbar==1.9.1 # https://github.com/jazzband/django-debug-toolbar
django-extensions==2.0.7 # https://github.com/django-extensions/django-extensions
django-coverage-plugin==1.5.0 # https://github.com/nedbat/django_coverage_plugin
pytest-django==3.3.2 # https://github.com/pytest-dev/pytest-django
djangorestframework-jwt==1.11.0 # https://github.com/GetBlimp/django-rest-framework-jwt
Output of pip list:
Package Version
------------------------ --------
alabaster 0.7.12
argon2-cffi 18.1.0
atomicwrites 1.3.0
attrs 19.1.0
Babel 2.6.0
backcall 0.1.0
certifi 2019.3.9
cffi 1.12.2
chardet 3.0.4
coreapi 2.3.3
coreschema 0.0.4
coverage 4.5.1
decorator 4.4.0
defusedxml 0.5.0
Django 2.0.7
django-allauth 0.36.0
django-coverage-plugin 1.5.0
django-crispy-forms 1.7.2
django-debug-toolbar 1.9.1
django-environ 0.4.5
django-extensions 2.0.7
django-model-utils 3.1.2
django-redis 4.9.0
django-widget-tweaks 1.4.3
djangorestframework 3.8.2
docutils 0.14
factory-boy 2.11.1
Faker 1.0.4
flake8 3.5.0
idna 2.8
imagesize 1.1.0
ipdb 0.11
ipython 7.4.0
ipython-genutils 0.2.0
itypes 1.1.0
jedi 0.13.3
Jinja2 2.10
MarkupSafe 1.1.1
mccabe 0.6.1
more-itertools 6.0.0
oauthlib 3.0.1
packaging 19.0
parso 0.3.4
pexpect 4.6.0
pickleshare 0.7.5
Pillow 5.2.0
pip 19.0.3
pluggy 0.6.0
prompt-toolkit 2.0.9
psycopg2 2.7.4
ptyprocess 0.6.0
py 1.8.0
pycodestyle 2.3.1
pycparser 2.19
pyflakes 1.6.0
Pygments 2.3.1
pyparsing 2.3.1
pytest 3.6.3
pytest-django 3.3.2
pytest-sugar 0.9.1
python-dateutil 2.8.0
python-slugify 1.2.5
python3-openid 3.1.0
pytz 2018.5
redis 3.2.1
requests 2.21.0
requests-oauthlib 1.2.0
setuptools 40.8.0
six 1.12.0
snowballstemmer 1.2.1
Sphinx 1.7.5
sphinxcontrib-websupport 1.1.0
sqlparse 0.3.0
termcolor 1.1.0
text-unidecode 1.2
traitlets 4.3.2
Unidecode 1.0.23
uritemplate 3.0.0
urllib3 1.24.1
wcwidth 0.1.7
Werkzeug 0.14.1
wheel 0.33.1
Any help is highly appreciated! Thanks...
docker-compose run starts a new container and executes the command in it. When used with --rm flag the container gets removed after command completes.
What happens is you get a new container created, and packages installed, or pip upgraded, inside this container. Once the command completes the container is removed.
If later on you run something like docker-compose -f local.yml run --rm pip list a brand new container will get created and pip list executed inside it, showing no packages from previous run since they were installed in a different container, which is already removed.
A better way would be to create docker image that includes your application and install pip packages during docker build. You can check a sample in this question
This way any time you start a container from your image it will have all packages inside.