Segmentation fault when importing torch-sparse (installing pytorch-geometric) - python

I am trying to install pytorch-geometric for a deep-learning project. Torch-sparse is throwing segmentation faults when I attempt to import it (see below). Initially I tried different versions of each required library, as I thought it might be a GPU issue, but I've since tried to simplify by installing cpu-only versions.
Python 3.9.12 (main, Apr 5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> import torch_scatter
>>> import torch_cluster
>>> import torch_sparse
Segmentation fault (core dumped)
And the same issue, presumably due to torch_sparse, when importing pytorch_geometric:
Python 3.9.12 (main, Apr 5 2022, 06:56:58)
[GCC 7.5.0] :: Anaconda, Inc. on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch_geometric
Segmentation fault (core dumped)
I'm on an Ubuntu distribution:
Distributor ID: Ubuntu
Description: Ubuntu 22.04.1 LTS
Release: 22.04
Codename: jammy
Here's my (lightweight for DL) conda installs:
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
blas 1.0 mkl
brotlipy 0.7.0 py310h7f8727e_1002
bzip2 1.0.8 h7b6447c_0
ca-certificates 2022.07.19 h06a4308_0
certifi 2022.9.24 py310h06a4308_0
cffi 1.15.1 py310h74dc2b5_0
charset-normalizer 2.0.4 pyhd3eb1b0_0
cpuonly 2.0 0 pytorch
cryptography 37.0.1 py310h9ce1e76_0
fftw 3.3.9 h27cfd23_1
idna 3.4 py310h06a4308_0
intel-openmp 2021.4.0 h06a4308_3561
jinja2 3.0.3 pyhd3eb1b0_0
joblib 1.1.1 py310h06a4308_0
ld_impl_linux-64 2.38 h1181459_1
libffi 3.3 he6710b0_2
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 11.2.0 h00389a5_1
libgfortran5 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
libuuid 1.0.3 h7f8727e_2
markupsafe 2.1.1 py310h7f8727e_0
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py310h7f8727e_0
mkl_fft 1.3.1 py310hd6ae3a3_0
mkl_random 1.2.2 py310h00e6091_0
ncurses 6.3 h5eee18b_3
numpy 1.23.3 py310hd5efca6_0
numpy-base 1.23.3 py310h8e6c178_0
openssl 1.1.1q h7f8727e_0
pip 22.2.2 py310h06a4308_0
pycparser 2.21 pyhd3eb1b0_0
pyg 2.1.0 py310_torch_1.12.0_cpu pyg
pyopenssl 22.0.0 pyhd3eb1b0_0
pyparsing 3.0.9 py310h06a4308_0
pysocks 1.7.1 py310h06a4308_0
python 3.10.6 haa1d7c7_0
pytorch 1.12.1 py3.10_cpu_0 pytorch
pytorch-cluster 1.6.0 py310_torch_1.12.0_cpu pyg
pytorch-mutex 1.0 cpu pytorch
pytorch-scatter 2.0.9 py310_torch_1.12.0_cpu pyg
pytorch-sparse 0.6.15 py310_torch_1.12.0_cpu pyg
readline 8.1.2 h7f8727e_1
requests 2.28.1 py310h06a4308_0
scikit-learn 1.1.2 py310h6a678d5_0
scipy 1.9.1 py310hd5efca6_0
setuptools 63.4.1 py310h06a4308_0
six 1.16.0 pyhd3eb1b0_1
sqlite 3.39.3 h5082296_0
threadpoolctl 2.2.0 pyh0d69192_0
tk 8.6.12 h1ccaba5_0
tqdm 4.64.1 py310h06a4308_0
typing_extensions 4.3.0 py310h06a4308_0
tzdata 2022e h04d1e81_0
urllib3 1.26.12 py310h06a4308_0
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.6 h5eee18b_0
zlib 1.2.13 h5eee18b_0
Any help would be greatly appreciated!

I've found a combination of packages that works for me - hopefully someone else will have this issue at some point and be able to reproduce the steps from me talking to myself here. The full process for getting stuff working was:
Fresh conda environment with forced Python=3.9 (conda create -n ENVNAME python=3.9)
Activate that environment
Install basic python packages (conda install numpy pandas matplotlib scikit-learn)
Check CUDA version if working with a GPU (nvidia-smi in terminal prints these details for NVIDIA cards)
Install Pytorch using their suggested conda command (conda install pytorch torchvision torchaudio cudatoolkit=CUDA_VERSION -c pytorch -c conda-forge). This had to go through the env solving process on my machine.
Install pytorch geometric (or just torch sparse if that's all you need) with conda install pyg -c pyg. Again this had a solving process.
Check that torch_sparse imports without fault
Here's the conda list for this working combination of packages:
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
blas 1.0 mkl
bottleneck 1.3.5 py39h7deecbd_0
brotli 1.0.9 h5eee18b_7
brotli-bin 1.0.9 h5eee18b_7
brotlipy 0.7.0 py39hb9d737c_1004 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2022.9.24 ha878542_0 conda-forge
certifi 2022.9.24 py39h06a4308_0
cffi 1.14.6 py39he32792d_0 conda-forge
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
cryptography 37.0.2 py39hd97740a_0 conda-forge
cudatoolkit 11.6.0 hecad31d_10 conda-forge
cycler 0.11.0 pyhd3eb1b0_0
dbus 1.13.18 hb2f20db_0
expat 2.4.9 h6a678d5_0
ffmpeg 4.3 hf484d3e_0 pytorch
fftw 3.3.9 h27cfd23_1
fontconfig 2.13.1 h6c09931_0
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.11.0 h70c0345_0
giflib 5.2.1 h7b6447c_0
glib 2.69.1 h4ff587b_1
gmp 6.2.1 h58526e2_0 conda-forge
gnutls 3.6.13 h85f3911_1 conda-forge
gst-plugins-base 1.14.0 h8213a91_2
gstreamer 1.14.0 h28cd5cc_2
icu 58.2 he6710b0_3
idna 3.4 pyhd8ed1ab_0 conda-forge
intel-openmp 2021.4.0 h06a4308_3561
jinja2 3.0.3 pyhd3eb1b0_0
joblib 1.1.1 py39h06a4308_0
jpeg 9e h7f8727e_0
kiwisolver 1.4.2 py39h295c915_0
krb5 1.19.2 hac12032_0
lame 3.100 h7f98852_1001 conda-forge
lcms2 2.12 h3be6417_0
ld_impl_linux-64 2.38 h1181459_1
lerc 3.0 h295c915_0
libbrotlicommon 1.0.9 h5eee18b_7
libbrotlidec 1.0.9 h5eee18b_7
libbrotlienc 1.0.9 h5eee18b_7
libclang 10.0.1 default_hb85057a_2
libdeflate 1.8 h7f8727e_5
libedit 3.1.20210910 h7f8727e_0
libevent 2.1.12 h8f2d780_0
libffi 3.3 he6710b0_2
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 11.2.0 h00389a5_1
libgfortran5 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libiconv 1.17 h166bdaf_0 conda-forge
libllvm10 10.0.1 hbcb73fb_5
libpng 1.6.37 hbc83047_0
libpq 12.9 h16c4e8d_3
libstdcxx-ng 11.2.0 h1234567_1
libtiff 4.4.0 hecacb30_0
libuuid 1.0.3 h7f8727e_2
libwebp 1.2.4 h11a3e52_0
libwebp-base 1.2.4 h5eee18b_0
libxcb 1.15 h7f8727e_0
libxkbcommon 1.0.1 hfa300c1_0
libxml2 2.9.14 h74e7548_0
libxslt 1.1.35 h4e12654_0
lz4-c 1.9.3 h295c915_1
markupsafe 2.1.1 py39h7f8727e_0
matplotlib 3.5.2 py39h06a4308_0
matplotlib-base 3.5.2 py39hf590b9c_0
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py39h7f8727e_0
mkl_fft 1.3.1 py39hd3c417c_0
mkl_random 1.2.2 py39h51133e4_0
munkres 1.1.4 py_0
ncurses 6.3 h5eee18b_3
nettle 3.6 he412f7d_0 conda-forge
nspr 4.33 h295c915_0
nss 3.74 h0370c37_0
numexpr 2.8.3 py39h807cd23_0
numpy 1.23.3 py39h14f4228_0
numpy-base 1.23.3 py39h31eccc5_0
openh264 2.1.1 h780b84a_0 conda-forge
openssl 1.1.1q h7f8727e_0
packaging 21.3 pyhd3eb1b0_0
pandas 1.4.4 py39h6a678d5_0
pcre 8.45 h295c915_0
pillow 9.2.0 py39hace64e9_1
pip 22.2.2 py39h06a4308_0
ply 3.11 py39h06a4308_0
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyg 2.1.0 py39_torch_1.12.0_cu116 pyg
pyopenssl 22.0.0 pyhd8ed1ab_1 conda-forge
pyparsing 3.0.9 py39h06a4308_0
pyqt 5.15.7 py39h6a678d5_1
pyqt5-sip 12.11.0 py39h6a678d5_1
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.9.13 haa1d7c7_2
python-dateutil 2.8.2 pyhd3eb1b0_0
python_abi 3.9 2_cp39 conda-forge
pytorch 1.12.1 py3.9_cuda11.6_cudnn8.3.2_0 pytorch
pytorch-cluster 1.6.0 py39_torch_1.12.0_cu116 pyg
pytorch-mutex 1.0 cuda pytorch
pytorch-scatter 2.0.9 py39_torch_1.12.0_cu116 pyg
pytorch-sparse 0.6.15 py39_torch_1.12.0_cu116 pyg
pytz 2022.1 py39h06a4308_0
qt-main 5.15.2 h327a75a_7
qt-webengine 5.15.9 hd2b0992_4
qtwebkit 5.212 h4eab89a_4
readline 8.2 h5eee18b_0
requests 2.28.1 pyhd8ed1ab_1 conda-forge
scikit-learn 1.1.2 py39h6a678d5_0
scipy 1.9.1 py39h14f4228_0
setuptools 63.4.1 py39h06a4308_0
sip 6.6.2 py39h6a678d5_0
six 1.16.0 pyhd3eb1b0_1
sqlite 3.39.3 h5082296_0
threadpoolctl 2.2.0 pyh0d69192_0
tk 8.6.12 h1ccaba5_0
toml 0.10.2 pyhd3eb1b0_0
torchaudio 0.12.1 py39_cu116 pytorch
torchvision 0.13.1 py39_cu116 pytorch
tornado 6.2 py39h5eee18b_0
tqdm 4.64.1 py39h06a4308_0
typing_extensions 4.4.0 pyha770c72_0 conda-forge
tzdata 2022e h04d1e81_0
urllib3 1.26.11 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.6 h5eee18b_0
zlib 1.2.13 h5eee18b_0
zstd 1.5.2 ha4553b6_0

Related

Why is pip not letting me install torch==1.9.1+cu111 in a new conda env when I have another conda env that has exactly that version?

When I run the pip install in the new conda env:
(base) brando9~ $ pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
ERROR: Could not find a version that satisfies the requirement torch==1.9.1+cu111 (from versions: 1.11.0, 1.11.0+cpu, 1.11.0+cu102, 1.11.0+cu113, 1.11.0+cu115, 1.11.0+rocm4.3.1, 1.11.0+rocm4.5.2, 1.12.0, 1.12.0+cpu, 1.12.0+cu102, 1.12.0+cu113, 1.12.0+cu116, 1.12.0+rocm5.0, 1.12.0+rocm5.1.1, 1.12.1, 1.12.1+cpu, 1.12.1+cu102, 1.12.1+cu113, 1.12.1+cu116, 1.12.1+rocm5.0, 1.12.1+rocm5.1.1, 1.13.0, 1.13.0+cpu, 1.13.0+cu116, 1.13.0+cu117, 1.13.0+cu117.with.pypi.cudnn, 1.13.0+rocm5.1.1, 1.13.0+rocm5.2, 1.13.1, 1.13.1+cpu, 1.13.1+cu116, 1.13.1+cu117, 1.13.1+cu117.with.pypi.cudnn, 1.13.1+rocm5.1.1, 1.13.1+rocm5.2)
ERROR: No matching distribution found for torch==1.9.1+cu111
the other env with that pytorch version:
(metalearning3.9) [pzy2#vision-submit ~]$ pip list
Package Version Location
---------------------------------- -------------------- ------------------
absl-py 1.0.0
aiohttp 3.8.3
aiosignal 1.3.1
alabaster 0.7.12
anaconda-client 1.9.0
anaconda-project 0.10.1
antlr4-python3-runtime 4.8
anyio 2.2.0
appdirs 1.4.4
argcomplete 2.0.0
argh 0.26.2
argon2-cffi 20.1.0
arrow 0.13.1
asn1crypto 1.4.0
astroid 2.6.6
astropy 4.3.1
asttokens 2.0.7
astunparse 1.6.3
async-generator 1.10
async-timeout 4.0.2
atomicwrites 1.4.0
attrs 21.2.0
autopep8 1.5.7
Babel 2.9.1
backcall 0.2.0
backports.shutil-get-terminal-size 1.0.0
beautifulsoup4 4.10.0
binaryornot 0.4.4
bitarray 2.3.0
bkcharts 0.2
black 19.10b0
bleach 4.0.0
bokeh 2.4.1
boto 2.49.0
Bottleneck 1.3.2
brotlipy 0.7.0
cached-property 1.5.2
cachetools 5.0.0
certifi 2021.10.8
cffi 1.14.6
chardet 4.0.0
charset-normalizer 2.0.4
cherry-rl 0.1.4
click 8.0.3
cloudpickle 2.0.0
clyent 1.2.2
colorama 0.4.4
conda 4.12.0
conda-content-trust 0+unknown
conda-pack 0.6.0
conda-package-handling 1.8.0
conda-token 0.3.0
configparser 5.3.0
contextlib2 0.6.0.post1
cookiecutter 1.7.2
crc32c 2.3
crcmod 1.7
cryptography 3.4.8
cycler 0.10.0
Cython 0.29.24
cytoolz 0.11.0
daal4py 2021.3.0
dask 2021.10.0
debugpy 1.4.1
decorator 5.1.0
defusedxml 0.7.1
diff-match-patch 20200713
dill 0.3.4
distributed 2021.10.0
docker-pycreds 0.4.0
docutils 0.17.1
entrypoints 0.3
et-xmlfile 1.1.0
executing 0.9.1
fairseq 0.12.2 /home/pzy2/fairseq
fastcache 1.1.0
fastcluster 1.2.6
fasteners 0.17.3
filelock 3.3.1
flake8 3.9.2
Flask 1.1.2
flatbuffers 2.0.7
fonttools 4.25.0
frozenlist 1.3.0
fsspec 2021.8.1
gast 0.4.0
gcs-oauth2-boto-plugin 3.0
gevent 21.8.0
gitdb 4.0.9
GitPython 3.1.27
glob2 0.7
gmpy2 2.0.8
google-apitools 0.5.32
google-auth 2.6.3
google-auth-oauthlib 0.4.6
google-pasta 0.2.0
google-reauth 0.1.1
gql 0.2.0
graphql-core 1.1
greenlet 1.1.1
grpcio 1.44.0
gsutil 5.9
gym 0.22.0
gym-notices 0.0.6
h5py 3.3.0
HeapDict 1.0.1
higher 0.2.1
html5lib 1.1
httplib2 0.20.4
huggingface-hub 0.5.1
hydra-core 1.0.7
idna 3.2
imagecodecs 2021.8.26
imageio 2.9.0
imagesize 1.2.0
importlib-metadata 4.12.0
inflection 0.5.1
iniconfig 1.1.1
intervaltree 3.1.0
ipykernel 6.4.1
ipython 7.29.0
ipython-genutils 0.2.0
ipywidgets 7.6.5
isort 5.9.3
itsdangerous 2.0.1
jdcal 1.4.1
jedi 0.18.0
jeepney 0.7.1
Jinja2 2.11.3
jinja2-time 0.2.0
joblib 1.1.0
json5 0.9.6
jsonschema 3.2.0
jupyter 1.0.0
jupyter-client 6.1.12
jupyter-console 6.4.0
jupyter-core 4.8.1
jupyter-server 1.4.1
jupyterlab 3.2.1
jupyterlab-pygments 0.1.2
jupyterlab-server 2.8.2
jupyterlab-widgets 1.0.0
keras 2.10.0
Keras-Preprocessing 1.1.2
keyring 23.1.0
kiwisolver 1.3.1
lark-parser 0.12.0
lazy-object-proxy 1.6.0
learn2learn 0.1.7
libarchive-c 2.9
libclang 14.0.6
littleutils 0.2.2
llvmlite 0.37.0
locket 0.2.1
loguru 0.6.0
lxml 4.6.3
Markdown 3.3.6
MarkupSafe 1.1.1
matplotlib 3.4.3
matplotlib-inline 0.1.2
mccabe 0.6.1
mistune 0.8.4
mkl-fft 1.3.1
mkl-random 1.2.2
mkl-service 2.4.0
mock 4.0.3
monotonic 1.6
more-itertools 8.10.0
mpmath 1.2.1
msgpack 1.0.2
multidict 6.0.2
multipledispatch 0.6.0
munkres 1.1.4
mypy-extensions 0.4.3
nbclassic 0.2.6
nbclient 0.5.3
nbconvert 6.1.0
nbformat 5.1.3
nest-asyncio 1.5.1
networkx 2.6.3
nltk 3.6.5
nose 1.3.7
notebook 6.4.5
numba 0.54.1
numexpr 2.7.3
numpy 1.20.3
numpydoc 1.1.0
nvidia-ml-py3 7.352.0
nvidia-smi 0.1.3
oauth2client 4.1.3
oauthlib 3.2.0
olefile 0.46
omegaconf 2.0.6
opencv-python 4.6.0.66
openpyxl 3.0.9
opt-einsum 3.3.0
ordered-set 4.1.0
packaging 21.0
pandas 1.3.4
pandocfilters 1.4.3
parso 0.8.2
partd 1.2.0
path 16.0.0
pathlib2 2.3.6
pathspec 0.7.0
pathtools 0.1.2
patsy 0.5.2
pep8 1.7.1
pexpect 4.8.0
pickleshare 0.7.5
Pillow 8.4.0
pip 22.2.2
pkginfo 1.7.1
plotly 5.7.0
pluggy 0.13.1
ply 3.11
portalocker 2.5.1
poyo 0.5.0
progressbar2 4.0.0
prometheus-client 0.11.0
promise 2.3
prompt-toolkit 3.0.20
protobuf 3.19.6
psutil 5.8.0
ptyprocess 0.7.0
py 1.10.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycodestyle 2.7.0
pycosat 0.6.3
pycparser 2.20
pycurl 7.44.1
pydocstyle 6.1.1
pyerfa 2.0.0
pyflakes 2.3.1
Pygments 2.10.0
pylint 2.9.6
pyls-spyder 0.4.0
pyodbc 4.0.0-unsupported
pyOpenSSL 21.0.0
pyparsing 3.0.4
pyrsistent 0.18.0
PySocks 1.7.1
pytest 6.2.4
python-dateutil 2.8.2
python-lsp-black 1.0.0
python-lsp-jsonrpc 1.0.0
python-lsp-server 1.2.4
python-slugify 5.0.2
python-utils 3.1.0
pytz 2021.3
pyu2f 0.1.5
PyWavelets 1.1.1
pyxdg 0.27
PyYAML 6.0
pyzmq 22.2.1
QDarkStyle 3.0.2
qpth 0.0.15
qstylizer 0.1.10
QtAwesome 1.0.2
qtconsole 5.1.1
QtPy 1.10.0
regex 2021.8.3
requests 2.26.0
requests-oauthlib 1.3.1
retry-decorator 1.1.1
rope 0.19.0
rsa 4.7.2
Rtree 0.9.7
ruamel-yaml-conda 0.15.100
sacrebleu 2.2.0
sacremoses 0.0.49
scikit-image 0.18.3
scikit-learn 0.24.2
scikit-learn-intelex 2021.20210714.170444
scipy 1.7.1
seaborn 0.11.2
SecretStorage 3.3.1
Send2Trash 1.8.0
sentry-sdk 1.5.9
setproctitle 1.2.2
setuptools 58.0.4
shortuuid 1.0.8
simplegeneric 0.8.1
singledispatch 3.7.0
sip 4.19.13
six 1.16.0
sklearn 0.0
smmap 5.0.0
sniffio 1.2.0
snowballstemmer 2.1.0
sorcery 0.2.2
sortedcollections 2.1.0
sortedcontainers 2.4.0
soupsieve 2.2.1
Sphinx 4.2.0
sphinxcontrib-applehelp 1.0.2
sphinxcontrib-devhelp 1.0.2
sphinxcontrib-htmlhelp 2.0.0
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.3
sphinxcontrib-serializinghtml 1.1.5
sphinxcontrib-websupport 1.2.4
spyder 5.1.5
spyder-kernels 2.1.3
SQLAlchemy 1.4.22
statsmodels 0.12.2
subprocess32 3.5.4
sympy 1.9
tables 3.6.1
TBB 0.2
tblib 1.7.0
tensorboard 2.10.1
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.1
tensorflow-estimator 2.10.0
tensorflow-gpu 2.10.1
tensorflow-io-gcs-filesystem 0.27.0
termcolor 2.0.1
terminado 0.9.4
testpath 0.5.0
text-unidecode 1.3
textdistance 4.2.1
tfrecord 1.14.1
threadpoolctl 2.2.0
three-merge 0.1.1
tifffile 2021.7.2
timm 0.6.11
tinycss 0.4
tokenizers 0.11.6
toml 0.10.2
toolz 0.11.1
torch 1.9.1+cu111
torchaudio 0.9.1
torchmeta 1.8.0
torchtext 0.10.1
torchvision 0.10.1+cu111
tornado 6.1
tqdm 4.62.3
traitlets 5.1.0
transformers 4.18.0
typed-ast 1.4.3
typing-extensions 3.10.0.2
ujson 4.0.2
ultimate-anatome 0.1.1
ultimate-aws-cv-task2vec 0.0.1
unicodecsv 0.14.1
Unidecode 1.2.0
urllib3 1.26.7
wandb 0.13.5
watchdog 2.1.3
wcwidth 0.2.5
webencodings 0.5.1
Werkzeug 2.0.2
wheel 0.37.0
whichcraft 0.6.1
widgetsnbextension 3.5.1
wrapt 1.12.1
wurlitzer 2.1.1
xlrd 2.0.1
XlsxWriter 3.0.1
xlwt 1.3.0
yapf 0.31.0
yarl 1.7.2
zict 2.0.0
zipp 3.6.0
zope.event 4.5.0
zope.interface 5.4.0
WARNING: You are using pip version 22.2.2; however, version 22.3.1 is available.
You should consider upgrading via the '/home/pzy2/miniconda3/envs/metalearning3.9/bin/python -m pip install --upgrade pip' command.
(metalearning3.9) [pzy2#vision-submit ~]$
I asked a related question because I can't install pytorch with cuda with conda, see details here: why does conda install the pytorch CPU version despite me putting explicitly to download the cuda toolkit version?
I think this works:
# -- Install PyTorch sometimes requires more careful versioning due to cuda, ref: official install instruction https://pytorch.org/get-started/previous-versions/
# you need python 3.9 for torch version 1.9.1 to work, due to torchmeta==1.8.0 requirement
if ! python -V 2>&1 | grep -q 'Python 3\.9'; then
echo "Error: Python 3.9 is required!"
exit 1
fi
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html
From looking at the link you've provided, I can see
cu111/torch-1.9.1%2Bcu111-cp36-cp36m-linux_x86_64.whl
cu111/torch-1.9.1%2Bcu111-cp36-cp36m-win_amd64.whl
cu111/torch-1.9.1%2Bcu111-cp37-cp37m-linux_x86_64.whl
cu111/torch-1.9.1%2Bcu111-cp37-cp37m-win_amd64.whl
cu111/torch-1.9.1%2Bcu111-cp38-cp38-linux_x86_64.whl
cu111/torch-1.9.1%2Bcu111-cp38-cp38-win_amd64.whl
cu111/torch-1.9.1%2Bcu111-cp39-cp39-linux_x86_64.whl
cu111/torch-1.9.1%2Bcu111-cp39-cp39-win_amd64.whl
cp39 means Python version 3.9. Notice there is no support for 3.10 or 3.11. In your new environment, you probably are running a newer version of Python, whereas in the other environment you have Python 3.6, 3.7, 3.8, or 3.9
To install pytorch 1.9.1cu11 you need python 3.9 to be avaiable. Added that to my bash install.sh
# - create conda env
conda create -n metalearning_gpu python=3.9
conda activate metalearning_gpu
## conda remove --name metalearning_gpu --all
# - make sure pip is up to date
which python
pip install --upgrade pip
pip3 install --upgrade pip
which pip
which pip3
# -- Install PyTorch sometimes requires more careful versioning due to cuda, ref: official install instruction https://pytorch.org/get-started/previous-versions/
# you need python 3.9 for torch version 1.9.1 to work, due to torchmeta==1.8.0 requirement
if ! python -V 2>&1 | grep -q 'Python 3\.9'; then
echo "Error: Python 3.9 is required!"
exit 1
fi
pip install torch==1.9.1+cu111 torchvision==0.10.1+cu111 torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

Getting DLL error when running tensorflow/keras program on python gpu 3.8

I am trying to run my simple AI program but I keep getting this DLL error:
ImportError: Could not find the DLL(s) 'msvcp140_1.dll'. TensorFlow requires that these DLLs be installed in a directory that is named in your %PATH% environment variable.
You may install these DLLs by downloading "Microsoft C++ Redistributable for Visual Studio 2015, 2017 and 2019" for your platform from this URL:
https://support.microsoft.com/help/2977003/the-latest-supported-visual-c-downloads
I went ahead and downloaded that support file and restarted my computer but that didn't work. I also found the specified dll file in my system 32 and made sure it was in my PATH environment for both system and user. Still doesn't work. I am using python3.8 with gpu enabled. Am
I missing something? Is this a version issue because it has never happened on my previous projects.
Here are my libraries:
_tflow_select 2.3.0
absl-py 0.12.0
aggdraw 1.3.12
aiofiles 0.7.0
aiohttp 3.7.4
alembic 1.6.4
argon2-cffi 20.1.0
astor 0.8.1
asttokens 2.0.5
astunparse 1.6.3
async-timeout 3.0.1
async_generator 1.10
attrs 20.3.0
autokeras 1.0.16
backcall 0.2.0
beautifulsoup4 4.9.3
blas 1.0
bleach 3.3.0
blinker 1.4
brotlipy 0.7.0
ca-certificates 2021.7.5
cachetools 4.2.2
certifi 2021.5.30
cffi 1.14.5
chardet 4.0.0
click 7.1.2
cliff 3.8.0
cmaes 0.8.2
cmd2 2.1.2
colorama 0.4.4
colorlog 5.0.1
coverage 5.5
cryptography 3.4.7
cycler 0.10.0
cython 0.29.23
decorator 5.0.9
defusedxml 0.7.1
docopt 0.6.2
entrypoints 0.3
executing 0.6.0
fastapi 0.65.1
flaml 0.6.9
flatbuffers 1.12
freetype 2.10.4
gast 0.4.0
gin-config 0.4.0
google-auth 1.30.0
google-auth-oauthlib 0.4.4
google-pasta 0.2.0
greenlet 1.1.0
grpcio 1.34.1
h11 0.12.0
h5py 3.1.0
hdf5 1.10.5
icc_rt 2019.0.0
icecream 2.1.0
icu 58.2
idna 2.10
importlib-metadata 3.10.0
importlib_metadata 3.10.0
intel-openmp 2021.2.0
ipykernel 5.3.4
ipython 7.25.0
ipython_genutils 0.2.0
ipywidgets 7.6.3
jedi 0.18.0
jinja2 3.0.1
joblib 1.0.1
jpeg 9b
jsonschema 3.2.0
jupyter_client 6.1.12
jupyter_core 4.7.1
jupyterlab_pygments 0.1.2
jupyterlab_widgets 1.0.0
keras 2.7.0
keras-applications 1.0.8
keras-nightly 2.5.0.dev2021032900
keras-preprocessing 1.1.2
keras-tuner 1.0.4
kiwisolver 1.3.1
kt-legacy 1.0.4
libclang 12.0.0
libpng 1.6.37
libprotobuf 3.14.0
libsodium 1.0.18
libtiff 4.1.0
lightgbm 3.3.1
llvmlite 0.37.0
lxml 4.6.3
lz4-c 1.9.3
m2w64-gcc-libgfortran 5.3.0
m2w64-gcc-libs 5.3.0
m2w64-gcc-libs-core 5.3.0
m2w64-gmp 6.1.0
m2w64-libwinpthread-git 5.0.0.4634.697f757
mako 1.1.4
markdown 3.3.4
markupsafe 2.0.1
matplotlib 3.3.4
matplotlib-base 3.3.4
matplotlib-inline 0.1.2
mistune 0.8.4
mkl 2021.2.0
mkl-service 2.3.0
mkl_fft 1.3.0
mkl_random 1.2.1
msys2-conda-epoch 20160418
multidict 5.1.0
nbclient 0.5.3
nbconvert 6.1.0
nbformat 5.1.3
nest-asyncio 1.5.1
notebook 6.4.0
numba 0.54.0
numexpr 2.7.3
numpy 1.19.5
oauthlib 3.1.0
olefile 0.46
opencv-contrib-python 4.5.3.56
opencv-python 4.5.3.56
openssl 1.1.1k
opt-einsum 3.3.0
opt_einsum 3.1.0
optuna 2.8.0
orca 1.6
packaging 20.9
pandas 1.2.4
pandocfilters 1.4.3
parso 0.8.2
pbr 5.6.0
pickleshare 0.7.5
pillow 8.2.0
pip 21.3.1
pipreqs 0.4.10
plotly 4.14.3
plotly-orca 1.3.1
prettytable 2.1.0
prometheus_client 0.11.0
prompt-toolkit 3.0.17
protobuf 3.16.0
psutil 5.8.0
pyasn1 0.4.8
pyasn1-modules 0.2.8
pycparser 2.20
pydantic 1.8.1
pydot 1.4.2
pygments 2.9.0
pyjwt 2.1.0
pyopenssl 20.0.1
pyparsing 2.4.7
pyperclip 1.8.2
pyqt 5.9.2
pyreadline 2.1
pyreadline3 3.3
pyrsistent 0.17.3
pysocks 1.7.1
python 3.8.0
python-dateutil 2.8.1
python-dotenv 0.17.1
python-editor 1.0.4
python-multipart 0.0.5
python_abi 3.8
pytz 2021.1
pywin32 227
pywinpty 0.5.7
pyyaml 5.4.1
pyzmq 20.0.0
qt 5.9.7
requests 2.25.1
requests-oauthlib 1.3.0
retrying 1.3.3
rsa 4.7.2
scikit-learn 0.24.2
scipy 1.6.2
seaborn 0.11.1
send2trash 1.5.0
setuptools 52.0.0
sip 4.19.13
six 1.15.0
soupsieve 2.2.1
sqlalchemy 1.4.22
sqlite 3.35.4
starlette 0.14.2
stevedore 3.3.0
tables 3.6.1
tensorboard 2.7.0
tensorboard-data-server 0.6.1
tensorboard-plugin-wit 1.8.0
tensorflow 2.5.0
tensorflow-addons 0.13.0
tensorflow-base 2.3.0
tensorflow-estimator 2.5.0
tensorflow-gpu 2.4.1
tensorflow-io-gcs-filesystem 0.21.0
termcolor 1.1.0
terminado 0.9.4
testpath 0.5.0
threadpoolctl 2.1.0
tk 8.6.10
toolz 0.11.1
tornado 6.1
tqdm 4.61.2
traitlets 5.0.5
typeguard 2.12.1
typing-extensions 3.7.4.3
typing_extensions 3.7.4.3
urllib3 1.26.4
uvicorn 0.13.4
vc 14.2
visualkeras 0.0.2
vs2015_runtime 14.27.29016
watchgod 0.6
wcwidth 0.2.5
webencodings 0.5.1
websockets 8.1
werkzeug 1.0.1
wheel 0.36.2
widgetsnbextension 3.5.1
win_inet_pton 1.1.0
wincertstore 0.2
winpty 0.4.3
wrapt 1.12.1
xgboost 1.3.3
xz 5.2.5
yaml 0.2.5
yarg 0.1.9
yarl 1.6.3
zeromq 4.3.3
zipp 3.4.1
zlib 1.2.11
zstd 1.4.9
First, check to see if msvcp140.dll (not msvcp140_1.dll) is installed.
Download it here: https://www.microsoft.com/en-us/download/details.aspx?id=53587
Restart your computer and see if it works.
If that doesn't work try copying msvcp140_1.dll into: C:\Users\PCName\AppData\Local\Programs\Python\Python38
If nothing still works, it could be a version issue.
As far as I know, Python 3.8 support requires TensorFlow 2.2 or later.
Run this:
pip install tensorflow ==2.2.0
Look here for system requirements: https://www.tensorflow.org/install/pip#system-requirements
Tensorflow will work only on Windows 7 or later (64-bit) (According to the above link)
Package location: https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-2.7.0-cp38-cp38-win_amd64.whl
If you are using 32-bit python,
replace your 32-bit python with a 64-bit version. Tensorflow does not support 32-bit architecture.

How does one see all the packages that have been installed in developer mode with conda?

I did conda list but I can't see what I am looking for. Are conda packages installed with conda developed . not shown?
# packages in environment at /Users/pinocchio/anaconda3/envs/myenv:
#
# Name Version Build Channel
absl-py 0.9.0 py37_0
appnope 0.1.0 py37hc8dfbb8_1001 conda-forge
asn1crypto 1.3.0 py37_0
astroid 2.3.3 py37_0
attrs 19.3.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
beautifulsoup4 4.8.2 py37_0
blas 1.0 mkl
bleach 3.1.4 pyh9f0ad1d_0 conda-forge
bzip2 1.0.8 h1de35cc_0
c-ares 1.15.0 h1de35cc_1001
ca-certificates 2020.1.1 0
cairo 1.14.12 hc4e6be7_4
certifi 2020.4.5.1 py37_0
cffi 1.14.0 py37hb5b8e2f_0
chardet 3.0.4 py37_1003
conda 4.8.3 py37_0
conda-build 3.18.11 py37_0
conda-package-handling 1.6.0 py37h1de35cc_0
cryptography 2.8 py37ha12b0ac_0
cycler 0.10.0 py37_0
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
entrypoints 0.3 py37hc8dfbb8_1001 conda-forge
expat 2.2.6 h0a44026_0
filelock 3.0.12 py_0
fontconfig 2.13.0 h5d5b041_1
freetype 2.9.1 hb4e5f40_0
fribidi 1.0.5 h1de35cc_0
gettext 0.19.8.1 h15daf44_3
glib 2.63.1 hd977a24_0
glob2 0.7 py_0
graphite2 1.3.13 h2098e52_0
graphviz 2.40.1 hefbbd9a_2
grpcio 1.16.1 py37h044775b_1
harfbuzz 1.8.8 hb8d4a28_0
icu 58.2 h4b95b61_1
idna 2.9 py_1
importlib-metadata 1.6.0 py37hc8dfbb8_0 conda-forge
importlib_metadata 1.6.0 0 conda-forge
intel-openmp 2019.4 233
ipykernel 5.2.0 py37h43977f1_1 conda-forge
ipython 7.13.0 py37hc8dfbb8_2 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isort 4.3.21 py37_0
jedi 0.16.0 py37hc8dfbb8_1 conda-forge
jinja2 2.11.1 py_0
jpeg 9b he5867d9_2
json5 0.9.0 py_0 conda-forge
jsonschema 3.2.0 py37hc8dfbb8_1 conda-forge
jupyter_client 6.1.2 py_0 conda-forge
jupyter_core 4.6.3 py37hc8dfbb8_1 conda-forge
jupyterlab 2.0.1 py_0 conda-forge
jupyterlab_server 1.0.7 py_0 conda-forge
kiwisolver 1.1.0 py37h0a44026_0
lazy-object-proxy 1.4.3 py37h1de35cc_0
libarchive 3.3.3 h786848e_5
libcxx 4.0.1 hcfea43d_1
libcxxabi 4.0.1 hcfea43d_1
libedit 3.1.20181209 hb402a30_0
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
liblief 0.9.0 h2a1bed3_2
libpng 1.6.37 ha441bb4_0
libprotobuf 3.11.4 hd9629dc_0
libsodium 1.0.17 h01d97ff_0 conda-forge
libtiff 4.1.0 hcb84e12_0
libxml2 2.9.9 hf6e021a_1
lz4-c 1.8.1.2 h1de35cc_0
lzo 2.10 h362108e_2
markdown 3.1.1 py37_0
markupsafe 1.1.1 py37h1de35cc_0
matplotlib 3.1.3 py37_0
matplotlib-base 3.1.3 py37h9aa3819_0
mccabe 0.6.1 py37_1
mistune 0.8.4 py37h0b31af3_1000 conda-forge
mkl 2019.4 233
mkl-service 2.3.0 py37hfbe908c_0
mkl_fft 1.0.15 py37h5e564d8_0
mkl_random 1.1.0 py37ha771720_0
nbconvert 5.6.1 py37_0 conda-forge
nbformat 5.0.4 py_0 conda-forge
ncurses 6.2 h0a44026_0
ninja 1.9.0 py37h04f5b5a_0
notebook 6.0.3 py37_0 conda-forge
numpy 1.18.1 py37h7241aed_0
numpy-base 1.18.1 py37h6575580_1
olefile 0.46 py37_0
openssl 1.1.1g h1de35cc_0
pandoc 2.9.2 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
pango 1.42.4 h060686c_0
parso 0.6.2 py_0 conda-forge
pcre 8.43 h0a44026_0
pexpect 4.8.0 py37hc8dfbb8_1 conda-forge
pickleshare 0.7.5 py37hc8dfbb8_1001 conda-forge
pillow 6.2.1 py37hb68e598_0
pip 20.0.2 py37_1
pixman 0.38.0 h1de35cc_0
pkginfo 1.5.0.1 py37_0
prometheus_client 0.7.1 py_0 conda-forge
prompt-toolkit 3.0.5 py_0 conda-forge
protobuf 3.11.4 py37h0a44026_0
psutil 5.7.0 py37h1de35cc_0
ptyprocess 0.6.0 py_1001 conda-forge
py-lief 0.9.0 py37h1413db1_2
pycosat 0.6.3 py37h1de35cc_0
pycparser 2.20 py_0
pygments 2.6.1 py_0 conda-forge
pylint 2.4.4 py37_0
pyopenssl 19.1.0 py37_0
pyparsing 2.4.6 py_0
pyrsistent 0.16.0 py37h9bfed18_0 conda-forge
pysocks 1.7.1 py37_0
python 3.7.7 hc70fcce_0_cpython
python-dateutil 2.8.1 py_0 conda-forge
python-graphviz 0.13.2 pypi_0 pypi
python-libarchive-c 2.8 py37_13
python_abi 3.7 1_cp37m conda-forge
pytorch 1.4.0 py3.7_0 pytorch
pytz 2019.3 py_0
pyyaml 5.3.1 py37h1de35cc_0
pyzmq 18.1.1 py37h0a44026_0
readline 8.0 h1de35cc_0
requests 2.23.0 py37_0
ripgrep 11.0.2 he32d670_0
ruamel_yaml 0.15.87 py37h1de35cc_0
send2trash 1.5.0 py_0 conda-forge
setuptools 46.1.1 py37_0
six 1.14.0 py37_0
soupsieve 2.0 py_0
sqlite 3.31.1 ha441bb4_0
tensorboard 2.0.0 pyhb38c66f_1
terminado 0.8.3 py37hc8dfbb8_1 conda-forge
testpath 0.4.4 py_0 conda-forge
tk 8.6.8 ha441bb4_0
torchvision 0.2.1 py_2 soumith
tornado 6.0.4 py37h9bfed18_1 conda-forge
tqdm 4.43.0 py_0
traitlets 4.3.3 py37hc8dfbb8_1 conda-forge
urllib3 1.25.8 py37_0
wcwidth 0.1.9 pyh9f0ad1d_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
werkzeug 1.0.0 py_0
wheel 0.34.2 py37_0
wrapt 1.12.1 py37h1de35cc_1
xz 5.2.4 h1de35cc_4
yaml 0.1.7 hc338f04_2
zeromq 4.3.2 h6de7cb9_2 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h1de35cc_3
zstd 1.3.7 h5bba6e5_0
utils or something like that should be there but its not.
Update:
I've tried what this suggests and it does not work. I conda develop . my package and then when I run the python interpreter and try to import it the import fails. Why?
>>> import my_pkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_pkg'
I also tried to install it with pip pip install -e and it doesn't find my package either after I go to the interpreter and try to import it. Why is that?
If you look carefully after running conda develop . (where the setup.py file is) you will notice that the command outputs the following:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop .
added /Users/brandBrandoParetoopareto/ultimate-utils/uutils
completed operation for: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
if you then check the sys.path python has then you can that new path has been added:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
/Users/brandBrandoParetoopareto/ultimate-utils/uutils
you can confirm this by removing it and printing the contents of sys.path:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop -u .
uninstalled: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
check sys path again:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
now it's gone!
Also, note that conda or python (not sure which) also adds the current path always. This is important to note because when I did import tests outside of the repo I was working it wasn't working but it was inside the repo which seemed very mysterious which I can only attribute to that.
Addendum
to check python sys.path from terminal run that command as a string with python cmd:
python -c "import sys; print(sys.path)"
even better:
python -c "import sys; [print(p) for p in sys.path]"
Original answer: https://stackoverflow.com/a/59903590/1601580

How is it that torch is not installed by torchvision?

Somehow when I do the install it installs torchvision but not torch. Command I am running as dictated from the main website:
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
then I do conda list but look:
$ conda list
# packages in environment at /home/ubuntu/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
alabaster 0.7.10 py36h306e16b_0
anaconda-client 1.6.14 py36_0
anaconda-project 0.8.2 py36h44fb852_0
argparse 1.4.0 <pip>
asn1crypto 0.24.0 py36_0
astroid 1.6.3 py36_0
astropy 3.0.2 py36h3010b51_1
attrs 18.1.0 py36_0
autovizwidget 0.12.7 <pip>
babel 2.5.3 py36_0
backcall 0.1.0 py36_0
backports 1.0 py36hfa02d7e_1
backports.shutil_get_terminal_size 1.0.0 py36hfea85ff_2
bcrypt 3.1.6 <pip>
beautifulsoup4 4.6.0 py36h49b8c8c_1
bitarray 0.8.1 py36h14c3975_1
bkcharts 0.2 py36h735825a_0
blas 1.0 mkl
blaze 0.11.3 py36h4e06776_0
bleach 2.1.3 py36_0
blosc 1.14.3 hdbcaa40_0
bokeh 1.0.4 py36_0
boto 2.48.0 py36h6e4cd66_1
boto3 1.9.146 <pip>
boto3 1.9.134 py_0
botocore 1.12.146 <pip>
botocore 1.12.134 py_0
bottleneck 1.2.1 py36haac1ea0_0
bzip2 1.0.6 h14c3975_5
ca-certificates 2019.1.23 0
cached-property 1.5.1 <pip>
cairo 1.14.12 h8948797_3
certifi 2019.3.9 py36_0
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
click 6.7 py36h5253387_0
cloudpickle 0.5.3 py36_0
clyent 1.2.2 py36h7e57e65_1
colorama 0.3.9 py36h489cec4_0
contextlib2 0.5.5 py36h6c84a62_0
cryptography 2.3.1 py36hc365091_0
cudatoolkit 10.0.130 0
curl 7.60.0 h84994c4_0
cycler 0.10.0 py36h93f1223_0
cymem 2.0.2 py36hfd86e86_0
cython 0.28.2 py36h14c3975_0
cytoolz 0.9.0.1 py36h14c3975_0
dask 0.17.5 py36_0
dask-core 0.17.5 py36_0
dataclasses 0.6 py_0 fastai
datashape 0.5.4 py36h3ad6b5c_0
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py36_0
defusedxml 0.6.0 py_0
dill 0.2.9 py36_0
distributed 1.21.8 py36_0
docker 3.7.2 <pip>
docker-compose 1.24.0 <pip>
docker-pycreds 0.4.0 <pip>
dockerpty 0.4.1 <pip>
docopt 0.6.2 <pip>
docutils 0.14 py36hb0f60f5_0
entrypoints 0.2.3 py36h1aec115_2
environment-kernels 1.1.1 <pip>
et_xmlfile 1.0.1 py36hd6bccc3_0
expat 2.2.5 he0dffb1_0
fastai 1.0.52 1 fastai
fastcache 1.0.2 py36h14c3975_2
fastprogress 0.1.21 py_0 fastai
filelock 3.0.4 py36_0
flask 1.0.2 py36_1
flask-cors 3.0.4 py36_0
fontconfig 2.13.0 h9420a91_0
freetype 2.9.1 h8a8886c_1
fribidi 1.0.5 h7b6447c_0
get_terminal_size 1.0.0 haa9412d_0
gevent 1.3.0 py36h14c3975_0
glib 2.56.1 h000015b_0
glob2 0.6 py36he249c77_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py36hc8893dd_2
graphite2 1.3.11 h16798f4_2
graphviz 2.40.1 h21bd128_2
greenlet 0.4.13 py36h14c3975_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.8.0 py36h989c5e5_3
harfbuzz 1.8.4 hec2c2bc_0
hdf5 1.10.2 hba1933b_1
hdijupyterutils 0.12.7 <pip>
heapdict 1.0.0 py36_2
html5lib 1.0.1 py36h2f9c1c0_0
icu 58.2 h9c2bf20_1
idna 2.6 py36h82fb2a8_1
imageio 2.3.0 py36_0
imagesize 1.0.0 py36_0
intel-openmp 2018.0.0 8
ipykernel 4.8.2 py36_0
ipyparallel 6.2.2 <pip>
ipython 6.4.0 py36_0
ipython_genutils 0.2.0 py36hb52b0d5_0
ipywidgets 7.2.1 py36_0
ipywidgets 7.4.0 <pip>
isort 4.3.4 py36_0
itsdangerous 0.24 py36h93cc618_1
jbig 2.1 hdba287a_0
jdcal 1.4 py36_0
jedi 0.12.0 py36_1
jinja2 2.10 py36ha16c418_0
jmespath 0.9.4 py_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py36h006f8b5_0
jupyter 1.0.0 py36_4
jupyter_client 5.2.3 py36_0
jupyter_console 5.2.0 py36he59e554_1
jupyter_core 4.4.0 py36h7c827e3_0
jupyterlab 0.32.1 py36_0
jupyterlab_launcher 0.10.5 py36_0
kiwisolver 1.0.1 py36h764f252_0
krb5 1.14.2 hcdc1b81_6
lazy-object-proxy 1.3.1 py36h10fcdad_0
libcurl 7.60.0 h1ad7b7a_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran 3.0.0 1 conda-forge
libgfortran-ng 7.2.0 hdf63c60_3
libpng 1.6.37 hbc83047_0
libprotobuf 3.5.2 hd28b015_1 conda-forge
libsodium 1.0.16 h1bed415_0
libssh2 1.8.0 h9cfc8f7_4
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_1
libtool 2.4.6 h544aabb_3
libuuid 1.0.3 h1bed415_2
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
libxslt 1.1.32 h1312cb7_0
llvmlite 0.23.1 py36hdbcaa40_0
locket 0.2.0 py36h787c0ad_1
lxml 4.2.1 py36h23eabaa_0
lzo 2.10 h49e0be7_2
markupsafe 1.0 py36hd9260cd_1
matplotlib 2.2.2 <pip>
matplotlib 3.0.3 py36h5429711_0
mccabe 0.6.1 py36h5ad9710_1
mistune 0.8.3 py36h14c3975_1
mkl 2018.0.3 1
mkl-service 1.1.2 py36h17a0993_4
mkl_fft 1.0.6 py36h7dd41cf_0
mkl_random 1.0.1 py36h629b387_0
mock 3.0.5 <pip>
more-itertools 4.1.0 py36_0
mpc 1.0.3 hec55b23_5
mpfr 3.1.5 h11a74b3_2
mpi 1.0 openmpi conda-forge
mpmath 1.0.0 py36hfeacd6b_2
msgpack 0.6.0 <pip>
msgpack-numpy 0.4.3.2 py36_0
msgpack-python 0.5.6 py36h6bb024c_0
multipledispatch 0.5.0 py36_0
murmurhash 1.0.2 py36he6710b0_0
nb_conda 2.2.1 py36_2 conda-forge
nb_conda_kernels 2.2.1 py36_0 conda-forge
nbconvert 5.4.1 py36_3
nbformat 4.4.0 py36h31c9010_0
ncurses 6.1 hf484d3e_0
networkx 2.1 py36_0
ninja 1.8.2 py36h6bb024c_1
nltk 3.3.0 py36_0
nose 1.3.7 py36hcdf7029_2
notebook 5.5.0 py36_0
numba 0.38.0 py36h637b7d7_0
numexpr 2.6.5 py36h7bf3b9c_0
numpy 1.15.4 py36h1d66e8a_0
numpy 1.15.4 <pip>
numpy-base 1.15.4 py36h81de0dd_0
numpydoc 0.8.0 py36_0
nvidia-ml-py3 7.352.0 py_0 fastai
odo 0.5.1 py36h90ed295_0
olefile 0.45.1 py36_0
onnx 1.4.1 <pip>
openmpi 3.1.0 h26a2512_3 conda-forge
openpyxl 2.5.3 py36_0
openssl 1.0.2r h7b6447c_0
packaging 17.1 py36_0
pandas 0.24.2 <pip>
pandas 0.23.0 py36h637b7d7_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py36ha6701b7_1
pango 1.42.3 h8589676_0
paramiko 2.4.2 <pip>
parso 0.2.0 py36_0
partd 0.3.8 py36h36fd896_0
patchelf 0.9 hf79760b_2
path.py 11.0.1 py36_0
pathlib2 2.3.2 py36_0
patsy 0.5.0 py36_0
pcre 8.42 h439df22_0
pep8 1.7.1 py36_0
pexpect 4.5.0 py36_0
pickleshare 0.7.4 py36h63277f8_0
pillow 5.2.0 py36heded4f4_0
pip 10.0.1 py36_0
pixman 0.34.0 hceecf20_3
pkginfo 1.4.2 py36_1
plac 0.9.6 py36_0
plotly 2.7.0 <pip>
pluggy 0.6.0 py36hb689045_0
ply 3.11 py36_0
preshed 2.0.1 py36he6710b0_0
prompt_toolkit 1.0.15 py36h17d85b1_0
protobuf 3.5.2 py36hd28b015_0 conda-forge
protobuf3-to-dict 0.1.5 <pip>
psutil 5.4.5 py36h14c3975_0
psycopg2 2.7.5 <pip>
ptyprocess 0.5.2 py36h69acd42_0
py 1.5.3 py36_0
py4j 0.10.7 <pip>
pyasn1 0.4.5 <pip>
pycodestyle 2.4.0 py36_0
pycosat 0.6.3 py36h0a5515d_0
pycparser 2.18 py36hf9f622e_1
pycrypto 2.6.1 py36h14c3975_8
pycurl 7.43.0.1 py36hb7f436b_0
pyflakes 1.6.0 py36h7bd6a15_0
pygal 2.4.0 <pip>
pygments 2.2.0 py36h0d3125c_0
pykerberos 1.2.1 py36h14c3975_0
pylint 1.8.4 py36_0
PyNaCl 1.3.0 <pip>
pyodbc 4.0.23 py36hf484d3e_0
pyopenssl 18.0.0 py36_0
pyparsing 2.2.0 py36hee85983_1
pyqt 5.9.2 py36h751905a_0
pysocks 1.6.8 py36_0
pyspark 2.3.2 <pip>
pytables 3.4.3 py36h02b9ad4_2
pytest 3.5.1 py36_0
pytest-arraydiff 0.2 py36_0
pytest-astropy 0.3.0 py36_0
pytest-doctestplus 0.1.3 py36_0
pytest-openfiles 0.3.0 py36_0
pytest-remotedata 0.2.1 py36_0
python 3.6.5 hc3d631a_2
python-dateutil 2.7.3 py36_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
pytz 2018.4 py36_0
pywavelets 0.5.2 py36he602eb0_0
pyyaml 3.12 py36hafb9ca4_1
pyzmq 17.0.0 py36h14c3975_0
qt 5.9.6 h52aff34_0
qtawesome 0.4.4 py36h609ed8c_0
qtconsole 4.3.1 py36h8f73b5b_0
qtpy 1.4.1 py36_0
readline 7.0 ha6073c6_4
regex 2018.01.10 py36h14c3975_1000 fastai
requests 2.20.0 py36_1000 conda-forge
requests-kerberos 0.12.0 <pip>
rope 0.10.7 py36h147e2ec_0
ruamel_yaml 0.15.35 py36h14c3975_1
s3fs 0.1.5 py36_0
s3transfer 0.2.0 <pip>
s3transfer 0.2.0 py36_0
sagemaker 1.20.1 <pip>
sagemaker-pyspark 1.2.4 <pip>
scikit-image 0.13.1 py36h14c3975_1
scikit-learn 0.19.1 py36h7aa7ec6_0
scikit-learn 0.20.3 <pip>
scipy 1.1.0 py36hfc37229_0
seaborn 0.8.1 py36hfad7ec4_0
send2trash 1.5.0 py36_0
setuptools 39.1.0 py36_0
simplegeneric 0.8.1 py36_2
singledispatch 3.4.0.3 py36h7a266c3_0
sip 4.19.8 py36hf484d3e_0
six 1.11.0 py36h372c433_1
snappy 1.1.7 hbae5bb6_3
snowballstemmer 1.2.1 py36h6febd40_0
sortedcollections 0.6.1 py36_0
sortedcontainers 1.5.10 py36_0
spacy 2.0.18 py36hf484d3e_1000 fastai
sparkmagic 0.12.5 <pip>
sphinx 1.7.4 py36_0
sphinxcontrib 1.0 py36h6d0f590_1
sphinxcontrib-websupport 1.0.1 py36hb5cb234_1
spyder 3.2.8 py36_0
SQLAlchemy 1.2.11 <pip>
sqlalchemy 1.2.7 py36h6b74fdf_0
sqlite 3.23.1 he433501_0
statsmodels 0.9.0 py36h3010b51_0
sympy 1.1.1 py36hc6d1c1c_0
tblib 1.3.2 py36h34cf8b6_0
terminado 0.8.1 py36_1
testpath 0.3.1 py36h8cadb63_0
texttable 0.9.1 <pip>
thinc 6.12.1 py36h637b7d7_1000 fastai
tk 8.6.8 hbc83047_0
toolz 0.9.0 py36_0
torchvision 0.2.2 py_3 pytorch
tornado 5.0.2 py36_0
tqdm 4.31.1 py36_1
traitlets 4.3.2 py36h674d592_0
typing 3.6.4 py36_0
typing-extensions 3.7.2 <pip>
ujson 1.35 py36h14c3975_0
unicodecsv 0.14.1 py36ha668878_0
unixodbc 2.3.6 h1bed415_0
urllib3 1.23 py36_0
wcwidth 0.1.7 py36hdf4376a_0
webencodings 0.5.1 py36h800622e_1
websocket-client 0.56.0 <pip>
werkzeug 0.14.1 py36_0
wheel 0.31.1 py36_0
widgetsnbextension 3.2.1 py36_0
widgetsnbextension 3.4.2 <pip>
wrapt 1.10.11 py36h28b7045_0
xlrd 1.1.0 py36h1db9f0c_1
xlsxwriter 1.0.4 py36_0
xlwt 1.3.0 py36h7b00a1f_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.2.5 h439df22_0
zict 0.1.3 py36h3a3bf81_0
zlib 1.2.11 ha838bed_2
I was told I do have pytorch installed but my script keeps giving me this error:
$ cat nohup.out
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
does that mean that I need to install it as pytroch and not torch? Is this not weird?
Note I am running this on an AWS instance p3.2xlarge. This keeps happening when I log out and then go back in that my torch package gets missing...?!?! :/
original post: https://discuss.pytorch.org/t/torchvision-installed-but-not-torch/51758
The issue persists even if I open just a python interactive and try to import it:
(pytorch_p36) ubuntu#ip-123-12-21-123:~$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
It also happens running the script directly:
(pytorch_p36) ubuntu#ip-123-12-21-123:~/project/folder$ python high_performing_data_point_models_cifar10.py
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
I can't import torchvision either!
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchvision
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torchvision'
>>>
Your conda list command shows that it was run from the environment called automl:
# packages in environment at /home/ubuntu/anaconda3/envs/automl:
However, when you show the commands that you are trying to run, you are doing so from the (pytorch_p36) environment.
You should run your conda install command while inside this pytorch_p36 environment.
Go to the official website and try installing it as said over there. As I just followed the official documentation and it is working fine.
Here is the step though.
conda install pytorch-cpu torchvision-cpu -c pytorch
You can follow the official documentation.
link
It is really easy to miss smth while working with several machines and several conda/python environments. I think it is a good idea to start from the very beginning. Here is installation process that works fine for me.
0. Connect to your aws-instance using SSH or PuTTY.
1. Create conda env named pytorch_p36 with python 3.6 on your aws machine:
user#aws-instance:~$ conda create -n pytorch_p36 python=3.6
2. Activate it:
user#aws-instance:~$ conda activate pytorch_p36
Or with (for older conda versions):
user#aws-instance:~$ source activate pytorch_p36
Now you should see (pytorch_p36) before your shell prompt:
(pytorch_p36) user#aws-instance:~$
3. Go to PyTorch website and choose appropriate installation command via conda. Run it in your shell:
(pytorch_p36) user#aws-instance:~$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
4. Verify the installation:
(pytorch_p36) user#aws-instance:~$ conda list
# packages in environment at /home/user/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
...
cudatoolkit 10.0.130 0
...
python 3.6.9 h265db76_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
...
torchvision 0.3.0 py36_cu10.0.130_1 pytorch
...
5. Verify the running with nohup:
(pytorch_p36) user#aws-instance:~$ nohup python -c "import torch; import torchvision; print('PyTorch is fine!')" && cat nohup.out
nohup: ignoring input and appending output to 'nohup.out'
PyTorch is fine!
AWS Machine Learning AMI already has pytorch v1.1.0 and torchvision v0.2.2 installed in the predefined pytorch_p36 virtual environment. So you are not required to create new venv and install pytorch each time you log in. All you need is to run :
$ source activate pytorch_p36
(without dollar sign, obviously). Then, if you run pip show torch you will see that it is already there.

Install pytorch fail on win10 external GPU

I can not import pytorch on my gpu conda env:
C:\Users\Jeffy\Desktop
$ python
Python 3.7.2 (default, Feb 11 2019, 14:11:50) [MSC v.1915 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\ProgramData\Anaconda3\envs\gpu\lib\site-packages\torch\__init__.py", line 84, in <module>
from torch._C import *
ImportError: DLL load failed: The specified module could not be found.
I have two conda env, one is gpu with external gpu GTX1050, one is base.
On my base env, I have installed pytorch-cpu version and it works well.
However, I cannot install pytorch gpu version on my gpu env.
on my gpu env, I have the following packages installed (including cudnn, intel-openmp, cmake and so on):
$ conda list
packages in environment at C:\ProgramData\Anaconda3\envs\gpu:
Name Version Build Channel
absl-py 0.7.0 pypi_0 pypi
astor 0.7.1 pypi_0 pypi
blas 1.0 mkl
ca-certificates 2019.1.23 0
certifi 2018.11.29 py37_0
cffi 1.11.5 py37h74b6da3_1
cmake 3.12.2 he025d50_0
cudatoolkit 10.0.130 0
cudnn 7.3.1 cuda10.0_0
freetype 2.9.1 ha9979f8_1
gast 0.2.2 pypi_0 pypi
grpcio 1.18.0 pypi_0 pypi
h5py 2.9.0 pypi_0 pypi
icc_rt 2019.0.0 h0cc432a_1
intel-openmp 2019.0 pypi_0 pypi
jpeg 9b hb83a4c4_2
keras-applications 1.0.7 pypi_0 pypi
keras-preprocessing 1.0.9 pypi_0 pypi
libpng 1.6.36 h2a8f88b_0
libtiff 4.0.10 hb898794_2
markdown 3.0.1 pypi_0 pypi
mkl 2019.1 144
mkl-include 2019.1 144
mkl_fft 1.0.10 py37h14836fe_0
mkl_random 1.0.2 py37h343c172_0
mock 2.0.0 pypi_0 pypi
ninja 1.8.2.post2 pypi_0 pypi
numpy 1.15.4 py37h19fb1c0_0
numpy-base 1.15.4 py37hc3f5095_0
olefile 0.46 py37_0
openssl 1.1.1a he774522_0
pbr 5.1.2 pypi_0 pypi
pillow 5.4.1 py37hdc69c19_0
pip 19.0.1 py37_0
protobuf 3.6.1 pypi_0 pypi
pycparser 2.19 py37_0
python 3.7.2 h8c8aaf0_2
pytorch 1.0.1 py3.7_cuda100_cudnn7_1 pytorch
pyyaml 3.13 py37hfa6e2cd_0
setuptools 40.7.3 py37_0
six 1.12.0 py37_0
sqlite 3.26.0 he774522_0
tensorboard 1.12.2 pypi_0 pypi
tensorflow-estimator 1.13.0rc0 pypi_0 pypi
tensorflow-gpu 1.13.0rc1 pypi_0 pypi
termcolor 1.1.0 pypi_0 pypi
tk 8.6.8 hfa6e2cd_0
torchvision 0.2.1 py_2 pytorch
typing 3.6.4 py37_0
vc 14.1 h21ff451_1 peterjc123
vs2015_runtime 14.15.26706 h3a45250_0
vs2017_runtime 15.4.27004.2010 1 peterjc123
werkzeug 0.14.1 pypi_0 pypi
wheel 0.32.3 py37_0
wincertstore 0.2 py37_0
xz 5.2.4 h2fa13f4_4
yaml 0.1.7 hc54c509_2
zlib 1.2.11 h62dcd97_3
zstd 1.3.7 h508b16e_0
Expecting Cuda and Cudnn has been already installed and Updated the Environment variable list.
Try installing pytorch using the command
conda install pytorch -c pytorch
or
conda install pytorch torchvision cudatoolkit=10.0.130 -c pytorch

Categories