I tried to install gdal package on my conda environment. I activated gcpy environment and installed the gdal package using conda install -c conda-forge gdal. The package installs successfully. But, when I tried to import the package, I get error:
In [1]: import gdal
---------------------------------------------------------------------------
ModuleNotFoundError Traceback (most recent call last)
<ipython-input-1-ee38efafc30b> in <module>
----> 1 import gdal
ModuleNotFoundError: No module named 'gdal'
I can see the package in the conda list.
gdal 3.2.1 py39h409cc32_1 conda-forge
geos 3.8.1 he1b5a44_0 conda-forge
geoschem-gcpy 1.0.0 py39hf3d152e_0 conda-forge
geotiff 1.6.0 h5d11630_3 conda-forge
gettext 0.19.8.1 h0b5b191_1005 conda-forge
giflib 5.2.1 h36c2ea0_2 conda-forge
glib 2.66.4 hc4f0c31_2 conda-forge
glib-tools 2.66.4 hc4f0c31_2 conda-forge
gst-plugins-base 1.14.5 h0935bb2_2 conda-forge
gstreamer 1.18.3 h3560a44_0 conda-forge
h5netcdf 0.8.1 py_0 conda-forge
h5py 3.1.0 nompi_py39h25020de_100 conda-forge
hdf4 4.2.13 h10796ff_1004 conda-forge
hdf5 1.10.6 nompi_h6a2412b_1114 conda-forge
heapdict 1.0.1 py_0 conda-forge
helpdev 0.7.1 pyhd8ed1ab_0 conda-forge
icu 68.1 h58526e2_0 conda-forge
idna 2.10 pyh9f0ad1d_0 conda-forge
imagesize 1.2.0 py_0 conda-forge
importlib-metadata 3.4.0 py39hf3d152e_0 conda-forge
importlib_metadata 3.4.0 hd8ed1ab_0 conda-forge
intervaltree 3.0.2 py_0 conda-forge
ipykernel 5.4.2 py39hef51801_0 conda-forge
ipython 7.19.0 py39hef51801_2 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isort 5.7.0 pyhd8ed1ab_0 conda-forge
jedi 0.17.2 py39hf3d152e_1 conda-forge
jeepney 0.6.0 pyhd8ed1ab_0 conda-forge
jinja2 2.11.2 pyh9f0ad1d_0 conda-forge
joblib 1.0.0 pyhd8ed1ab_0 conda-forge
jpeg 9d h36c2ea0_0 conda-forge
json-c 0.13.1 hbfbb72e_1002 conda-forge
jsonschema 3.2.0 py_2 conda-forge
jupyter_client 6.1.11 pyhd8ed1ab_1 conda-forge
jupyter_core 4.7.0 py39hf3d152e_1 conda-forge
jupyterlab_pygments 0.1.2 pyh9f0ad1d_0 conda-forge
kealib 1.4.14 h0042707_0 conda-forge
keyring 22.0.1 py39hf3d152e_0 conda-forge
kiwisolver 1.3.1 py39h1a9c180_1 conda-forge
krb5 1.17.2 h926e7f8_0 conda-forge
lazy-object-proxy 1.4.3 py39h07f9747_2 conda-forge
lcms2 2.11 hcbb858e_1 conda-forge
ld_impl_linux-64 2.35.1 hea4e1c9_1 conda-forge
libblas 3.9.0 7_openblas conda-forge
libcblas 3.9.0 7_openblas conda-forge
libclang 11.0.1 default_ha53f305_1 conda-forge
libcurl 7.71.1 hcdd3856_8 conda-forge
libdap4 3.20.6 h1d1bd15_1 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libevent 2.1.10 hcdb4288_3 conda-forge
libffi 3.3 h58526e2_2 conda-forge
libgcc-ng 9.3.0 h2828fa1_18 conda-forge
libgdal 3.2.1 h52563cd_1 conda-forge
libgfortran-ng 9.3.0 hff62375_18 conda-forge
libgfortran5 9.3.0 hff62375_18 conda-forge
libglib 2.66.4 h748fe8e_2 conda-forge
libgomp 9.3.0 h2828fa1_18 conda-forge
libiconv 1.16 h516909a_0 conda-forge
libkml 1.3.0 h74f7ee3_1012 conda-forge
liblapack 3.9.0 7_openblas conda-forge
libllvm11 11.0.1 hf817b99_0 conda-forge
libnetcdf 4.7.4 nompi_h56d31a8_107 conda-forge
libnghttp2 1.41.0 h8cfc5f6_2 conda-forge
libopenblas 0.3.12 pthreads_h4812303_1 conda-forge
libpng 1.6.37 h21135ba_2 conda-forge
libpq 12.3 h255efa7_3 conda-forge
libsodium 1.0.18 h36c2ea0_1 conda-forge
libspatialindex 1.9.3 he1b5a44_3 conda-forge
libspatialite 5.0.0 heaf302f_0 conda-forge
libssh2 1.9.0 hab1572f_5 conda-forge
libstdcxx-ng 9.3.0 h6de172a_18 conda-forge
libtiff 4.2.0 hdc55705_0 conda-forge
libuuid 2.32.1 h7f98852_1000 conda-forge
libwebp-base 1.1.0 h36c2ea0_3 conda-forge
libxcb 1.13 h7f98852_1003 conda-forge
libxkbcommon 1.0.3 he3ba5ed_0 conda-forge
libxml2 2.9.10 h72842e0_3 conda-forge
I tried conda update --all as said Here . I also tried all similar question. But, still no solution.
Reference 1
Reference 2
There are five major modules that are included with the GDAL Python bindings.:
from osgeo import gdal
from osgeo import ogr
from osgeo import osr
from osgeo import gdal_array
from osgeo import gdalconst
Additionally, there are five compatibility modules that are included but provide notices to state that they are deprecated and will be going away. If you are using GDAL 1.7 bindings, you should update your imports to utilize the usage above, but the following will work until at least GDAL 2.1.
import gdal
import ogr
import osr
import gdalnumeric
import gdalconst
If you have previous code that imported the global module and still need to support the old import, a simple try…except import can silence the deprecation warning and keep things named essentially the same as before:
try:
from osgeo import gdal
except ImportError:
import gdal
Related
I'm working on applying a clustering algorithm (sklearn.AgglomerativeClustering) on a dataset. I've tried running this same block of code in Spyder IDE and VScode, each time the cell runs for about 35-45s then returns a message that the kernel crashed unexpectedly and and new kernel is created. Im using Python 3.10 with Anaconda package manager.
Spyder had no information about the kernel crash but in VScode I was pointed to this github post Kernel-crashes. I figured somehow my numpy installation was affecting kernel performance or execution. I created a new virtual env using conda; re-installed numpy, pandas, sci-kit learn, etc. Same kernel crash occurring with the same message.
I am using a new Macbook with the m1 chip. Unsure if that's has an influence or is a hint at how to solve this issue. Did see the sci-kit learn installation docs had separate section sci-kit learn install docs about installing with m1 chip but to be honest wasn't exactly sure how to interpret and use that info.
this is the code Im trying to run that causes the kernel to crash. the pca_features is the numpy array return after running .fit_transform method on my raw data. PCA executed fine for what it's worth and showed no issues. Tried using subset of rows, first 5000, to see if that helped cell run at all but still no luck.
Error Message:
The Kernel crashed while executing code in the the current cell or a previous cell. Please review the code in the cell(s) to identify a possible cause of the failure. Click here for more info. View Jupyter log for further details.
Canceled future for execute_request message before replies were done
Im open to other clustering algorithms but fact that I can't get this to execute makes me less confident any others (e.g. KMeans) would successfully run. But I could be wrong.
My goal is get a unsupervised clustering algorithm fitted to the data so I can get labels for each observation and compare it against primary components from PCA.
```
aggclus = AgglomerativeClustering(n_clusters = 4, affinity="euclidean",linkage="ward")
subset_pcafeatures = pca_features[:5000,5]
cluster_labels = aggclus.fit_predict(pca_features)
```
Any tips/advice/help/assistance would be much appreciated. Thank you
Updated 29-01-23
output from $ conda list
# packages in environment at ../anaconda3/envs/work-env:
#
# Name Version Build Channel
appnope 0.1.3 pyhd8ed1ab_0 conda-forge
asttokens 2.2.1 pyhd8ed1ab_0 conda-forge
backcall 0.2.0 pyh9f0ad1d_0 conda-forge
backports 1.0 pyhd8ed1ab_3 conda-forge
backports.functools_lru_cache 1.6.4 pyhd8ed1ab_0 conda-forge
blas 2.116 openblas conda-forge
blas-devel 3.9.0 16_osxarm64_openblas conda-forge
bottleneck 1.3.5 py310h96f19d2_0
brotli 1.0.9 h1a28f6b_7
brotli-bin 1.0.9 h1a28f6b_7
bzip2 1.0.8 h620ffc9_4
ca-certificates 2023.01.10 hca03da5_0
certifi 2022.12.7 py310hca03da5_0
comm 0.1.2 pyhd8ed1ab_0 conda-forge
contourpy 1.0.5 py310h525c30c_0
cycler 0.11.0 pyhd3eb1b0_0
debugpy 1.5.1 py310hc377ac9_0
decorator 5.1.1 pyhd8ed1ab_0 conda-forge
entrypoints 0.4 pyhd8ed1ab_0 conda-forge
executing 1.2.0 pyhd8ed1ab_0 conda-forge
fftw 3.3.9 h1a28f6b_1
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.12.1 h1192e45_0
giflib 5.2.1 h80987f9_1
ipykernel 6.20.2 pyh736e0ef_0 conda-forge
ipython 8.8.0 pyhd1c38e8_0 conda-forge
jedi 0.18.2 pyhd8ed1ab_0 conda-forge
joblib 1.1.1 py310hca03da5_0
jpeg 9e h1a28f6b_0
jupyter_client 7.4.9 pyhd8ed1ab_0 conda-forge
jupyter_core 5.1.1 py310hca03da5_0
kiwisolver 1.4.4 py310h313beb8_0
lcms2 2.12 hba8e193_0
lerc 3.0 hc377ac9_0
libblas 3.9.0 16_osxarm64_openblas conda-forge
libbrotlicommon 1.0.9 h1a28f6b_7
libbrotlidec 1.0.9 h1a28f6b_7
libbrotlienc 1.0.9 h1a28f6b_7
libcblas 3.9.0 16_osxarm64_openblas conda-forge
libcxx 14.0.6 h848a8c0_0
libdeflate 1.8 h1a28f6b_5
libffi 3.4.2 hca03da5_6
libgfortran 5.0.0 11_3_0_hca03da5_28
libgfortran5 11.3.0 h009349e_28
liblapack 3.9.0 16_osxarm64_openblas conda-forge
liblapacke 3.9.0 16_osxarm64_openblas conda-forge
libopenblas 0.3.21 openmp_hc731615_3 conda-forge
libpng 1.6.37 hb8d0fd4_0
libsodium 1.0.18 h27ca646_1 conda-forge
libtiff 4.5.0 h2fd578a_0
libwebp 1.2.4 h68602c7_0
libwebp-base 1.2.4 h1a28f6b_0
llvm-openmp 14.0.6 hc6e5704_0
lz4-c 1.9.4 h313beb8_0
matplotlib 3.6.2 py310hca03da5_0
matplotlib-base 3.6.2 py310h8bbb115_0
matplotlib-inline 0.1.6 pyhd8ed1ab_0 conda-forge
missingno 0.4.2 pyhd3eb1b0_1
munkres 1.1.4 py_0
ncurses 6.4 h313beb8_0
nest-asyncio 1.5.6 pyhd8ed1ab_0 conda-forge
numexpr 2.8.4 py310hecc3335_0
numpy 1.23.5 py310hb93e574_0
numpy-base 1.23.5 py310haf87e8b_0
openblas 0.3.21 openmp_hf78f355_3 conda-forge
openssl 1.1.1s h1a28f6b_0
packaging 23.0 pyhd8ed1ab_0 conda-forge
pandas 1.5.2 py310h46d7db6_0
parso 0.8.3 pyhd8ed1ab_0 conda-forge
pexpect 4.8.0 pyh1a96a4e_2 conda-forge
pickleshare 0.7.5 py_1003 conda-forge
pillow 9.3.0 py310hf4a492f_1
pip 22.3.1 py310hca03da5_0
platformdirs 2.6.2 pyhd8ed1ab_0 conda-forge
prompt-toolkit 3.0.36 pyha770c72_0 conda-forge
psutil 5.9.0 py310h1a28f6b_0
ptyprocess 0.7.0 pyhd3deb0d_0 conda-forge
pure_eval 0.2.2 pyhd8ed1ab_0 conda-forge
pygments 2.14.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 py310hca03da5_0
python 3.10.9 hc0d8a6c_0
python-dateutil 2.8.2 pyhd8ed1ab_0 conda-forge
pytz 2022.7 py310hca03da5_0
pyzmq 23.2.0 py310hc377ac9_0
readline 8.2 h1a28f6b_0
scikit-learn 1.2.0 py310h313beb8_0
scipy 1.9.3 py310h20cbe94_0
seaborn 0.12.2 py310hca03da5_0
setuptools 65.6.3 py310hca03da5_0
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.40.1 h7a7dc30_0
stack_data 0.6.2 pyhd8ed1ab_0 conda-forge
threadpoolctl 2.2.0 pyh0d69192_0
tk 8.6.12 hb8d0fd4_0
tornado 6.2 py310h1a28f6b_0
traitlets 5.8.1 pyhd8ed1ab_0 conda-forge
typing-extensions 4.4.0 hd8ed1ab_0 conda-forge
typing_extensions 4.4.0 pyha770c72_0 conda-forge
tzdata 2022g h04d1e81_0
wcwidth 0.2.6 pyhd8ed1ab_0 conda-forge
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.10 h80987f9_1
zeromq 4.3.4 hbdafb3b_1 conda-forge
zlib 1.2.13 h5a0b063_0
zstd 1.5.2 h8574219_0
I did conda list but I can't see what I am looking for. Are conda packages installed with conda developed . not shown?
# packages in environment at /Users/pinocchio/anaconda3/envs/myenv:
#
# Name Version Build Channel
absl-py 0.9.0 py37_0
appnope 0.1.0 py37hc8dfbb8_1001 conda-forge
asn1crypto 1.3.0 py37_0
astroid 2.3.3 py37_0
attrs 19.3.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
beautifulsoup4 4.8.2 py37_0
blas 1.0 mkl
bleach 3.1.4 pyh9f0ad1d_0 conda-forge
bzip2 1.0.8 h1de35cc_0
c-ares 1.15.0 h1de35cc_1001
ca-certificates 2020.1.1 0
cairo 1.14.12 hc4e6be7_4
certifi 2020.4.5.1 py37_0
cffi 1.14.0 py37hb5b8e2f_0
chardet 3.0.4 py37_1003
conda 4.8.3 py37_0
conda-build 3.18.11 py37_0
conda-package-handling 1.6.0 py37h1de35cc_0
cryptography 2.8 py37ha12b0ac_0
cycler 0.10.0 py37_0
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
entrypoints 0.3 py37hc8dfbb8_1001 conda-forge
expat 2.2.6 h0a44026_0
filelock 3.0.12 py_0
fontconfig 2.13.0 h5d5b041_1
freetype 2.9.1 hb4e5f40_0
fribidi 1.0.5 h1de35cc_0
gettext 0.19.8.1 h15daf44_3
glib 2.63.1 hd977a24_0
glob2 0.7 py_0
graphite2 1.3.13 h2098e52_0
graphviz 2.40.1 hefbbd9a_2
grpcio 1.16.1 py37h044775b_1
harfbuzz 1.8.8 hb8d4a28_0
icu 58.2 h4b95b61_1
idna 2.9 py_1
importlib-metadata 1.6.0 py37hc8dfbb8_0 conda-forge
importlib_metadata 1.6.0 0 conda-forge
intel-openmp 2019.4 233
ipykernel 5.2.0 py37h43977f1_1 conda-forge
ipython 7.13.0 py37hc8dfbb8_2 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isort 4.3.21 py37_0
jedi 0.16.0 py37hc8dfbb8_1 conda-forge
jinja2 2.11.1 py_0
jpeg 9b he5867d9_2
json5 0.9.0 py_0 conda-forge
jsonschema 3.2.0 py37hc8dfbb8_1 conda-forge
jupyter_client 6.1.2 py_0 conda-forge
jupyter_core 4.6.3 py37hc8dfbb8_1 conda-forge
jupyterlab 2.0.1 py_0 conda-forge
jupyterlab_server 1.0.7 py_0 conda-forge
kiwisolver 1.1.0 py37h0a44026_0
lazy-object-proxy 1.4.3 py37h1de35cc_0
libarchive 3.3.3 h786848e_5
libcxx 4.0.1 hcfea43d_1
libcxxabi 4.0.1 hcfea43d_1
libedit 3.1.20181209 hb402a30_0
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
liblief 0.9.0 h2a1bed3_2
libpng 1.6.37 ha441bb4_0
libprotobuf 3.11.4 hd9629dc_0
libsodium 1.0.17 h01d97ff_0 conda-forge
libtiff 4.1.0 hcb84e12_0
libxml2 2.9.9 hf6e021a_1
lz4-c 1.8.1.2 h1de35cc_0
lzo 2.10 h362108e_2
markdown 3.1.1 py37_0
markupsafe 1.1.1 py37h1de35cc_0
matplotlib 3.1.3 py37_0
matplotlib-base 3.1.3 py37h9aa3819_0
mccabe 0.6.1 py37_1
mistune 0.8.4 py37h0b31af3_1000 conda-forge
mkl 2019.4 233
mkl-service 2.3.0 py37hfbe908c_0
mkl_fft 1.0.15 py37h5e564d8_0
mkl_random 1.1.0 py37ha771720_0
nbconvert 5.6.1 py37_0 conda-forge
nbformat 5.0.4 py_0 conda-forge
ncurses 6.2 h0a44026_0
ninja 1.9.0 py37h04f5b5a_0
notebook 6.0.3 py37_0 conda-forge
numpy 1.18.1 py37h7241aed_0
numpy-base 1.18.1 py37h6575580_1
olefile 0.46 py37_0
openssl 1.1.1g h1de35cc_0
pandoc 2.9.2 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
pango 1.42.4 h060686c_0
parso 0.6.2 py_0 conda-forge
pcre 8.43 h0a44026_0
pexpect 4.8.0 py37hc8dfbb8_1 conda-forge
pickleshare 0.7.5 py37hc8dfbb8_1001 conda-forge
pillow 6.2.1 py37hb68e598_0
pip 20.0.2 py37_1
pixman 0.38.0 h1de35cc_0
pkginfo 1.5.0.1 py37_0
prometheus_client 0.7.1 py_0 conda-forge
prompt-toolkit 3.0.5 py_0 conda-forge
protobuf 3.11.4 py37h0a44026_0
psutil 5.7.0 py37h1de35cc_0
ptyprocess 0.6.0 py_1001 conda-forge
py-lief 0.9.0 py37h1413db1_2
pycosat 0.6.3 py37h1de35cc_0
pycparser 2.20 py_0
pygments 2.6.1 py_0 conda-forge
pylint 2.4.4 py37_0
pyopenssl 19.1.0 py37_0
pyparsing 2.4.6 py_0
pyrsistent 0.16.0 py37h9bfed18_0 conda-forge
pysocks 1.7.1 py37_0
python 3.7.7 hc70fcce_0_cpython
python-dateutil 2.8.1 py_0 conda-forge
python-graphviz 0.13.2 pypi_0 pypi
python-libarchive-c 2.8 py37_13
python_abi 3.7 1_cp37m conda-forge
pytorch 1.4.0 py3.7_0 pytorch
pytz 2019.3 py_0
pyyaml 5.3.1 py37h1de35cc_0
pyzmq 18.1.1 py37h0a44026_0
readline 8.0 h1de35cc_0
requests 2.23.0 py37_0
ripgrep 11.0.2 he32d670_0
ruamel_yaml 0.15.87 py37h1de35cc_0
send2trash 1.5.0 py_0 conda-forge
setuptools 46.1.1 py37_0
six 1.14.0 py37_0
soupsieve 2.0 py_0
sqlite 3.31.1 ha441bb4_0
tensorboard 2.0.0 pyhb38c66f_1
terminado 0.8.3 py37hc8dfbb8_1 conda-forge
testpath 0.4.4 py_0 conda-forge
tk 8.6.8 ha441bb4_0
torchvision 0.2.1 py_2 soumith
tornado 6.0.4 py37h9bfed18_1 conda-forge
tqdm 4.43.0 py_0
traitlets 4.3.3 py37hc8dfbb8_1 conda-forge
urllib3 1.25.8 py37_0
wcwidth 0.1.9 pyh9f0ad1d_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
werkzeug 1.0.0 py_0
wheel 0.34.2 py37_0
wrapt 1.12.1 py37h1de35cc_1
xz 5.2.4 h1de35cc_4
yaml 0.1.7 hc338f04_2
zeromq 4.3.2 h6de7cb9_2 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h1de35cc_3
zstd 1.3.7 h5bba6e5_0
utils or something like that should be there but its not.
Update:
I've tried what this suggests and it does not work. I conda develop . my package and then when I run the python interpreter and try to import it the import fails. Why?
>>> import my_pkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_pkg'
I also tried to install it with pip pip install -e and it doesn't find my package either after I go to the interpreter and try to import it. Why is that?
If you look carefully after running conda develop . (where the setup.py file is) you will notice that the command outputs the following:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop .
added /Users/brandBrandoParetoopareto/ultimate-utils/uutils
completed operation for: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
if you then check the sys.path python has then you can that new path has been added:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
/Users/brandBrandoParetoopareto/ultimate-utils/uutils
you can confirm this by removing it and printing the contents of sys.path:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop -u .
uninstalled: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
check sys path again:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
now it's gone!
Also, note that conda or python (not sure which) also adds the current path always. This is important to note because when I did import tests outside of the repo I was working it wasn't working but it was inside the repo which seemed very mysterious which I can only attribute to that.
Addendum
to check python sys.path from terminal run that command as a string with python cmd:
python -c "import sys; print(sys.path)"
even better:
python -c "import sys; [print(p) for p in sys.path]"
Original answer: https://stackoverflow.com/a/59903590/1601580
I want to install rpy into my conda installation, but I get the UnsatisfiableError
$ conda install rpy2
Collecting package metadata (current_repodata.json): done
Solving environment: failed
Collecting package metadata (repodata.json): done
Solving environment: failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
- blas
- conda-forge/linux-64::statsmodels==0.10.1=py37hc1659b7_1 -> numpy[version='>=1.14.6,<2.0a0'] -> blas==1.0=mkl
- conda-forge/noarch::descartes==1.1.0=py_3 -> matplotlib -> numpy -> blas==1.0=mkl
- conda-forge/noarch::mizani==0.6.0=py_0 -> matplotlib[version='>=3.1.1'] -> numpy -> blas==1.0=mkl
- conda-forge/noarch::patsy==0.5.1=py_0 -> numpy[version='>=1.4.0'] -> blas==1.0=mkl
- conda-forge/noarch::plotnine==0.6.0=py_0 -> descartes[version='>=1.1.0'] -> matplotlib -> numpy -> blas==1.0=mkl
- mkl_fft -> blas==1.0=mkl
- mkl_random -> numpy[version='>=1.14.6,<2.0a0'] -> blas==1.0=mkl
- numpy -> blas==1.0=mkl
- numpy-base -> blas==1.0=mkl
- pkgs/main/linux-64::matplotlib==3.1.1=py37h5429711_0 -> numpy -> blas==1.0=mkl
- pkgs/main/linux-64::pandas==0.25.2=py37he6710b0_0 -> numpy[version='>=1.14.6,<2.0a0'] -> blas==1.0=mkl
- scipy -> blas==1.0=mkl
I have tried other conda installations (like using conda-forge) but I still get UnsatisfiableError - though with different details.
Funny enough, installing it with pip worked...
I have the conda installation
# Name Version Build Channel
_libgcc_mutex 0.1 main
blas 1.0 mkl
ca-certificates 2019.9.11 hecc5488_0 conda-forge
certifi 2019.9.11 py37_0 conda-forge
cycler 0.10.0 py_2 conda-forge
dbus 1.13.6 he372182_0 conda-forge
descartes 1.1.0 py_3 conda-forge
expat 2.2.5 he1b5a44_1004 conda-forge
fontconfig 2.13.1 he4413a7_1000 conda-forge
freetype 2.10.0 he983fc9_1 conda-forge
gettext 0.19.8.1 hc5be6a0_1002 conda-forge
glib 2.58.3 h6f030ca_1002 conda-forge
gst-plugins-base 1.14.5 h0935bb2_0 conda-forge
gstreamer 1.14.5 h36ae1b5_0 conda-forge
icu 58.2 hf484d3e_1000 conda-forge
intel-openmp 2019.4 243
jpeg 9c h14c3975_1001 conda-forge
kiwisolver 1.1.0 py37hc9558a2_0 conda-forge
libedit 3.1.20181209 hc058e9b_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0
libiconv 1.15 h516909a_1005 conda-forge
libpng 1.6.37 hed695b0_0 conda-forge
libstdcxx-ng 9.1.0 hdf63c60_0
libuuid 2.32.1 h14c3975_1000 conda-forge
libxcb 1.13 h14c3975_1002 conda-forge
libxml2 2.9.9 h13577e0_2 conda-forge
matplotlib 3.1.1 py37h5429711_0
mizani 0.6.0 py_0 conda-forge
mkl 2019.4 243
mkl-service 2.3.0 py37he904b0f_0
mkl_fft 1.0.14 py37ha843d7b_0
mkl_random 1.1.0 py37hd6b4f25_0
ncurses 6.1 he6710b0_1
numpy 1.17.2 py37haad9e8e_0
numpy-base 1.17.2 py37hde5b4d6_0
openssl 1.1.1c h516909a_0 conda-forge
palettable 3.3.0 py_0 conda-forge
pandas 0.25.2 py37he6710b0_0
patsy 0.5.1 py_0 conda-forge
pcre 8.43 he1b5a44_0 conda-forge
pip 19.3.1 py37_0
plotnine 0.6.0 py_0 conda-forge
pthread-stubs 0.4 h14c3975_1001 conda-forge
pyparsing 2.4.2 py_0 conda-forge
pyqt 5.9.2 py37hcca6a23_4 conda-forge
python 3.7.4 h265db76_1
python-dateutil 2.8.0 py37_0
pytz 2019.3 py_0
qt 5.9.7 h52cfd70_2 conda-forge
readline 7.0 h7b6447c_5
scipy 1.3.1 py37h7c811a0_0
setuptools 41.4.0 py37_0
sip 4.19.8 py37hf484d3e_0
six 1.12.0 py37_0
sqlite 3.30.1 h7b6447c_0
statsmodels 0.10.1 py37hc1659b7_1 conda-forge
tk 8.6.8 hbc83047_0
tornado 6.0.3 py37h516909a_0 conda-forge
wheel 0.33.6 py37_0
xorg-libxau 1.0.9 h14c3975_0 conda-forge
xorg-libxdmcp 1.1.3 h516909a_0 conda-forge
xz 5.2.4 h14c3975_4
zlib 1.2.11 h7b6447c_3
Somehow when I do the install it installs torchvision but not torch. Command I am running as dictated from the main website:
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
then I do conda list but look:
$ conda list
# packages in environment at /home/ubuntu/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
alabaster 0.7.10 py36h306e16b_0
anaconda-client 1.6.14 py36_0
anaconda-project 0.8.2 py36h44fb852_0
argparse 1.4.0 <pip>
asn1crypto 0.24.0 py36_0
astroid 1.6.3 py36_0
astropy 3.0.2 py36h3010b51_1
attrs 18.1.0 py36_0
autovizwidget 0.12.7 <pip>
babel 2.5.3 py36_0
backcall 0.1.0 py36_0
backports 1.0 py36hfa02d7e_1
backports.shutil_get_terminal_size 1.0.0 py36hfea85ff_2
bcrypt 3.1.6 <pip>
beautifulsoup4 4.6.0 py36h49b8c8c_1
bitarray 0.8.1 py36h14c3975_1
bkcharts 0.2 py36h735825a_0
blas 1.0 mkl
blaze 0.11.3 py36h4e06776_0
bleach 2.1.3 py36_0
blosc 1.14.3 hdbcaa40_0
bokeh 1.0.4 py36_0
boto 2.48.0 py36h6e4cd66_1
boto3 1.9.146 <pip>
boto3 1.9.134 py_0
botocore 1.12.146 <pip>
botocore 1.12.134 py_0
bottleneck 1.2.1 py36haac1ea0_0
bzip2 1.0.6 h14c3975_5
ca-certificates 2019.1.23 0
cached-property 1.5.1 <pip>
cairo 1.14.12 h8948797_3
certifi 2019.3.9 py36_0
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
click 6.7 py36h5253387_0
cloudpickle 0.5.3 py36_0
clyent 1.2.2 py36h7e57e65_1
colorama 0.3.9 py36h489cec4_0
contextlib2 0.5.5 py36h6c84a62_0
cryptography 2.3.1 py36hc365091_0
cudatoolkit 10.0.130 0
curl 7.60.0 h84994c4_0
cycler 0.10.0 py36h93f1223_0
cymem 2.0.2 py36hfd86e86_0
cython 0.28.2 py36h14c3975_0
cytoolz 0.9.0.1 py36h14c3975_0
dask 0.17.5 py36_0
dask-core 0.17.5 py36_0
dataclasses 0.6 py_0 fastai
datashape 0.5.4 py36h3ad6b5c_0
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py36_0
defusedxml 0.6.0 py_0
dill 0.2.9 py36_0
distributed 1.21.8 py36_0
docker 3.7.2 <pip>
docker-compose 1.24.0 <pip>
docker-pycreds 0.4.0 <pip>
dockerpty 0.4.1 <pip>
docopt 0.6.2 <pip>
docutils 0.14 py36hb0f60f5_0
entrypoints 0.2.3 py36h1aec115_2
environment-kernels 1.1.1 <pip>
et_xmlfile 1.0.1 py36hd6bccc3_0
expat 2.2.5 he0dffb1_0
fastai 1.0.52 1 fastai
fastcache 1.0.2 py36h14c3975_2
fastprogress 0.1.21 py_0 fastai
filelock 3.0.4 py36_0
flask 1.0.2 py36_1
flask-cors 3.0.4 py36_0
fontconfig 2.13.0 h9420a91_0
freetype 2.9.1 h8a8886c_1
fribidi 1.0.5 h7b6447c_0
get_terminal_size 1.0.0 haa9412d_0
gevent 1.3.0 py36h14c3975_0
glib 2.56.1 h000015b_0
glob2 0.6 py36he249c77_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py36hc8893dd_2
graphite2 1.3.11 h16798f4_2
graphviz 2.40.1 h21bd128_2
greenlet 0.4.13 py36h14c3975_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.8.0 py36h989c5e5_3
harfbuzz 1.8.4 hec2c2bc_0
hdf5 1.10.2 hba1933b_1
hdijupyterutils 0.12.7 <pip>
heapdict 1.0.0 py36_2
html5lib 1.0.1 py36h2f9c1c0_0
icu 58.2 h9c2bf20_1
idna 2.6 py36h82fb2a8_1
imageio 2.3.0 py36_0
imagesize 1.0.0 py36_0
intel-openmp 2018.0.0 8
ipykernel 4.8.2 py36_0
ipyparallel 6.2.2 <pip>
ipython 6.4.0 py36_0
ipython_genutils 0.2.0 py36hb52b0d5_0
ipywidgets 7.2.1 py36_0
ipywidgets 7.4.0 <pip>
isort 4.3.4 py36_0
itsdangerous 0.24 py36h93cc618_1
jbig 2.1 hdba287a_0
jdcal 1.4 py36_0
jedi 0.12.0 py36_1
jinja2 2.10 py36ha16c418_0
jmespath 0.9.4 py_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py36h006f8b5_0
jupyter 1.0.0 py36_4
jupyter_client 5.2.3 py36_0
jupyter_console 5.2.0 py36he59e554_1
jupyter_core 4.4.0 py36h7c827e3_0
jupyterlab 0.32.1 py36_0
jupyterlab_launcher 0.10.5 py36_0
kiwisolver 1.0.1 py36h764f252_0
krb5 1.14.2 hcdc1b81_6
lazy-object-proxy 1.3.1 py36h10fcdad_0
libcurl 7.60.0 h1ad7b7a_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran 3.0.0 1 conda-forge
libgfortran-ng 7.2.0 hdf63c60_3
libpng 1.6.37 hbc83047_0
libprotobuf 3.5.2 hd28b015_1 conda-forge
libsodium 1.0.16 h1bed415_0
libssh2 1.8.0 h9cfc8f7_4
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_1
libtool 2.4.6 h544aabb_3
libuuid 1.0.3 h1bed415_2
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
libxslt 1.1.32 h1312cb7_0
llvmlite 0.23.1 py36hdbcaa40_0
locket 0.2.0 py36h787c0ad_1
lxml 4.2.1 py36h23eabaa_0
lzo 2.10 h49e0be7_2
markupsafe 1.0 py36hd9260cd_1
matplotlib 2.2.2 <pip>
matplotlib 3.0.3 py36h5429711_0
mccabe 0.6.1 py36h5ad9710_1
mistune 0.8.3 py36h14c3975_1
mkl 2018.0.3 1
mkl-service 1.1.2 py36h17a0993_4
mkl_fft 1.0.6 py36h7dd41cf_0
mkl_random 1.0.1 py36h629b387_0
mock 3.0.5 <pip>
more-itertools 4.1.0 py36_0
mpc 1.0.3 hec55b23_5
mpfr 3.1.5 h11a74b3_2
mpi 1.0 openmpi conda-forge
mpmath 1.0.0 py36hfeacd6b_2
msgpack 0.6.0 <pip>
msgpack-numpy 0.4.3.2 py36_0
msgpack-python 0.5.6 py36h6bb024c_0
multipledispatch 0.5.0 py36_0
murmurhash 1.0.2 py36he6710b0_0
nb_conda 2.2.1 py36_2 conda-forge
nb_conda_kernels 2.2.1 py36_0 conda-forge
nbconvert 5.4.1 py36_3
nbformat 4.4.0 py36h31c9010_0
ncurses 6.1 hf484d3e_0
networkx 2.1 py36_0
ninja 1.8.2 py36h6bb024c_1
nltk 3.3.0 py36_0
nose 1.3.7 py36hcdf7029_2
notebook 5.5.0 py36_0
numba 0.38.0 py36h637b7d7_0
numexpr 2.6.5 py36h7bf3b9c_0
numpy 1.15.4 py36h1d66e8a_0
numpy 1.15.4 <pip>
numpy-base 1.15.4 py36h81de0dd_0
numpydoc 0.8.0 py36_0
nvidia-ml-py3 7.352.0 py_0 fastai
odo 0.5.1 py36h90ed295_0
olefile 0.45.1 py36_0
onnx 1.4.1 <pip>
openmpi 3.1.0 h26a2512_3 conda-forge
openpyxl 2.5.3 py36_0
openssl 1.0.2r h7b6447c_0
packaging 17.1 py36_0
pandas 0.24.2 <pip>
pandas 0.23.0 py36h637b7d7_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py36ha6701b7_1
pango 1.42.3 h8589676_0
paramiko 2.4.2 <pip>
parso 0.2.0 py36_0
partd 0.3.8 py36h36fd896_0
patchelf 0.9 hf79760b_2
path.py 11.0.1 py36_0
pathlib2 2.3.2 py36_0
patsy 0.5.0 py36_0
pcre 8.42 h439df22_0
pep8 1.7.1 py36_0
pexpect 4.5.0 py36_0
pickleshare 0.7.4 py36h63277f8_0
pillow 5.2.0 py36heded4f4_0
pip 10.0.1 py36_0
pixman 0.34.0 hceecf20_3
pkginfo 1.4.2 py36_1
plac 0.9.6 py36_0
plotly 2.7.0 <pip>
pluggy 0.6.0 py36hb689045_0
ply 3.11 py36_0
preshed 2.0.1 py36he6710b0_0
prompt_toolkit 1.0.15 py36h17d85b1_0
protobuf 3.5.2 py36hd28b015_0 conda-forge
protobuf3-to-dict 0.1.5 <pip>
psutil 5.4.5 py36h14c3975_0
psycopg2 2.7.5 <pip>
ptyprocess 0.5.2 py36h69acd42_0
py 1.5.3 py36_0
py4j 0.10.7 <pip>
pyasn1 0.4.5 <pip>
pycodestyle 2.4.0 py36_0
pycosat 0.6.3 py36h0a5515d_0
pycparser 2.18 py36hf9f622e_1
pycrypto 2.6.1 py36h14c3975_8
pycurl 7.43.0.1 py36hb7f436b_0
pyflakes 1.6.0 py36h7bd6a15_0
pygal 2.4.0 <pip>
pygments 2.2.0 py36h0d3125c_0
pykerberos 1.2.1 py36h14c3975_0
pylint 1.8.4 py36_0
PyNaCl 1.3.0 <pip>
pyodbc 4.0.23 py36hf484d3e_0
pyopenssl 18.0.0 py36_0
pyparsing 2.2.0 py36hee85983_1
pyqt 5.9.2 py36h751905a_0
pysocks 1.6.8 py36_0
pyspark 2.3.2 <pip>
pytables 3.4.3 py36h02b9ad4_2
pytest 3.5.1 py36_0
pytest-arraydiff 0.2 py36_0
pytest-astropy 0.3.0 py36_0
pytest-doctestplus 0.1.3 py36_0
pytest-openfiles 0.3.0 py36_0
pytest-remotedata 0.2.1 py36_0
python 3.6.5 hc3d631a_2
python-dateutil 2.7.3 py36_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
pytz 2018.4 py36_0
pywavelets 0.5.2 py36he602eb0_0
pyyaml 3.12 py36hafb9ca4_1
pyzmq 17.0.0 py36h14c3975_0
qt 5.9.6 h52aff34_0
qtawesome 0.4.4 py36h609ed8c_0
qtconsole 4.3.1 py36h8f73b5b_0
qtpy 1.4.1 py36_0
readline 7.0 ha6073c6_4
regex 2018.01.10 py36h14c3975_1000 fastai
requests 2.20.0 py36_1000 conda-forge
requests-kerberos 0.12.0 <pip>
rope 0.10.7 py36h147e2ec_0
ruamel_yaml 0.15.35 py36h14c3975_1
s3fs 0.1.5 py36_0
s3transfer 0.2.0 <pip>
s3transfer 0.2.0 py36_0
sagemaker 1.20.1 <pip>
sagemaker-pyspark 1.2.4 <pip>
scikit-image 0.13.1 py36h14c3975_1
scikit-learn 0.19.1 py36h7aa7ec6_0
scikit-learn 0.20.3 <pip>
scipy 1.1.0 py36hfc37229_0
seaborn 0.8.1 py36hfad7ec4_0
send2trash 1.5.0 py36_0
setuptools 39.1.0 py36_0
simplegeneric 0.8.1 py36_2
singledispatch 3.4.0.3 py36h7a266c3_0
sip 4.19.8 py36hf484d3e_0
six 1.11.0 py36h372c433_1
snappy 1.1.7 hbae5bb6_3
snowballstemmer 1.2.1 py36h6febd40_0
sortedcollections 0.6.1 py36_0
sortedcontainers 1.5.10 py36_0
spacy 2.0.18 py36hf484d3e_1000 fastai
sparkmagic 0.12.5 <pip>
sphinx 1.7.4 py36_0
sphinxcontrib 1.0 py36h6d0f590_1
sphinxcontrib-websupport 1.0.1 py36hb5cb234_1
spyder 3.2.8 py36_0
SQLAlchemy 1.2.11 <pip>
sqlalchemy 1.2.7 py36h6b74fdf_0
sqlite 3.23.1 he433501_0
statsmodels 0.9.0 py36h3010b51_0
sympy 1.1.1 py36hc6d1c1c_0
tblib 1.3.2 py36h34cf8b6_0
terminado 0.8.1 py36_1
testpath 0.3.1 py36h8cadb63_0
texttable 0.9.1 <pip>
thinc 6.12.1 py36h637b7d7_1000 fastai
tk 8.6.8 hbc83047_0
toolz 0.9.0 py36_0
torchvision 0.2.2 py_3 pytorch
tornado 5.0.2 py36_0
tqdm 4.31.1 py36_1
traitlets 4.3.2 py36h674d592_0
typing 3.6.4 py36_0
typing-extensions 3.7.2 <pip>
ujson 1.35 py36h14c3975_0
unicodecsv 0.14.1 py36ha668878_0
unixodbc 2.3.6 h1bed415_0
urllib3 1.23 py36_0
wcwidth 0.1.7 py36hdf4376a_0
webencodings 0.5.1 py36h800622e_1
websocket-client 0.56.0 <pip>
werkzeug 0.14.1 py36_0
wheel 0.31.1 py36_0
widgetsnbextension 3.2.1 py36_0
widgetsnbextension 3.4.2 <pip>
wrapt 1.10.11 py36h28b7045_0
xlrd 1.1.0 py36h1db9f0c_1
xlsxwriter 1.0.4 py36_0
xlwt 1.3.0 py36h7b00a1f_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.2.5 h439df22_0
zict 0.1.3 py36h3a3bf81_0
zlib 1.2.11 ha838bed_2
I was told I do have pytorch installed but my script keeps giving me this error:
$ cat nohup.out
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
does that mean that I need to install it as pytroch and not torch? Is this not weird?
Note I am running this on an AWS instance p3.2xlarge. This keeps happening when I log out and then go back in that my torch package gets missing...?!?! :/
original post: https://discuss.pytorch.org/t/torchvision-installed-but-not-torch/51758
The issue persists even if I open just a python interactive and try to import it:
(pytorch_p36) ubuntu#ip-123-12-21-123:~$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
It also happens running the script directly:
(pytorch_p36) ubuntu#ip-123-12-21-123:~/project/folder$ python high_performing_data_point_models_cifar10.py
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
I can't import torchvision either!
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchvision
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torchvision'
>>>
Your conda list command shows that it was run from the environment called automl:
# packages in environment at /home/ubuntu/anaconda3/envs/automl:
However, when you show the commands that you are trying to run, you are doing so from the (pytorch_p36) environment.
You should run your conda install command while inside this pytorch_p36 environment.
Go to the official website and try installing it as said over there. As I just followed the official documentation and it is working fine.
Here is the step though.
conda install pytorch-cpu torchvision-cpu -c pytorch
You can follow the official documentation.
link
It is really easy to miss smth while working with several machines and several conda/python environments. I think it is a good idea to start from the very beginning. Here is installation process that works fine for me.
0. Connect to your aws-instance using SSH or PuTTY.
1. Create conda env named pytorch_p36 with python 3.6 on your aws machine:
user#aws-instance:~$ conda create -n pytorch_p36 python=3.6
2. Activate it:
user#aws-instance:~$ conda activate pytorch_p36
Or with (for older conda versions):
user#aws-instance:~$ source activate pytorch_p36
Now you should see (pytorch_p36) before your shell prompt:
(pytorch_p36) user#aws-instance:~$
3. Go to PyTorch website and choose appropriate installation command via conda. Run it in your shell:
(pytorch_p36) user#aws-instance:~$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
4. Verify the installation:
(pytorch_p36) user#aws-instance:~$ conda list
# packages in environment at /home/user/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
...
cudatoolkit 10.0.130 0
...
python 3.6.9 h265db76_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
...
torchvision 0.3.0 py36_cu10.0.130_1 pytorch
...
5. Verify the running with nohup:
(pytorch_p36) user#aws-instance:~$ nohup python -c "import torch; import torchvision; print('PyTorch is fine!')" && cat nohup.out
nohup: ignoring input and appending output to 'nohup.out'
PyTorch is fine!
AWS Machine Learning AMI already has pytorch v1.1.0 and torchvision v0.2.2 installed in the predefined pytorch_p36 virtual environment. So you are not required to create new venv and install pytorch each time you log in. All you need is to run :
$ source activate pytorch_p36
(without dollar sign, obviously). Then, if you run pip show torch you will see that it is already there.
I'd like to create a python3.6 env with default anaconda packages. The manual and many online resources say the command is conda create -n py36 python=3.6 anaconda. On other computers that works, but on one particular computer it only installs very few packages. I can't figure out why. I've checked $HOME/anaconda2/env and there's nothing in there.
(ubuntu 16.04, installed from anaconda, not miniconda)
$ conda update anaconda
Solving environment: done
# All requested packages already installed.
$ conda update conda
Solving environment: done
# All requested packages already installed.
$conda create -n py36 python=3.6 anaconda
Solving environment: done
## Package Plan ##
environment location: /home/memo/anaconda2/envs/py36
added / updated specs:
- anaconda
- python=3.6
The following NEW packages will be INSTALLED:
anaconda: custom-py36hbbc8b67_0
ca-certificates: 2018.10.15-ha4d7672_0 conda-forge
certifi: 2018.10.15-py36_1000 conda-forge
libffi: 3.2.1-hfc679d8_5 conda-forge
libgcc-ng: 7.2.0-hdf63c60_3 conda-forge
libstdcxx-ng: 7.2.0-hdf63c60_3 conda-forge
ncurses: 6.1-hfc679d8_1 conda-forge
openssl: 1.0.2p-h470a237_1 conda-forge
pip: 18.1-py36_1000 conda-forge
python: 3.6.6-h5001a0f_3 conda-forge
readline: 7.0-haf1bffa_1 conda-forge
setuptools: 40.5.0-py36_0 conda-forge
sqlite: 3.25.3-hb1c47c0_0 conda-forge
tk: 8.6.8-ha92aebf_0 conda-forge
wheel: 0.32.2-py36_0 conda-forge
xz: 5.2.4-h470a237_1 conda-forge
zlib: 1.2.11-h470a237_3 conda-forge
Proceed ([y]/n)? n
$ conda list
# packages in environment at /home/memo/anaconda2:
#
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py27h08a7f0c_0
absl-py 0.6.1 <pip>
absl-py 0.5.0 <pip>
alabaster 0.7.10 py27he5a193a_0
anaconda custom py27h4a00acb_0
anaconda-client 1.6.9 py27_0
anaconda-navigator 1.8.4 py27_0
anaconda-project 0.8.2 py27h236b58a_0
asn1crypto 0.24.0 py27_0
astor 0.7.1 <pip>
astroid 1.6.1 py27_0
astropy 2.0.3 py27h14c3975_0
attrs 17.4.0 py27_0
audioread 2.1.4 py27_1 conda-forge
babel 2.5.3 py27_0
backports 1.0 py27h63c9359_1
backports.functools_lru_cache 1.4 py27he8db605_1
backports.shutil_get_terminal_size 1.0.0 py27h5bc021e_2
backports.weakref 1.0.post1 <pip>
backports_abc 0.5 py27h7b3c97b_0
beautifulsoup4 4.6.0 py27h3f86ba9_1
bibtexparser 1.0.1 <pip>
bitarray 0.8.1 py27h14c3975_1
bkcharts 0.2 py27h241ae91_0
blas 1.0 mkl
blaze 0.11.3 py27h5f341da_0
bleach 1.5.0 <pip>
bleach 2.1.2 py27_0
blosc 1.14.4 hfc679d8_0 conda-forge
bokeh 0.12.13 py27h5233db4_0
boost 1.59.0 py27_0 menpo
boto 2.48.0 py27h9556ac2_1
bottleneck 1.2.1 py27h21b16a3_0
bzip2 1.0.6 h9a117a8_4
ca-certificates 2018.10.15 ha4d7672_0 conda-forge
cairo 1.14.6 4 conda-forge
cdecimal 2.3 py27h14c3975_3
certifi 2018.10.15 py27_1000 conda-forge
cffi 1.11.4 py27h9745a5d_0
cgal-bindings 1.2 <pip>
chardet 3.0.4 py27hfa10054_1
click 6.7 py27h4225b90_0
cloudpickle 0.5.2 py27_1
clyent 1.2.2 py27h7276e6c_1
colorama 0.3.9 py27h5cde069_0
conda 4.5.11 py27_1000 conda-forge
conda-build 3.4.1 py27_0
conda-env 2.6.0 h36134e3_1
conda-verify 2.0.0 py27hf052a9d_0
configparser 3.5.0 py27h5117587_0
contextlib2 0.5.5 py27hbf4c468_0
cryptography 2.3.1 py27hdffb7b8_0 conda-forge
cryptography-vectors 2.3.1 py27_1000 conda-forge
curl 7.61.0 h93b3f91_2 conda-forge
cycler 0.10.0 py27hc7354d3_0
cython 0.27.3 py27hc56b35e_0
cytoolz 0.9.0 py27h14c3975_0
dask 0.16.1 py27_0
dask-core 0.16.1 py27_0
datashape 0.5.4 py27hf507385_0
dbus 1.13.0 h3a4f0e9_0 conda-forge
decorator 4.3.0 <pip>
decorator 4.2.1 py27_0
distributed 1.20.2 py27_0
dlib 19.16.0 <pip>
dlib 19.4 py27_0 menpo
docopt 0.6.2 <pip>
docutils 0.14 py27hae222c1_0
dominate 2.3.4 <pip>
dxfwrite 1.2.1 <pip>
entrypoints 0.2.3 py27h502b47d_2
enum34 1.1.6 py27h99a27e9_1
enum34 1.1.6 <pip>
et_xmlfile 1.0.1 py27h75840f5_0
expat 2.2.5 he0dffb1_0
fastcache 1.0.2 py27h14c3975_2
ffmpeg 3.1.3 0 menpo
filelock 2.0.13 py27h61a9c69_0
flask 0.12.2 py27h6d5c1cd_0
flask-cors 3.0.3 py27h1a8a27f_0
flickr-api 0.6.1 <pip>
fontconfig 2.12.1 4 conda-forge
freetype 2.7 1 conda-forge
funcsigs 1.0.2 <pip>
funcsigs 1.0.2 py27h83f16ab_0
functools32 3.2.3.2 py27h4ead58f_1
future 0.17.1 py27_0 anaconda
futures 3.2.0 py27h7b459c0_0
futures 3.2.0 <pip>
gast 0.2.0 <pip>
geomstats 1.5 <pip>
get_terminal_size 1.0.0 haa9412d_0
gettext 0.19.8.1 h5e8e0c9_1 conda-forge
gevent 1.2.2 py27h475ea6a_0
glib 2.51.4 0 conda-forge
glob2 0.6 py27hcea9cbd_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py27h4cf3fa8_2
graphite2 1.3.10 hf63cedd_1
greenlet 0.4.12 py27hac09c53_0
grin 1.2.1 py27_4
grpcio 1.15.0 <pip>
grpcio 1.16.0 <pip>
gst-plugins-base 1.8.0 0 conda-forge
gstreamer 1.8.0 1 conda-forge
h5py 2.8.0 <pip>
h5py 2.8.0 py27hb794570_1 conda-forge
harfbuzz 1.4.3 0 conda-forge
hdf5 1.10.2 hc401514_1 conda-forge
heapdict 1.0.0 py27_2
html5lib 0.9999999 <pip>
html5lib 1.0.1 py27h5233db4_0
httplib2 0.11.3 <pip>
icu 58.2 hfc679d8_0 conda-forge
idna 2.6 py27h5722d68_1
idna 2.7 <pip>
imageio 2.4.1 py_0 conda-forge
imagesize 0.7.1 py27hd17bf80_0
imutils 0.5.1 <pip>
intel-openmp 2018.0.0 hc7b2577_8
ipaddress 1.0.19 py27_0
ipykernel 4.8.0 py27_0
ipython 5.4.1 py27_2
ipython_genutils 0.2.0 py27h89fb69b_0
ipywidgets 7.1.1 py27_0
isort 4.2.15 py27hcfa4749_0
itsdangerous 0.24 py27hb8295c1_1
jbig 2.1 hdba287a_0
jdcal 1.3 py27h2cc5433_0
jedi 0.11.1 py27_0
jinja2 2.10 py27h4114e70_0
joblib 0.12.5 py_0 conda-forge
jpeg 9c h470a237_1 conda-forge
jsonschema 2.6.0 py27h7ed5aa4_0
jupyter 1.0.0 py27_4
jupyter_client 5.2.2 py27_0
jupyter_console 5.2.0 py27hc6bee7e_1
jupyter_core 4.4.0 py27h345911c_0
jupyterlab 0.31.5 py27_0
jupyterlab_launcher 0.10.2 py27_0
Keras 2.2.2 <pip>
Keras-Applications 1.0.6 <pip>
Keras-Preprocessing 1.0.5 <pip>
kiwisolver 1.0.1 py27_1 conda-forge
krb5 1.14.6 0 conda-forge
lazy-object-proxy 1.3.1 py27h682c727_0
libcurl 7.61.1 heec0ca6_0
libedit 3.1.20170329 haf1bffa_1 conda-forge
libffi 3.2.1 hd88cf55_4
libgcc 7.2.0 h69d50b8_2 conda-forge
libgcc-ng 8.2.0 hdf63c60_1 anaconda
libgfortran 3.0.0 1
libgfortran-ng 7.2.0 h9f7466a_2
libiconv 1.15 h470a237_3 conda-forge
libopenblas 0.2.20 h9ac9557_7
libopus 1.2.1 hb9ed12e_0
libpng 1.6.34 hb9fc6fc_0
librosa 0.6.2 py_0 conda-forge
libsodium 1.0.15 hf101ebd_0
libssh2 1.8.0 h5b517e9_2 conda-forge
libstdcxx-ng 8.2.0 hdf63c60_1 anaconda
libtiff 4.0.7 1 conda-forge
libtool 2.4.6 h544aabb_3
libuuid 2.32.1 h470a237_2 conda-forge
libvpx 1.6.1 h888fd40_0
libxcb 1.12 hcd93eb1_4
libxml2 2.9.8 h422b904_5 conda-forge
libxslt 1.1.32 h88dbc4e_2 conda-forge
linecache2 1.0.0 py27_0 conda-forge
llvmlite 0.23.0 py27_1 conda-forge
locket 0.2.0 py27h73929a2_1
lws 1.2 <pip>
lxml 4.1.1 py27hdd00cef_0
lzo 2.10 h49e0be7_2
Markdown 3.0.1 <pip>
markupsafe 1.0 py27h97b2822_1
matplotlib 2.1.0 py27_0 conda-forge
mccabe 0.6.1 py27h0e7c7be_1
mido 1.2.9 <pip>
mistune 0.8.3 py27_0
mkl 2018.0.3 1
mkl-service 1.1.2 py27hb2d42c5_4
mkl_fft 1.0.5 py27_0 conda-forge
mkl_random 1.0.1 py27_0 conda-forge
mock 2.0.0 <pip>
moviepy 0.2.3.5 <pip>
mpc 1.0.3 hec55b23_5
mpfr 3.1.5 h11a74b3_2
mpmath 1.0.0 py27h9669132_2
msgpack-python 0.5.1 py27h6bb024c_0
multipledispatch 0.4.9 py27h9b5f95a_0
navigator-updater 0.2.0 py27_0
nbconvert 5.3.1 py27he041f76_0
nbformat 4.4.0 py27hed7f2b2_0
ncurses 6.1 hfc679d8_1 conda-forge
networkx 2.1 py27_0
ninja 1.8.2 h2d50403_1 conda-forge
nltk 3.2.5 py27hec5f4de_0
nose 1.3.7 py27heec2199_2
notebook 5.4.0 py27_0
numba 0.38.1 py27_0 conda-forge
numexpr 2.6.6 py27_0 conda-forge
numpy 1.15.4 <pip>
numpy 1.15.3 py27h1d66e8a_0
numpy 1.14.5 <pip>
numpy-base 1.15.3 py27h81de0dd_0
numpy-stl 2.7.0 <pip>
numpydoc 0.7.0 py27h9647a75_0
oauth2 1.9.0.post1 <pip>
odo 0.5.1 py27h9170de3_0
olefile 0.45.1 py27_0
openblas 0.2.20 8 conda-forge
opencv-contrib-python 3.4.3.18 <pip>
opencv-python 3.4.3.18 <pip>
openpyxl 2.4.10 py27_0
openssl 1.0.2p h470a237_1 conda-forge
packaging 16.8 py27h5e07c7c_1
pandas 0.23.0 py27h637b7d7_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py27h428e1e5_1
pango 1.40.4 0 conda-forge
parso 0.1.1 py27h718acc2_0
partd 0.3.8 py27h4e55004_0
patchelf 0.9 hf79760b_2
path.py 10.5 py27hefe4bee_0
pathlib2 2.3.0 py27h6e9d198_0
patsy 0.5.0 py27_0
pbr 5.1.0 <pip>
pbr 4.2.0 <pip>
pcre 8.39 1
pep8 1.7.1 py27_0
pexpect 4.3.1 py27_0
pickleshare 0.7.4 py27h09770e1_0
Pillow 5.3.0 <pip>
Pillow 5.2.0 <pip>
pillow 4.3.0 py27_1 conda-forge
pip 9.0.1 py27ha730c48_4
pixman 0.34.0 h470a237_3 conda-forge
pkginfo 1.4.1 py27hee1a9ad_1
pluggy 0.6.0 py27h1f4f128_0
ply 3.10 py27hd6d9ae5_0
plyfile 0.6 <pip>
prompt_toolkit 1.0.15 py27h1b593e1_0
protobuf 3.6.1 <pip>
psutil 5.4.3 py27h14c3975_0
ptyprocess 0.5.2 py27h4ccb14c_0
py 1.5.2 py27h203d672_0
PyAudio 0.2.11 <pip>
pycairo 1.10.0 py27_0
pycodestyle 2.3.1 py27h904819d_0
pycosat 0.6.3 py27ha4109ae_0
pycparser 2.18 py27hefa08c5_1
pycrypto 2.6.1 py27h14c3975_7
pycurl 7.19.0 py27_0 anaconda
pyflakes 1.6.0 py27h904a57d_0
pygame 1.9.4 <pip>
pyglet 1.3.2 <pip>
pygments 2.2.0 py27h4a8b6f5_0
pylint 1.8.2 py27_0
pymesh2 0.2.0 <pip>
pyodbc 4.0.22 py27hf484d3e_0
PyOpenGL 3.1.0 <pip>
pyopenssl 17.5.0 py27hcee3be0_0
pyOSC 0.3.5b5294 <pip>
pyparsing 2.2.0 py27hf1513f8_1
pyqt 5.6.0 py27h8210e8a_7 conda-forge
pyqtgraph 0.10.0 py27h28b3542_3 anaconda
pysocks 1.6.7 py27he2db6d2_1
pytables 3.4.4 py27h4f72b40_1 conda-forge
pytest 3.3.2 py27_0
python 2.7.11 0
python-dateutil 2.6.1 py27h4ca5741_1
python-rtmidi 1.1.2 <pip>
python-utils 2.3.0 <pip>
python-xlib 0.23 <pip>
pytorch 0.4.1 py27__9.0.176_7.1.2_2 pytorch
pytz 2017.3 py27h001bace_0
pywavelets 0.5.2 py27hecda097_0
pyxhook 1.0.0 <pip>
PyYAML 3.13 <pip>
pyyaml 3.12 py27h2d70dd7_1
pyzmq 17.1.0 <pip>
pyzmq 16.0.3 py27hc579512_0
qt 5.6.2 3 conda-forge
qtawesome 0.4.4 py27hd7914c3_0
qtconsole 4.3.1 py27hc444b0d_0
qtpy 1.3.1 py27h63d3751_0
readline 6.2 2
requests 2.19.1 <pip>
requests 2.18.4 py27hc5b0589_1
resampy 0.2.0 py27_1 conda-forge
rope 0.10.7 py27hfe459b0_0
ruamel_yaml 0.15.35 py27h14c3975_1
scandir 1.6 py27hf7388dc_0
scikit-image 0.14.0 py27hf484d3e_1 anaconda
scikit-learn 0.20.0 py27h4989274_1
scipy 1.1.0 <pip>
scipy 1.1.0 py27hfa4b5c9_1
seaborn 0.8.1 py27h633ea1e_0
send2trash 1.4.2 py27_0
setuptools 39.1.0 <pip>
setuptools 38.4.0 py27_0
simplegeneric 0.8.1 py27_2
singledispatch 3.4.0.3 py27h9bcb476_0
sip 4.18 py27_0
six 1.11.0 py27h5f960f1_1
six 1.11.0 <pip>
sk-video 1.1.10 <pip>
smop 0.41 <pip>
snowballstemmer 1.2.1 py27h44e2768_0
sortedcollections 0.5.3 py27h135218e_0
sortedcontainers 1.5.9 py27_0
sphinx 1.6.6 py27_0
sphinxcontrib 1.0 py27h1512b58_1
sphinxcontrib-websupport 1.0.1 py27hf906f22_1
spyder 3.2.6 py27_0
sqlalchemy 1.2.1 py27h14c3975_0
sqlite 3.13.0 1 conda-forge
ssl_match_hostname 3.5.0.1 py27h4ec10b9_2
statsmodels 0.8.0 py27hc87d62d_0
subprocess32 3.2.7 py27h373dbce_0
sympy 1.1.1 py27hc28188a_0
tblib 1.3.2 py27h51fe5ba_0
tensorboard 1.11.0 <pip>
tensorflow-gpu 1.11.0 <pip>
tensorflow-hub 0.1.1 <pip>
tensorflowjs 0.6.1 <pip>
termcolor 1.1.0 <pip>
terminado 0.8.1 py27_1
testpath 0.3.1 py27hc38d2c4_0
tk 8.5.19 2 conda-forge
toolz 0.9.0 py27_0
torchfile 0.1.0 py_0 conda-forge
torchfile 0.1.0 <pip>
torchvision 0.2.1 py27_1 pytorch
tornado 5.1 <pip>
tornado 4.5.3 py27_0
tqdm 4.28.1 py27h28b3542_0 anaconda
traceback2 1.4.0 py27_0 conda-forge
traitlets 4.3.2 py27hd6ce930_0
typing 3.6.2 py27h66f49e2_0
unicodecsv 0.14.1 py27h5062da9_0
unittest2 1.1.0 py_0 conda-forge
unixodbc 2.3.4 hc36303a_1
urllib3 1.22 py27ha55213b_0
urllib3 1.23 <pip>
visdom 0.1.8.4 <pip>
vispy 0.5.3 py27_0 conda-forge
wcwidth 0.1.7 py27h9e3e1ab_0
webencodings 0.5.1 py27hff10b21_1
websocket-client 0.48.0 <pip>
websocket-client 0.48.0 py_0 conda-forge
Werkzeug 0.14.1 <pip>
werkzeug 0.14.1 py27_0
wheel 0.32.2 <pip>
wheel 0.30.0 py27h2bc6bb2_1
wheel 0.32.0 <pip>
widgetsnbextension 3.1.0 py27_0
wrapt 1.10.11 py27h04f6869_0
x264 20131218 0 conda-forge
xlrd 1.1.0 py27ha77178f_1
xlsxwriter 1.0.2 py27h12cbc6b_0
xlwt 1.3.0 py27h3d85d97_0
xorg-kbproto 1.0.7 h470a237_2 conda-forge
xorg-libice 1.0.9 h470a237_4 conda-forge
xorg-libsm 1.2.2 h8c8a85c_6 conda-forge
xorg-libx11 1.6.6 h470a237_0 conda-forge
xorg-libxext 1.3.3 h470a237_4 conda-forge
xorg-libxrender 0.9.10 h470a237_2 conda-forge
xorg-renderproto 0.11.1 h470a237_2 conda-forge
xorg-xextproto 7.3.0 h470a237_2 conda-forge
xorg-xproto 7.0.31 h470a237_7 conda-forge
xz 5.2.4 h470a237_1 conda-forge
yaml 0.1.7 had09818_2
yarg 0.1.9 <pip>
zeromq 4.2.2 hbedb6e5_2
zict 0.1.3 py27h12c336c_0
zlib 1.2.11 ha838bed_2
$ conda config --show
add_anaconda_token: True
add_pip_as_python_dependency: True
aggressive_update_packages:
- ca-certificates
- certifi
- openssl
allow_non_channel_urls: False
allow_softlinks: False
always_copy: False
always_softlink: False
always_yes: None
anaconda_upload: None
auto_update_conda: True
changeps1: True
channel_alias: https://conda.anaconda.org
channel_priority: True
channels:
- conda-forge
- defaults
client_ssl_cert: None
client_ssl_cert_key: None
clobber: False
create_default_packages: []
custom_channels:
pkgs/r: https://repo.anaconda.com
home/memo/anaconda2/conda-bld: file://
pkgs/main: https://repo.anaconda.com
pkgs/pro: https://repo.anaconda.com
pkgs/free: https://repo.anaconda.com
custom_multichannels:
local: ["file:///home/memo/anaconda2/conda-bld"]
defaults: ["https://repo.anaconda.com/pkgs/main", "https://repo.anaconda.com/pkgs/free", "https://repo.anaconda.com/pkgs/r", "https://repo.anaconda.com/pkgs/pro"]
default_channels:
- https://repo.anaconda.com/pkgs/main
- https://repo.anaconda.com/pkgs/free
- https://repo.anaconda.com/pkgs/r
- https://repo.anaconda.com/pkgs/pro
disallowed_packages: []
download_only: False
envs_dirs:
- /home/memo/anaconda2/envs
- /home/memo/.conda/envs
force: False
json: False
local_repodata_ttl: 1
max_shlvl: 2
migrated_channel_aliases: []
no_dependencies: False
non_admin_enabled: True
notify_outdated_conda: True
offline: False
override_channels_enabled: True
path_conflict: clobber
pinned_packages: []
pkgs_dirs:
- /home/memo/anaconda2/pkgs
- /home/memo/.conda/pkgs
proxy_servers: {}
quiet: False
remote_connect_timeout_secs: 9.15
remote_max_retries: 3
remote_read_timeout_secs: 60.0
report_errors: None
rollback_enabled: True
safety_checks: warn
shortcuts: True
show_channel_urls: None
ssl_verify: True
track_features: []
use_index_cache: False
use_pip: True
verbosity: 0
whitelist_channels: []
It seems that the problem is that conda-forge appears first in the channel list. You can remove this channel with
conda config --remove channels conda-forge
While removing Conda Forge from the global config will get around the particular issue, it could impact future solves on existing envs that had previously installed from Conda Forge. Instead, one can enforce the anaconda channel just for this particular env creation:
conda create -n py36 -c anaconda --override-channels python=3.6 anaconda