Pythonistas!
Not sure what I am doing wrong while reading a parquet file here.
I have all the necessary packages installed - pandas, fastparquet & pyarrow
The code literally is reading the parquet file
import pandas as pd
FILE = 'file://minute_equities_all/minute/A/AA.parquet'
pd = pd.read_parquet(FILE,engine='fastparquet')
This keeps giving me,
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pandas/io/parquet.py", line 493, in read_parquet
return impl.read(
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pandas/io/parquet.py", line 347, in read
result = parquet_file.to_pandas(columns=columns, **kwargs)
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/fastparquet/api.py", line 722, in to_pandas
rgs = filter_row_groups(self, filters) if filters else self.row_groups
AttributeError: 'ParquetFile' object has no attribute 'row_groups'
Changing the engine to pyarrow also doesn't help -
Traceback (most recent call last):
File "/Users/ppatel/Desktop/development/data-sync/main.py", line 5, in <module>
pf = pd.read_parquet(FILE,engine='pyarrow')
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pandas/io/parquet.py", line 493, in read_parquet
return impl.read(
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pandas/io/parquet.py", line 240, in read
result = self.api.parquet.read_table(
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pyarrow/parquet/__init__.py", line 2737, in read_table
dataset = _ParquetDatasetV2(
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pyarrow/parquet/__init__.py", line 2351, in __init__
self._dataset = ds.dataset(path_or_paths, filesystem=filesystem,
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pyarrow/dataset.py", line 694, in dataset
return _filesystem_dataset(source, **kwargs)
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pyarrow/dataset.py", line 439, in _filesystem_dataset
fs, paths_or_selector = _ensure_single_source(source, filesystem)
File "/Users/ppatel/opt/miniconda3/envs/quant/lib/python3.10/site-packages/pyarrow/dataset.py", line 415, in _ensure_single_source
raise FileNotFoundError(path)
FileNotFoundError: /Users/ppatel/Desktop/development/data-sync/minute_equities_all/minute/A/AA.parquet
Am I missing something here? Any help will be appreciated!
Attaching an example file here (expires in 30 days) - https://easyupload.io/xsqhhw
Adding some details around package versions -
# Name Version Build Channel
abseil-cpp 20211102.0 he49afe7_0 conda-forge
arrow-cpp 8.0.0 py310h72c8010_0
aws-c-common 0.4.57 hb1e8313_1
aws-c-event-stream 0.1.6 h23ab428_5
aws-checksums 0.1.9 hb1e8313_0
aws-sdk-cpp 1.8.185 he271ece_0
beautifulsoup4 4.11.1 py310hecd8cb5_0
blas 1.0 mkl
boost-cpp 1.70.0 hd59e818_1 conda-forge
bottleneck 1.3.5 py310h4e76f89_0
brotli 1.0.9 hca72f7f_7
brotli-bin 1.0.9 hca72f7f_7
brotlipy 0.7.0 py310h1961e1f_1004 conda-forge
bs4 4.11.1 hd3eb1b0_0
bzip2 1.0.8 h1de35cc_0
c-ares 1.18.1 h0d85af4_0 conda-forge
ca-certificates 2022.6.15.1 h033912b_0 conda-forge
certifi 2022.6.15.1 pyhd8ed1ab_0 conda-forge
cffi 1.15.1 py310hc55c11b_0
charset-normalizer 2.1.1 pyhd8ed1ab_0 conda-forge
cramjam 2.5.0 py310hcf052fe_0 conda-forge
cryptography 37.0.4 py310h52c3658_0 conda-forge
cycler 0.11.0 pyhd3eb1b0_0
et_xmlfile 1.0.1 py_1001 conda-forge
fastparquet 0.8.3 py310h936d966_0 conda-forge
fonttools 4.25.0 pyhd3eb1b0_0
freetype 2.11.0 hd8bbffd_0
fsspec 2022.8.2 pyhd8ed1ab_0 conda-forge
gflags 2.2.2 hb1e8313_1004 conda-forge
giflib 5.2.1 haf1e3a3_0
glog 0.5.0 h25b26a9_0 conda-forge
grpc-cpp 1.46.1 h067a048_0
icu 58.2 h0a44026_1000 conda-forge
idna 3.3 pyhd8ed1ab_0 conda-forge
intel-openmp 2021.4.0 hecd8cb5_3538
jpeg 9e hca72f7f_0
kiwisolver 1.4.2 py310he9d5cce_0
krb5 1.19.2 hcd88c3b_0
lcms2 2.12 hf1fd2bf_0
lerc 3.0 he9d5cce_0
libbrotlicommon 1.0.9 hca72f7f_7
libbrotlidec 1.0.9 hca72f7f_7
libbrotlienc 1.0.9 hca72f7f_7
libcurl 7.84.0 h6dfd666_0
libcxx 12.0.0 h2f01273_0
libdeflate 1.8 h9ed2024_5
libedit 3.1.20210910 hca72f7f_0
libev 4.33 haf1e3a3_1 conda-forge
libevent 2.1.10 h815e4d9_4 conda-forge
libffi 3.3 hb1e8313_2
libiconv 1.17 hac89ed1_0 conda-forge
libnghttp2 1.46.0 ha29bfda_0
libpng 1.6.37 ha441bb4_0
libprotobuf 3.20.1 h8346a28_0
libssh2 1.10.0 h7535e13_3 conda-forge
libta-lib 0.4.0 haf1e3a3_0 conda-forge
libthrift 0.15.0 h054ceb0_0
libtiff 4.4.0 h2ef1027_0
libwebp 1.2.2 h56c3ce4_0
libwebp-base 1.2.2 hca72f7f_0
libxml2 2.9.14 hbf8cd5e_0
libxslt 1.1.35 h5b33f42_0
libzlib 1.2.12 hfe4f2af_2 conda-forge
lxml 4.9.1 py310h6c45266_0 conda-forge
lz4-c 1.9.3 h23ab428_1
matplotlib 3.5.2 py310hecd8cb5_0
matplotlib-base 3.5.2 py310hfb0c5b7_0
mkl 2021.4.0 hecd8cb5_637
mkl-service 2.4.0 py310hca72f7f_0
mkl_fft 1.3.1 py310hf879493_0
mkl_random 1.2.2 py310hc081a56_0
multitasking 0.0.9 pyhd8ed1ab_0 conda-forge
munkres 1.1.4 py_0
ncurses 6.3 hca72f7f_3
numexpr 2.8.3 py310hdcd3fac_0
numpy 1.23.1 py310hdcd3fac_0
numpy-base 1.23.1 py310hfd2de13_0
oandapyv20 0.7.2 pypi_0 pypi
openpyxl 3.0.9 pyhd8ed1ab_0 conda-forge
openssl 1.1.1q hfe4f2af_0 conda-forge
orc 1.7.4 h9274d09_0
packaging 21.3 pyhd3eb1b0_0
pandas 1.4.3 py310he9d5cce_0
pillow 9.2.0 py310hde71d04_1
pip 22.1.2 py310hecd8cb5_0
pyarrow 8.0.0 py310h53e4f6e_0
pycparser 2.21 pyhd8ed1ab_0 conda-forge
pyopenssl 22.0.0 pyhd8ed1ab_0 conda-forge
pyparsing 3.0.9 py310hecd8cb5_0
pysocks 1.7.1 pyha2e5f31_6 conda-forge
python 3.10.4 hdfd78df_0
python-dateutil 2.8.2 pyhd3eb1b0_0
python_abi 3.10 2_cp310 conda-forge
pytz 2022.1 py310hecd8cb5_0
re2 2022.04.01 he9d5cce_0
readline 8.1.2 hca72f7f_1
requests 2.28.1 pyhd8ed1ab_1 conda-forge
setuptools 63.4.1 py310hecd8cb5_0
six 1.16.0 pyhd3eb1b0_1
snappy 1.1.9 he9d5cce_0
soupsieve 2.3.1 pyhd3eb1b0_0
sqlite 3.39.2 h707629a_0
ta-lib 0.4.19 py310h7f5fb2b_4 conda-forge
tk 8.6.12 h5d9f67b_0
tornado 6.2 py310hca72f7f_0
tzdata 2022a hda174b7_0
urllib3 1.26.11 pyhd8ed1ab_0 conda-forge
utf8proc 2.6.1 h9ed2024_0
wheel 0.37.1 pyhd3eb1b0_0
xz 5.2.5 hca72f7f_1
yfinance 0.1.74 pyhd8ed1ab_0 conda-forge
zlib 1.2.12 h4dc903c_2
zstd 1.5.2 hcb37349_0
I am suffering similar issue with you. It can be possible to have file path issue. In my case, when i tried to access S3 file, happened that error. and this is permission issue to access S3.
Related
I did conda list but I can't see what I am looking for. Are conda packages installed with conda developed . not shown?
# packages in environment at /Users/pinocchio/anaconda3/envs/myenv:
#
# Name Version Build Channel
absl-py 0.9.0 py37_0
appnope 0.1.0 py37hc8dfbb8_1001 conda-forge
asn1crypto 1.3.0 py37_0
astroid 2.3.3 py37_0
attrs 19.3.0 py_0 conda-forge
backcall 0.1.0 py_0 conda-forge
beautifulsoup4 4.8.2 py37_0
blas 1.0 mkl
bleach 3.1.4 pyh9f0ad1d_0 conda-forge
bzip2 1.0.8 h1de35cc_0
c-ares 1.15.0 h1de35cc_1001
ca-certificates 2020.1.1 0
cairo 1.14.12 hc4e6be7_4
certifi 2020.4.5.1 py37_0
cffi 1.14.0 py37hb5b8e2f_0
chardet 3.0.4 py37_1003
conda 4.8.3 py37_0
conda-build 3.18.11 py37_0
conda-package-handling 1.6.0 py37h1de35cc_0
cryptography 2.8 py37ha12b0ac_0
cycler 0.10.0 py37_0
decorator 4.4.2 py_0 conda-forge
defusedxml 0.6.0 py_0 conda-forge
entrypoints 0.3 py37hc8dfbb8_1001 conda-forge
expat 2.2.6 h0a44026_0
filelock 3.0.12 py_0
fontconfig 2.13.0 h5d5b041_1
freetype 2.9.1 hb4e5f40_0
fribidi 1.0.5 h1de35cc_0
gettext 0.19.8.1 h15daf44_3
glib 2.63.1 hd977a24_0
glob2 0.7 py_0
graphite2 1.3.13 h2098e52_0
graphviz 2.40.1 hefbbd9a_2
grpcio 1.16.1 py37h044775b_1
harfbuzz 1.8.8 hb8d4a28_0
icu 58.2 h4b95b61_1
idna 2.9 py_1
importlib-metadata 1.6.0 py37hc8dfbb8_0 conda-forge
importlib_metadata 1.6.0 0 conda-forge
intel-openmp 2019.4 233
ipykernel 5.2.0 py37h43977f1_1 conda-forge
ipython 7.13.0 py37hc8dfbb8_2 conda-forge
ipython_genutils 0.2.0 py_1 conda-forge
isort 4.3.21 py37_0
jedi 0.16.0 py37hc8dfbb8_1 conda-forge
jinja2 2.11.1 py_0
jpeg 9b he5867d9_2
json5 0.9.0 py_0 conda-forge
jsonschema 3.2.0 py37hc8dfbb8_1 conda-forge
jupyter_client 6.1.2 py_0 conda-forge
jupyter_core 4.6.3 py37hc8dfbb8_1 conda-forge
jupyterlab 2.0.1 py_0 conda-forge
jupyterlab_server 1.0.7 py_0 conda-forge
kiwisolver 1.1.0 py37h0a44026_0
lazy-object-proxy 1.4.3 py37h1de35cc_0
libarchive 3.3.3 h786848e_5
libcxx 4.0.1 hcfea43d_1
libcxxabi 4.0.1 hcfea43d_1
libedit 3.1.20181209 hb402a30_0
libffi 3.2.1 h475c297_4
libgfortran 3.0.1 h93005f0_2
libiconv 1.15 hdd342a3_7
liblief 0.9.0 h2a1bed3_2
libpng 1.6.37 ha441bb4_0
libprotobuf 3.11.4 hd9629dc_0
libsodium 1.0.17 h01d97ff_0 conda-forge
libtiff 4.1.0 hcb84e12_0
libxml2 2.9.9 hf6e021a_1
lz4-c 1.8.1.2 h1de35cc_0
lzo 2.10 h362108e_2
markdown 3.1.1 py37_0
markupsafe 1.1.1 py37h1de35cc_0
matplotlib 3.1.3 py37_0
matplotlib-base 3.1.3 py37h9aa3819_0
mccabe 0.6.1 py37_1
mistune 0.8.4 py37h0b31af3_1000 conda-forge
mkl 2019.4 233
mkl-service 2.3.0 py37hfbe908c_0
mkl_fft 1.0.15 py37h5e564d8_0
mkl_random 1.1.0 py37ha771720_0
nbconvert 5.6.1 py37_0 conda-forge
nbformat 5.0.4 py_0 conda-forge
ncurses 6.2 h0a44026_0
ninja 1.9.0 py37h04f5b5a_0
notebook 6.0.3 py37_0 conda-forge
numpy 1.18.1 py37h7241aed_0
numpy-base 1.18.1 py37h6575580_1
olefile 0.46 py37_0
openssl 1.1.1g h1de35cc_0
pandoc 2.9.2 0 conda-forge
pandocfilters 1.4.2 py_1 conda-forge
pango 1.42.4 h060686c_0
parso 0.6.2 py_0 conda-forge
pcre 8.43 h0a44026_0
pexpect 4.8.0 py37hc8dfbb8_1 conda-forge
pickleshare 0.7.5 py37hc8dfbb8_1001 conda-forge
pillow 6.2.1 py37hb68e598_0
pip 20.0.2 py37_1
pixman 0.38.0 h1de35cc_0
pkginfo 1.5.0.1 py37_0
prometheus_client 0.7.1 py_0 conda-forge
prompt-toolkit 3.0.5 py_0 conda-forge
protobuf 3.11.4 py37h0a44026_0
psutil 5.7.0 py37h1de35cc_0
ptyprocess 0.6.0 py_1001 conda-forge
py-lief 0.9.0 py37h1413db1_2
pycosat 0.6.3 py37h1de35cc_0
pycparser 2.20 py_0
pygments 2.6.1 py_0 conda-forge
pylint 2.4.4 py37_0
pyopenssl 19.1.0 py37_0
pyparsing 2.4.6 py_0
pyrsistent 0.16.0 py37h9bfed18_0 conda-forge
pysocks 1.7.1 py37_0
python 3.7.7 hc70fcce_0_cpython
python-dateutil 2.8.1 py_0 conda-forge
python-graphviz 0.13.2 pypi_0 pypi
python-libarchive-c 2.8 py37_13
python_abi 3.7 1_cp37m conda-forge
pytorch 1.4.0 py3.7_0 pytorch
pytz 2019.3 py_0
pyyaml 5.3.1 py37h1de35cc_0
pyzmq 18.1.1 py37h0a44026_0
readline 8.0 h1de35cc_0
requests 2.23.0 py37_0
ripgrep 11.0.2 he32d670_0
ruamel_yaml 0.15.87 py37h1de35cc_0
send2trash 1.5.0 py_0 conda-forge
setuptools 46.1.1 py37_0
six 1.14.0 py37_0
soupsieve 2.0 py_0
sqlite 3.31.1 ha441bb4_0
tensorboard 2.0.0 pyhb38c66f_1
terminado 0.8.3 py37hc8dfbb8_1 conda-forge
testpath 0.4.4 py_0 conda-forge
tk 8.6.8 ha441bb4_0
torchvision 0.2.1 py_2 soumith
tornado 6.0.4 py37h9bfed18_1 conda-forge
tqdm 4.43.0 py_0
traitlets 4.3.3 py37hc8dfbb8_1 conda-forge
urllib3 1.25.8 py37_0
wcwidth 0.1.9 pyh9f0ad1d_0 conda-forge
webencodings 0.5.1 py_1 conda-forge
werkzeug 1.0.0 py_0
wheel 0.34.2 py37_0
wrapt 1.12.1 py37h1de35cc_1
xz 5.2.4 h1de35cc_4
yaml 0.1.7 hc338f04_2
zeromq 4.3.2 h6de7cb9_2 conda-forge
zipp 3.1.0 py_0 conda-forge
zlib 1.2.11 h1de35cc_3
zstd 1.3.7 h5bba6e5_0
utils or something like that should be there but its not.
Update:
I've tried what this suggests and it does not work. I conda develop . my package and then when I run the python interpreter and try to import it the import fails. Why?
>>> import my_pkg
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'my_pkg'
I also tried to install it with pip pip install -e and it doesn't find my package either after I go to the interpreter and try to import it. Why is that?
If you look carefully after running conda develop . (where the setup.py file is) you will notice that the command outputs the following:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop .
added /Users/brandBrandoParetoopareto/ultimate-utils/uutils
completed operation for: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
if you then check the sys.path python has then you can that new path has been added:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
/Users/brandBrandoParetoopareto/ultimate-utils/uutils
you can confirm this by removing it and printing the contents of sys.path:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ conda develop -u .
uninstalled: /Users/brandBrandoParetoopareto/ultimate-utils/uutils
check sys path again:
(automl) brandBrandoParetoopareto~/ultimate-utils/uutils $ python
Python 3.7.7 (default, Mar 26 2020, 10:32:53)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> for p in sys.path:
... print(p)
...
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python37.zip
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/lib-dynload
/Users/brandBrandoParetoopareto/anaconda3/envs/automl/lib/python3.7/site-packages
/Users/brandBrandoParetoopareto/automl-meta-learning/automl
/Users/brandBrandoParetoopareto/higher
now it's gone!
Also, note that conda or python (not sure which) also adds the current path always. This is important to note because when I did import tests outside of the repo I was working it wasn't working but it was inside the repo which seemed very mysterious which I can only attribute to that.
Addendum
to check python sys.path from terminal run that command as a string with python cmd:
python -c "import sys; print(sys.path)"
even better:
python -c "import sys; [print(p) for p in sys.path]"
Original answer: https://stackoverflow.com/a/59903590/1601580
I'm new to Python and am using Anaconda on Windows 10 to learn how to implement machine learning. Running this code on Spyder:
import sklearn as skl
Originally got me this:
Traceback (most recent call last):
File "<ipython-input-1-7135d3f24347>", line 1, in <module>
runfile('C:/Users/julia/.spyder-py3/temp.py', wdir='C:/Users/julia/.spyder-py3')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/julia/.spyder-py3/temp.py", line 3, in <module>
from sklearn.family import Model
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\__init__.py", line 76, in <module>
from .base import clone
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\base.py", line 16, in <module>
from .utils import _IS_32BIT
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\__init__.py", line 20, in <module>
from .validation import (as_float_array,
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\validation.py", line 21, in <module>
from .fixes import _object_dtype_isnan
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\fixes.py", line 289, in <module>
from scipy.sparse.linalg import lsqr as sparse_lsqr
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\__init__.py", line 114, in <module>
from .isolve import *
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\isolve\__init__.py", line 6, in <module>
from .iterative import *
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\isolve\iterative.py", line 10, in <module>
from . import _iterative
ImportError: DLL load failed: The specified module could not be found.
I then went to the command line and did
pip uninstall scipy
pip install scipy
pip uninstall scikit-learn
pip install scikit-learn
and got no errors when doing so, with scipy 1.3.1 (along with numpy 1.17.0) and scikit-learn 0.21.3 being installed according to the command line.
However, now when I try to import sklearn I get a different error:
File "<ipython-input-2-7135d3f24347>", line 1, in <module>
runfile('C:/Users/julia/.spyder-py3/temp.py', wdir='C:/Users/julia/.spyder-py3')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/julia/.spyder-py3/temp.py", line 3, in <module>
from sklearn.family import Model
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\__init__.py", line 76, in <module>
from .base import clone
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\base.py", line 16, in <module>
from .utils import _IS_32BIT
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\__init__.py", line 20, in <module>
from .validation import (as_float_array,
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\validation.py", line 21, in <module>
from .fixes import _object_dtype_isnan
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\utils\fixes.py", line 289, in <module>
from scipy.sparse.linalg import lsqr as sparse_lsqr
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\__init__.py", line 113, in <module>
from .isolve import *
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\isolve\__init__.py", line 6, in <module>
from .iterative import *
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\sparse\linalg\isolve\iterative.py", line 136, in <module>
def bicg(A, b, x0=None, tol=1e-5, maxiter=None, M=None, callback=None, atol=None):
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\_lib\_threadsafety.py", line 59, in decorator
return lock.decorate(func)
File "C:\ProgramData\Anaconda3\lib\site-packages\scipy\_lib\_threadsafety.py", line 47, in decorate
return scipy._lib.decorator.decorate(func, caller)
AttributeError: module 'scipy' has no attribute '_lib'
Any suggestions? I've uninstalled and reinstalled Anaconda and I'm still getting the same issue.
EDIT: When I do
conda list --show-channel-urls
I get
# packages in environment at C:\ProgramData\Anaconda3:
#
# Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py37_0 defaults
alabaster 0.7.12 py37_0 defaults
anaconda-client 1.7.2 py37_0 defaults
anaconda-navigator 1.9.7 py37_0 defaults
asn1crypto 0.24.0 py37_0 defaults
astroid 2.2.5 py37_0 defaults
attrs 19.1.0 py37_1 defaults
babel 2.7.0 py_0 defaults
backcall 0.1.0 py37_0 defaults
backports 1.0 py_2 defaults
backports.functools_lru_cache 1.5 py_2 defaults
backports.tempfile 1.0 py_1 defaults
backports.weakref 1.0.post1 py_1 defaults
beautifulsoup4 4.7.1 py37_1 defaults
blas 1.0 mkl defaults
bleach 3.1.0 py37_0 defaults
bzip2 1.0.8 he774522_0 defaults
ca-certificates 2019.5.15 1 defaults
certifi 2019.6.16 py37_1 defaults
cffi 1.12.3 py37h7a1dbc1_0 defaults
chardet 3.0.4 py37_1003 defaults
click 7.0 py37_0 defaults
cloudpickle 1.2.1 py_0 defaults
clyent 1.2.2 py37_1 defaults
colorama 0.4.1 py37_0 defaults
conda 4.7.11 py37_0 defaults
conda-build 3.18.8 py37_0 defaults
conda-env 2.6.0 1 defaults
conda-package-handling 1.3.11 py37_0 defaults
conda-verify 3.4.2 py_1 defaults
console_shortcut 0.1.1 3 defaults
cryptography 2.7 py37h7a1dbc1_0 defaults
decorator 4.4.0 py37_1 defaults
defusedxml 0.6.0 py_0 defaults
docutils 0.15.1 py37_0 defaults
entrypoints 0.3 py37_0 defaults
filelock 3.0.12 py_0 defaults
freetype 2.9.1 ha9979f8_1 defaults
future 0.17.1 py37_0 defaults
glob2 0.7 py_0 defaults
icc_rt 2019.0.0 h0cc432a_1 defaults
icu 58.2 ha66f8fd_1 defaults
idna 2.8 py37_0 defaults
imagesize 1.1.0 py37_0 defaults
intel-openmp 2019.4 245 defaults
ipykernel 5.1.1 py37h39e3cac_0 defaults
ipython 7.7.0 py37h39e3cac_0 defaults
ipython_genutils 0.2.0 py37_0 defaults
ipywidgets 7.5.1 py_0 defaults
isort 4.3.21 py37_0 defaults
jedi 0.13.3 py37_0 defaults
jinja2 2.10.1 py37_0 defaults
joblib 0.13.2 py37_0 defaults
jpeg 9b hb83a4c4_2 defaults
json5 0.8.5 py_0 defaults
jsonschema 3.0.1 py37_0 defaults
jupyter_client 5.3.1 py_0 defaults
jupyter_core 4.5.0 py_0 defaults
jupyterlab 1.0.2 py37hf63ae98_0 defaults
jupyterlab_server 1.0.0 py_1 defaults
keyring 18.0.0 py37_0 defaults
lazy-object-proxy 1.4.1 py37he774522_0 defaults
libarchive 3.3.3 h0643e63_5 defaults
libiconv 1.15 h1df5818_7 defaults
liblief 0.9.0 ha925a31_2 defaults
libpng 1.6.37 h2a8f88b_0 defaults
libsodium 1.0.16 h9d3ae62_0 defaults
libtiff 4.0.10 hb898794_2 defaults
libxml2 2.9.9 h464c3ec_0 defaults
lz4-c 1.8.1.2 h2fa13f4_0 defaults
lzo 2.10 h6df0209_2 defaults
m2w64-gcc-libgfortran 5.3.0 6 defaults
m2w64-gcc-libs 5.3.0 7 defaults
m2w64-gcc-libs-core 5.3.0 7 defaults
m2w64-gmp 6.1.0 2 defaults
m2w64-libwinpthread-git 5.0.0.4634.697f757 2 defaults
markupsafe 1.1.1 py37he774522_0 defaults
mccabe 0.6.1 py37_1 defaults
menuinst 1.4.16 py37he774522_0 defaults
mistune 0.8.4 py37he774522_0 defaults
mkl 2019.4 245 defaults
mkl-service 2.0.2 py37he774522_0 defaults
mkl_fft 1.0.12 py37h14836fe_0 defaults
mkl_random 1.0.2 py37h343c172_0 defaults
msys2-conda-epoch 20160418 1 defaults
navigator-updater 0.2.1 py37_0 defaults
nbconvert 5.5.0 py_0 defaults
nbformat 4.4.0 py37_0 defaults
notebook 6.0.0 py37_0 defaults
numpy 1.17.0 pypi_0 pypi
numpy-base 1.16.4 py37hc3f5095_0 defaults
numpydoc 0.9.1 py_0 defaults
olefile 0.46 py37_0 defaults
openssl 1.1.1c he774522_1 defaults
packaging 19.0 py37_0 defaults
pandas 0.25.0 py37ha925a31_0 defaults
pandoc 2.2.3.2 0 defaults
pandocfilters 1.4.2 py37_1 defaults
parso 0.5.0 py_0 defaults
pickleshare 0.7.5 py37_0 defaults
pillow 6.1.0 py37hdc69c19_0 defaults
pip 19.2.2 pypi_0 pypi
pkginfo 1.5.0.1 py37_0 defaults
powershell_shortcut 0.0.1 2 defaults
prometheus_client 0.7.1 py_0 defaults
prompt_toolkit 2.0.9 py37_0 defaults
psutil 5.6.3 py37he774522_0 defaults
py-lief 0.9.0 py37ha925a31_2 defaults
pycodestyle 2.5.0 py37_0 defaults
pycosat 0.6.3 py37hfa6e2cd_0 defaults
pycparser 2.19 py37_0 defaults
pyflakes 2.1.1 py37_0 defaults
pygments 2.4.2 py_0 defaults
pylint 2.3.1 py37_0 defaults
pyopenssl 19.0.0 py37_0 defaults
pyparsing 2.4.0 py_0 defaults
pyqt 5.9.2 py37h6538335_2 defaults
pyrsistent 0.14.11 py37he774522_0 defaults
pysocks 1.7.0 py37_0 defaults
python 3.7.3 h8c8aaf0_1 defaults
python-dateutil 2.8.0 py37_0 defaults
python-libarchive-c 2.8 py37_13 defaults
pytz 2019.1 py_0 defaults
pywin32 223 py37hfa6e2cd_1 defaults
pywinpty 0.5.5 py37_1000 defaults
pyyaml 5.1.1 py37he774522_0 defaults
pyzmq 18.0.0 py37ha925a31_0 defaults
qt 5.9.7 vc14h73c81de_0 defaults
qtawesome 0.5.7 py37_1 defaults
qtconsole 4.5.2 py_0 defaults
qtpy 1.8.0 py_0 defaults
requests 2.22.0 py37_0 defaults
rope 0.14.0 py_0 defaults
ruamel_yaml 0.15.46 py37hfa6e2cd_0 defaults
scikit-learn 0.21.3 pypi_0 pypi
scipy 1.3.0 pypi_0 pypi
send2trash 1.5.0 py37_0 defaults
setuptools 41.0.1 py37_0 defaults
sip 4.19.8 py37h6538335_0 defaults
six 1.12.0 py37_0 defaults
snowballstemmer 1.9.0 py_0 defaults
soupsieve 1.9.2 py37_0 defaults
sphinx 2.1.2 py_0 defaults
sphinxcontrib-applehelp 1.0.1 py_0 defaults
sphinxcontrib-devhelp 1.0.1 py_0 defaults
sphinxcontrib-htmlhelp 1.0.2 py_0 defaults
sphinxcontrib-jsmath 1.0.1 py_0 defaults
sphinxcontrib-qthelp 1.0.2 py_0 defaults
sphinxcontrib-serializinghtml 1.1.3 py_0 defaults
spyder 3.3.6 py37_0 defaults
spyder-kernels 0.5.1 py37_0 defaults
sqlite 3.29.0 he774522_0 defaults
terminado 0.8.2 py37_0 defaults
testpath 0.4.2 py37_0 defaults
tk 8.6.8 hfa6e2cd_0 defaults
tornado 6.0.3 py37he774522_0 defaults
tqdm 4.32.1 py_0 defaults
traitlets 4.3.2 py37_0 defaults
urllib3 1.24.2 py37_0 defaults
vc 14.1 h0510ff6_4 defaults
vs2015_runtime 14.15.26706 h3a45250_4 defaults
wcwidth 0.1.7 py37_0 defaults
webencodings 0.5.1 py37_1 defaults
wheel 0.33.4 py37_0 defaults
widgetsnbextension 3.5.0 py37_0 defaults
win_inet_pton 1.1.0 py37_0 defaults
wincertstore 0.2 py37_0 defaults
winpty 0.4.3 4 defaults
wrapt 1.11.2 py37he774522_0 defaults
xz 5.2.4 h2fa13f4_4 defaults
yaml 0.1.7 hc54c509_2 defaults
zeromq 4.3.1 h33f27b4_3 defaults
zlib 1.2.11 h62dcd97_3 defaults
zstd 1.3.7 h508b16e_0 defaults
with the version of scipy not matching up with the version that pip installed. Not sure how significant it is but it seemed strange to me.
EDIT 2:
Doing pip list returns
Package Version
----------------------------- ---------
-cipy 1.3.0
alabaster 0.7.12
anaconda-client 1.7.2
anaconda-navigator 1.9.7
asn1crypto 0.24.0
astroid 2.2.5
attrs 19.1.0
Babel 2.7.0
backcall 0.1.0
backports.functools-lru-cache 1.5
backports.tempfile 1.0
backports.weakref 1.0.post1
beautifulsoup4 4.7.1
bleach 3.1.0
certifi 2019.6.16
cffi 1.12.3
chardet 3.0.4
Click 7.0
cloudpickle 1.2.1
clyent 1.2.2
colorama 0.4.1
conda 4.7.11
conda-build 3.18.8
conda-package-handling 1.3.11
conda-verify 3.4.2
cryptography 2.7
decorator 4.4.0
defusedxml 0.6.0
docutils 0.15.1
entrypoints 0.3
filelock 3.0.12
future 0.17.1
glob2 0.7
idna 2.8
imagesize 1.1.0
ipykernel 5.1.1
ipython 7.7.0
ipython-genutils 0.2.0
ipywidgets 7.5.1
isort 4.3.21
jedi 0.13.3
Jinja2 2.10.1
joblib 0.13.2
json5 0.8.5
jsonschema 3.0.1
jupyter-client 5.3.1
jupyter-core 4.5.0
jupyterlab 1.0.2
jupyterlab-server 1.0.0
keyring 18.0.0
lazy-object-proxy 1.4.1
libarchive-c 2.8
MarkupSafe 1.1.1
mccabe 0.6.1
menuinst 1.4.16
mistune 0.8.4
mkl-fft 1.0.12
mkl-random 1.0.2
mkl-service 2.0.2
navigator-updater 0.2.1
nbconvert 5.5.0
nbformat 4.4.0
notebook 6.0.0
numpy 1.17.0
numpydoc 0.9.1
olefile 0.46
packaging 19.0
pandas 0.25.0
pandocfilters 1.4.2
parso 0.5.0
pickleshare 0.7.5
Pillow 6.1.0
pio 0.0.3
pip 19.2.2
pkginfo 1.5.0.1
prometheus-client 0.7.1
prompt-toolkit 2.0.9
psutil 5.6.3
pycodestyle 2.5.0
pycosat 0.6.3
pycparser 2.19
pyflakes 2.1.1
Pygments 2.4.2
pylint 2.3.1
pyOpenSSL 19.0.0
pyparsing 2.4.0
pyrsistent 0.14.11
PySocks 1.7.0
python-dateutil 2.8.0
pytz 2019.1
pywin32 223
pywinpty 0.5.5
PyYAML 5.1.1
pyzmq 18.0.0
QtAwesome 0.5.7
qtconsole 4.5.2
QtPy 1.8.0
requests 2.22.0
rope 0.14.0
ruamel-yaml 0.15.46
scikit-learn 0.21.3
scipy 1.3.1
Send2Trash 1.5.0
setuptools 41.0.1
six 1.12.0
snowballstemmer 1.9.0
soupsieve 1.9.2
Sphinx 2.1.2
sphinxcontrib-applehelp 1.0.1
sphinxcontrib-devhelp 1.0.1
sphinxcontrib-htmlhelp 1.0.2
sphinxcontrib-jsmath 1.0.1
sphinxcontrib-qthelp 1.0.2
sphinxcontrib-serializinghtml 1.1.3
spyder 3.3.6
spyder-kernels 0.5.1
terminado 0.8.2
testpath 0.4.2
tornado 6.0.3
tqdm 4.32.1
traitlets 4.3.2
urllib3 1.24.2
wcwidth 0.1.7
webencodings 0.5.1
wheel 0.33.4
widgetsnbextension 3.5.0
win-inet-pton 1.1.0
wincertstore 0.2
wrapt 1.11.2
pip list says scipy is version 1.3.1, while conda list says it's version 1.3.0. Again, not sure how relevant it is, but seems strange
EDIT 3: I got this error after putting the following lines (suggested by #Brennan) in my command prompt then running the file
pip uninstall scikit-learn
pip uninstall scipy
conda uninstall scikit-learn
conda uninstall scipy
conda update --all
conda install scipy
conda install scikit-learn
This is the new error I get when trying to import sklearn:
Traceback (most recent call last):
File "<ipython-input-15-7135d3f24347>", line 1, in <module>
runfile('C:/Users/julia/.spyder-py3/temp.py', wdir='C:/Users/julia/.spyder-py3')
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 827, in runfile
execfile(filename, namespace)
File "C:\ProgramData\Anaconda3\lib\site-packages\spyder_kernels\customize\spydercustomize.py", line 110, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/julia/.spyder-py3/temp.py", line 2, in <module>
import sklearn as skl
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\__init__.py", line 76, in <module>
from .base import clone
File "C:\ProgramData\Anaconda3\lib\site-packages\sklearn\base.py", line 13, in <module>
import numpy as np
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\__init__.py", line 140, in <module>
from . import _distributor_init
File "C:\ProgramData\Anaconda3\lib\site-packages\numpy\_distributor_init.py", line 34, in <module>
from . import _mklinit
ImportError: DLL load failed: The specified module could not be found.
A possible cause of this might be me deleting the mkl_rt.dll file from my Anaconda/Library/bin after encountering the error described here: https://github.com/ContinuumIO/anaconda-issues/issues/10182
This puts me in a predicament, because reinstalling Anaconda to repair this will get me the same "ordinal 242 could not be located" error that I faced earlier, but not repairing it will continue the issue with sklearn...
FINAL EDIT: Solved by installing old version of Anaconda. Will mark as solved when I am able to (2 days)
I ended up fixing this by uninstalling my current version of Anaconda and installing a version from a few months ago. I didn't get the "ordinal 242" error nor the issues with scikit-learn.
I encountered the same error after letting my PC sit for 4 days unattended. Restarting the kernel solved it.
This probably won't work for everyone, but it might save someone a little agony.
Somehow when I do the install it installs torchvision but not torch. Command I am running as dictated from the main website:
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
then I do conda list but look:
$ conda list
# packages in environment at /home/ubuntu/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
alabaster 0.7.10 py36h306e16b_0
anaconda-client 1.6.14 py36_0
anaconda-project 0.8.2 py36h44fb852_0
argparse 1.4.0 <pip>
asn1crypto 0.24.0 py36_0
astroid 1.6.3 py36_0
astropy 3.0.2 py36h3010b51_1
attrs 18.1.0 py36_0
autovizwidget 0.12.7 <pip>
babel 2.5.3 py36_0
backcall 0.1.0 py36_0
backports 1.0 py36hfa02d7e_1
backports.shutil_get_terminal_size 1.0.0 py36hfea85ff_2
bcrypt 3.1.6 <pip>
beautifulsoup4 4.6.0 py36h49b8c8c_1
bitarray 0.8.1 py36h14c3975_1
bkcharts 0.2 py36h735825a_0
blas 1.0 mkl
blaze 0.11.3 py36h4e06776_0
bleach 2.1.3 py36_0
blosc 1.14.3 hdbcaa40_0
bokeh 1.0.4 py36_0
boto 2.48.0 py36h6e4cd66_1
boto3 1.9.146 <pip>
boto3 1.9.134 py_0
botocore 1.12.146 <pip>
botocore 1.12.134 py_0
bottleneck 1.2.1 py36haac1ea0_0
bzip2 1.0.6 h14c3975_5
ca-certificates 2019.1.23 0
cached-property 1.5.1 <pip>
cairo 1.14.12 h8948797_3
certifi 2019.3.9 py36_0
cffi 1.11.5 py36h9745a5d_0
chardet 3.0.4 py36h0f667ec_1
click 6.7 py36h5253387_0
cloudpickle 0.5.3 py36_0
clyent 1.2.2 py36h7e57e65_1
colorama 0.3.9 py36h489cec4_0
contextlib2 0.5.5 py36h6c84a62_0
cryptography 2.3.1 py36hc365091_0
cudatoolkit 10.0.130 0
curl 7.60.0 h84994c4_0
cycler 0.10.0 py36h93f1223_0
cymem 2.0.2 py36hfd86e86_0
cython 0.28.2 py36h14c3975_0
cytoolz 0.9.0.1 py36h14c3975_0
dask 0.17.5 py36_0
dask-core 0.17.5 py36_0
dataclasses 0.6 py_0 fastai
datashape 0.5.4 py36h3ad6b5c_0
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py36_0
defusedxml 0.6.0 py_0
dill 0.2.9 py36_0
distributed 1.21.8 py36_0
docker 3.7.2 <pip>
docker-compose 1.24.0 <pip>
docker-pycreds 0.4.0 <pip>
dockerpty 0.4.1 <pip>
docopt 0.6.2 <pip>
docutils 0.14 py36hb0f60f5_0
entrypoints 0.2.3 py36h1aec115_2
environment-kernels 1.1.1 <pip>
et_xmlfile 1.0.1 py36hd6bccc3_0
expat 2.2.5 he0dffb1_0
fastai 1.0.52 1 fastai
fastcache 1.0.2 py36h14c3975_2
fastprogress 0.1.21 py_0 fastai
filelock 3.0.4 py36_0
flask 1.0.2 py36_1
flask-cors 3.0.4 py36_0
fontconfig 2.13.0 h9420a91_0
freetype 2.9.1 h8a8886c_1
fribidi 1.0.5 h7b6447c_0
get_terminal_size 1.0.0 haa9412d_0
gevent 1.3.0 py36h14c3975_0
glib 2.56.1 h000015b_0
glob2 0.6 py36he249c77_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py36hc8893dd_2
graphite2 1.3.11 h16798f4_2
graphviz 2.40.1 h21bd128_2
greenlet 0.4.13 py36h14c3975_0
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.8.0 py36h989c5e5_3
harfbuzz 1.8.4 hec2c2bc_0
hdf5 1.10.2 hba1933b_1
hdijupyterutils 0.12.7 <pip>
heapdict 1.0.0 py36_2
html5lib 1.0.1 py36h2f9c1c0_0
icu 58.2 h9c2bf20_1
idna 2.6 py36h82fb2a8_1
imageio 2.3.0 py36_0
imagesize 1.0.0 py36_0
intel-openmp 2018.0.0 8
ipykernel 4.8.2 py36_0
ipyparallel 6.2.2 <pip>
ipython 6.4.0 py36_0
ipython_genutils 0.2.0 py36hb52b0d5_0
ipywidgets 7.2.1 py36_0
ipywidgets 7.4.0 <pip>
isort 4.3.4 py36_0
itsdangerous 0.24 py36h93cc618_1
jbig 2.1 hdba287a_0
jdcal 1.4 py36_0
jedi 0.12.0 py36_1
jinja2 2.10 py36ha16c418_0
jmespath 0.9.4 py_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py36h006f8b5_0
jupyter 1.0.0 py36_4
jupyter_client 5.2.3 py36_0
jupyter_console 5.2.0 py36he59e554_1
jupyter_core 4.4.0 py36h7c827e3_0
jupyterlab 0.32.1 py36_0
jupyterlab_launcher 0.10.5 py36_0
kiwisolver 1.0.1 py36h764f252_0
krb5 1.14.2 hcdc1b81_6
lazy-object-proxy 1.3.1 py36h10fcdad_0
libcurl 7.60.0 h1ad7b7a_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 8.2.0 hdf63c60_1
libgfortran 3.0.0 1 conda-forge
libgfortran-ng 7.2.0 hdf63c60_3
libpng 1.6.37 hbc83047_0
libprotobuf 3.5.2 hd28b015_1 conda-forge
libsodium 1.0.16 h1bed415_0
libssh2 1.8.0 h9cfc8f7_4
libstdcxx-ng 8.2.0 hdf63c60_1
libtiff 4.0.9 he85c1e1_1
libtool 2.4.6 h544aabb_3
libuuid 1.0.3 h1bed415_2
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
libxslt 1.1.32 h1312cb7_0
llvmlite 0.23.1 py36hdbcaa40_0
locket 0.2.0 py36h787c0ad_1
lxml 4.2.1 py36h23eabaa_0
lzo 2.10 h49e0be7_2
markupsafe 1.0 py36hd9260cd_1
matplotlib 2.2.2 <pip>
matplotlib 3.0.3 py36h5429711_0
mccabe 0.6.1 py36h5ad9710_1
mistune 0.8.3 py36h14c3975_1
mkl 2018.0.3 1
mkl-service 1.1.2 py36h17a0993_4
mkl_fft 1.0.6 py36h7dd41cf_0
mkl_random 1.0.1 py36h629b387_0
mock 3.0.5 <pip>
more-itertools 4.1.0 py36_0
mpc 1.0.3 hec55b23_5
mpfr 3.1.5 h11a74b3_2
mpi 1.0 openmpi conda-forge
mpmath 1.0.0 py36hfeacd6b_2
msgpack 0.6.0 <pip>
msgpack-numpy 0.4.3.2 py36_0
msgpack-python 0.5.6 py36h6bb024c_0
multipledispatch 0.5.0 py36_0
murmurhash 1.0.2 py36he6710b0_0
nb_conda 2.2.1 py36_2 conda-forge
nb_conda_kernels 2.2.1 py36_0 conda-forge
nbconvert 5.4.1 py36_3
nbformat 4.4.0 py36h31c9010_0
ncurses 6.1 hf484d3e_0
networkx 2.1 py36_0
ninja 1.8.2 py36h6bb024c_1
nltk 3.3.0 py36_0
nose 1.3.7 py36hcdf7029_2
notebook 5.5.0 py36_0
numba 0.38.0 py36h637b7d7_0
numexpr 2.6.5 py36h7bf3b9c_0
numpy 1.15.4 py36h1d66e8a_0
numpy 1.15.4 <pip>
numpy-base 1.15.4 py36h81de0dd_0
numpydoc 0.8.0 py36_0
nvidia-ml-py3 7.352.0 py_0 fastai
odo 0.5.1 py36h90ed295_0
olefile 0.45.1 py36_0
onnx 1.4.1 <pip>
openmpi 3.1.0 h26a2512_3 conda-forge
openpyxl 2.5.3 py36_0
openssl 1.0.2r h7b6447c_0
packaging 17.1 py36_0
pandas 0.24.2 <pip>
pandas 0.23.0 py36h637b7d7_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py36ha6701b7_1
pango 1.42.3 h8589676_0
paramiko 2.4.2 <pip>
parso 0.2.0 py36_0
partd 0.3.8 py36h36fd896_0
patchelf 0.9 hf79760b_2
path.py 11.0.1 py36_0
pathlib2 2.3.2 py36_0
patsy 0.5.0 py36_0
pcre 8.42 h439df22_0
pep8 1.7.1 py36_0
pexpect 4.5.0 py36_0
pickleshare 0.7.4 py36h63277f8_0
pillow 5.2.0 py36heded4f4_0
pip 10.0.1 py36_0
pixman 0.34.0 hceecf20_3
pkginfo 1.4.2 py36_1
plac 0.9.6 py36_0
plotly 2.7.0 <pip>
pluggy 0.6.0 py36hb689045_0
ply 3.11 py36_0
preshed 2.0.1 py36he6710b0_0
prompt_toolkit 1.0.15 py36h17d85b1_0
protobuf 3.5.2 py36hd28b015_0 conda-forge
protobuf3-to-dict 0.1.5 <pip>
psutil 5.4.5 py36h14c3975_0
psycopg2 2.7.5 <pip>
ptyprocess 0.5.2 py36h69acd42_0
py 1.5.3 py36_0
py4j 0.10.7 <pip>
pyasn1 0.4.5 <pip>
pycodestyle 2.4.0 py36_0
pycosat 0.6.3 py36h0a5515d_0
pycparser 2.18 py36hf9f622e_1
pycrypto 2.6.1 py36h14c3975_8
pycurl 7.43.0.1 py36hb7f436b_0
pyflakes 1.6.0 py36h7bd6a15_0
pygal 2.4.0 <pip>
pygments 2.2.0 py36h0d3125c_0
pykerberos 1.2.1 py36h14c3975_0
pylint 1.8.4 py36_0
PyNaCl 1.3.0 <pip>
pyodbc 4.0.23 py36hf484d3e_0
pyopenssl 18.0.0 py36_0
pyparsing 2.2.0 py36hee85983_1
pyqt 5.9.2 py36h751905a_0
pysocks 1.6.8 py36_0
pyspark 2.3.2 <pip>
pytables 3.4.3 py36h02b9ad4_2
pytest 3.5.1 py36_0
pytest-arraydiff 0.2 py36_0
pytest-astropy 0.3.0 py36_0
pytest-doctestplus 0.1.3 py36_0
pytest-openfiles 0.3.0 py36_0
pytest-remotedata 0.2.1 py36_0
python 3.6.5 hc3d631a_2
python-dateutil 2.7.3 py36_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
pytz 2018.4 py36_0
pywavelets 0.5.2 py36he602eb0_0
pyyaml 3.12 py36hafb9ca4_1
pyzmq 17.0.0 py36h14c3975_0
qt 5.9.6 h52aff34_0
qtawesome 0.4.4 py36h609ed8c_0
qtconsole 4.3.1 py36h8f73b5b_0
qtpy 1.4.1 py36_0
readline 7.0 ha6073c6_4
regex 2018.01.10 py36h14c3975_1000 fastai
requests 2.20.0 py36_1000 conda-forge
requests-kerberos 0.12.0 <pip>
rope 0.10.7 py36h147e2ec_0
ruamel_yaml 0.15.35 py36h14c3975_1
s3fs 0.1.5 py36_0
s3transfer 0.2.0 <pip>
s3transfer 0.2.0 py36_0
sagemaker 1.20.1 <pip>
sagemaker-pyspark 1.2.4 <pip>
scikit-image 0.13.1 py36h14c3975_1
scikit-learn 0.19.1 py36h7aa7ec6_0
scikit-learn 0.20.3 <pip>
scipy 1.1.0 py36hfc37229_0
seaborn 0.8.1 py36hfad7ec4_0
send2trash 1.5.0 py36_0
setuptools 39.1.0 py36_0
simplegeneric 0.8.1 py36_2
singledispatch 3.4.0.3 py36h7a266c3_0
sip 4.19.8 py36hf484d3e_0
six 1.11.0 py36h372c433_1
snappy 1.1.7 hbae5bb6_3
snowballstemmer 1.2.1 py36h6febd40_0
sortedcollections 0.6.1 py36_0
sortedcontainers 1.5.10 py36_0
spacy 2.0.18 py36hf484d3e_1000 fastai
sparkmagic 0.12.5 <pip>
sphinx 1.7.4 py36_0
sphinxcontrib 1.0 py36h6d0f590_1
sphinxcontrib-websupport 1.0.1 py36hb5cb234_1
spyder 3.2.8 py36_0
SQLAlchemy 1.2.11 <pip>
sqlalchemy 1.2.7 py36h6b74fdf_0
sqlite 3.23.1 he433501_0
statsmodels 0.9.0 py36h3010b51_0
sympy 1.1.1 py36hc6d1c1c_0
tblib 1.3.2 py36h34cf8b6_0
terminado 0.8.1 py36_1
testpath 0.3.1 py36h8cadb63_0
texttable 0.9.1 <pip>
thinc 6.12.1 py36h637b7d7_1000 fastai
tk 8.6.8 hbc83047_0
toolz 0.9.0 py36_0
torchvision 0.2.2 py_3 pytorch
tornado 5.0.2 py36_0
tqdm 4.31.1 py36_1
traitlets 4.3.2 py36h674d592_0
typing 3.6.4 py36_0
typing-extensions 3.7.2 <pip>
ujson 1.35 py36h14c3975_0
unicodecsv 0.14.1 py36ha668878_0
unixodbc 2.3.6 h1bed415_0
urllib3 1.23 py36_0
wcwidth 0.1.7 py36hdf4376a_0
webencodings 0.5.1 py36h800622e_1
websocket-client 0.56.0 <pip>
werkzeug 0.14.1 py36_0
wheel 0.31.1 py36_0
widgetsnbextension 3.2.1 py36_0
widgetsnbextension 3.4.2 <pip>
wrapt 1.10.11 py36h28b7045_0
xlrd 1.1.0 py36h1db9f0c_1
xlsxwriter 1.0.4 py36_0
xlwt 1.3.0 py36h7b00a1f_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.2.5 h439df22_0
zict 0.1.3 py36h3a3bf81_0
zlib 1.2.11 ha838bed_2
I was told I do have pytorch installed but my script keeps giving me this error:
$ cat nohup.out
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
does that mean that I need to install it as pytroch and not torch? Is this not weird?
Note I am running this on an AWS instance p3.2xlarge. This keeps happening when I log out and then go back in that my torch package gets missing...?!?! :/
original post: https://discuss.pytorch.org/t/torchvision-installed-but-not-torch/51758
The issue persists even if I open just a python interactive and try to import it:
(pytorch_p36) ubuntu#ip-123-12-21-123:~$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
>>>
It also happens running the script directly:
(pytorch_p36) ubuntu#ip-123-12-21-123:~/project/folder$ python high_performing_data_point_models_cifar10.py
Traceback (most recent call last):
File "high_performing_data_point_models_cifar10.py", line 5, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
I can't import torchvision either!
$ python
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torchvision
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torchvision'
>>>
Your conda list command shows that it was run from the environment called automl:
# packages in environment at /home/ubuntu/anaconda3/envs/automl:
However, when you show the commands that you are trying to run, you are doing so from the (pytorch_p36) environment.
You should run your conda install command while inside this pytorch_p36 environment.
Go to the official website and try installing it as said over there. As I just followed the official documentation and it is working fine.
Here is the step though.
conda install pytorch-cpu torchvision-cpu -c pytorch
You can follow the official documentation.
link
It is really easy to miss smth while working with several machines and several conda/python environments. I think it is a good idea to start from the very beginning. Here is installation process that works fine for me.
0. Connect to your aws-instance using SSH or PuTTY.
1. Create conda env named pytorch_p36 with python 3.6 on your aws machine:
user#aws-instance:~$ conda create -n pytorch_p36 python=3.6
2. Activate it:
user#aws-instance:~$ conda activate pytorch_p36
Or with (for older conda versions):
user#aws-instance:~$ source activate pytorch_p36
Now you should see (pytorch_p36) before your shell prompt:
(pytorch_p36) user#aws-instance:~$
3. Go to PyTorch website and choose appropriate installation command via conda. Run it in your shell:
(pytorch_p36) user#aws-instance:~$ conda install pytorch torchvision cudatoolkit=10.0 -c pytorch
4. Verify the installation:
(pytorch_p36) user#aws-instance:~$ conda list
# packages in environment at /home/user/anaconda3/envs/pytorch_p36:
#
# Name Version Build Channel
...
cudatoolkit 10.0.130 0
...
python 3.6.9 h265db76_0
pytorch 1.1.0 py3.6_cuda10.0.130_cudnn7.5.1_0 pytorch
...
torchvision 0.3.0 py36_cu10.0.130_1 pytorch
...
5. Verify the running with nohup:
(pytorch_p36) user#aws-instance:~$ nohup python -c "import torch; import torchvision; print('PyTorch is fine!')" && cat nohup.out
nohup: ignoring input and appending output to 'nohup.out'
PyTorch is fine!
AWS Machine Learning AMI already has pytorch v1.1.0 and torchvision v0.2.2 installed in the predefined pytorch_p36 virtual environment. So you are not required to create new venv and install pytorch each time you log in. All you need is to run :
$ source activate pytorch_p36
(without dollar sign, obviously). Then, if you run pip show torch you will see that it is already there.
I created anaconda environment with Python=3.7 and have trouble with the error of _ssl and DLL. When I tried to get back to my base environment, I have trouble getting the background processes to complete as shown in the figure below, this goes on forever.
The error:
C:\Users\abhil\AppData\Local\Continuum\anaconda3\envs\HeisenbergPy37\python.exe "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\pydev\pydevconsole.py" --mode=client --port=63950
Traceback (most recent call last):
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\pydev\pydevconsole.py", line 5, in <module>
from _pydev_comm.rpc import make_rpc_client, start_rpc_server, start_rpc_server_and_make_client
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\pydev\_pydev_comm\rpc.py", line 4, in <module>
from _pydev_comm.server import TSingleThreadedServer
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\pydev\_pydev_comm\server.py", line 4, in <module>
from _shaded_thriftpy.server import TServer
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\third_party\thriftpy\_shaded_thriftpy\server.py", line 9, in <module>
from _shaded_thriftpy.transport import (
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\third_party\thriftpy\_shaded_thriftpy\transport\__init__.py", line 57, in <module>
from .sslsocket import TSSLSocket, TSSLServerSocket # noqa
File "C:\Program Files\JetBrains\PyCharm Community Edition 2018.1.1\helpers\third_party\thriftpy\_shaded_thriftpy\transport\sslsocket.py", line 7, in <module>
import ssl
File "C:\Users\abhil\AppData\Local\Continuum\anaconda3\envs\HeisenbergPy37\lib\ssl.py", line 98, in <module>
import _ssl # if we can't import it, let the error propagate
ImportError: DLL load failed: The specified module could not be found.
Process finished with exit code 1
All packages installed with
conda create -n anaconda python=3.7
C:\WINDOWS\system32>conda list
# packages in environment at C:\Users\abhil\AppData\Local\Continuum\anaconda3\envs\HeisenbergPy37:
#
# Name Version Build Channel
alabaster 0.7.12 py37_0
anaconda 2018.12 py37_0
anaconda-client 1.7.2 py37_0
anaconda-project 0.8.2 py37_0
asn1crypto 0.24.0 py37_0
astroid 2.1.0 py37_0
astropy 3.1 py37he774522_0
atomicwrites 1.2.1 py37_0
attrs 18.2.0 py37h28b3542_0
babel 2.6.0 py37_0
backcall 0.1.0 py37_0
backports 1.0 py37_1
backports.os 0.1.1 py37_0
backports.shutil_get_terminal_size 1.0.0 py37_2
beautifulsoup4 4.6.3 py37_0
bitarray 0.8.3 py37hfa6e2cd_0
bkcharts 0.2 py37_0
blas 1.0 mkl
blaze 0.11.3 py37_0
bleach 3.0.2 py37_0
blosc 1.14.4 he51fdeb_0
bokeh 1.0.2 py37_0
boto 2.49.0 py37_0
bottleneck 1.2.1 py37h452e1ab_1
bzip2 1.0.6 hfa6e2cd_5
ca-certificates 2018.03.07 0
certifi 2018.11.29 py37_0
cffi 1.11.5 py37h74b6da3_1
chardet 3.0.4 py37_1
click 7.0 py37_0
cloudpickle 0.6.1 py37_0
clyent 1.2.2 py37_1
colorama 0.4.1 py37_0
comtypes 1.1.7 py37_0
console_shortcut 0.1.1 3
contextlib2 0.5.5 py37_0
cryptography 2.4.2 py37h7a1dbc1_0
curl 7.63.0 h2a8f88b_1000
cycler 0.10.0 py37_0
cython 0.29.2 py37ha925a31_0
cytoolz 0.9.0.1 py37hfa6e2cd_1
dask 1.0.0 py37_0
dask-core 1.0.0 py37_0
datashape 0.5.4 py37_1
decorator 4.3.0 py37_0
defusedxml 0.5.0 py37_1
distributed 1.25.1 py37_0
docutils 0.14 py37_0
entrypoints 0.2.3 py37_2
et_xmlfile 1.0.1 py37_0
fastcache 1.0.2 py37hfa6e2cd_2
filelock 3.0.10 py37_0
flask 1.0.2 py37_1
flask-cors 3.0.7 py37_0
freetype 2.9.1 ha9979f8_1
get_terminal_size 1.0.0 h38e98db_0
gevent 1.3.7 py37he774522_1
glob2 0.6 py37_1
greenlet 0.4.15 py37hfa6e2cd_0
h5py 2.8.0 py37h3bdd7fb_2
hdf5 1.10.2 hac2f561_1
heapdict 1.0.0 py37_2
html5lib 1.0.1 py37_0
icc_rt 2019.0.0 h0cc432a_1
icu 58.2 ha66f8fd_1
idna 2.8 py37_0
imageio 2.4.1 py37_0
imagesize 1.1.0 py37_0
importlib_metadata 0.6 py37_0
intel-openmp 2019.1 144
ipykernel 5.1.0 py37h39e3cac_0
ipython 7.2.0 py37h39e3cac_0
ipython_genutils 0.2.0 py37_0
ipywidgets 7.4.2 py37_0
isort 4.3.4 py37_0
itsdangerous 1.1.0 py37_0
jdcal 1.4 py37_0
jedi 0.13.2 py37_0
jinja2 2.10 py37_0
jpeg 9b hb83a4c4_2
jsonschema 2.6.0 py37_0
jupyter 1.0.0 py37_7
jupyter_client 5.2.4 py37_0
jupyter_console 6.0.0 py37_0
jupyter_core 4.4.0 py37_0
jupyterlab 0.35.3 py37_0
jupyterlab_server 0.2.0 py37_0
keyring 17.0.0 py37_0
kiwisolver 1.0.1 py37h6538335_0
krb5 1.16.1 hc04afaa_7
lazy-object-proxy 1.3.1 py37hfa6e2cd_2
libarchive 3.3.3 h0643e63_5
libcurl 7.63.0 h2a8f88b_1000
libiconv 1.15 h1df5818_7
libpng 1.6.35 h2a8f88b_0
libsodium 1.0.16 h9d3ae62_0
libssh2 1.8.0 h7a1dbc1_4
libtiff 4.0.9 h36446d0_2
libxml2 2.9.8 hadb2253_1
libxslt 1.1.32 hf6f1972_0
llvmlite 0.26.0 py37ha925a31_0
locket 0.2.0 py37_1
lxml 4.2.5 py37hef2cd61_0
lz4-c 1.8.1.2 h2fa13f4_0
lzo 2.10 h6df0209_2
m2w64-gcc-libgfortran 5.3.0 6
m2w64-gcc-libs 5.3.0 7
m2w64-gcc-libs-core 5.3.0 7
m2w64-gmp 6.1.0 2
m2w64-libwinpthread-git 5.0.0.4634.697f757 2
markupsafe 1.1.0 py37he774522_0
matplotlib 3.0.2 py37hc8f65d3_0
mccabe 0.6.1 py37_1
menuinst 1.4.14 py37hfa6e2cd_0
mistune 0.8.4 py37he774522_0
mkl 2019.1 144
mkl-service 1.1.2 py37hb782905_5
mkl_fft 1.0.6 py37h6288b17_0
mkl_random 1.0.2 py37h343c172_0
more-itertools 4.3.0 py37_0
mpmath 1.1.0 py37_0
msgpack-python 0.5.6 py37he980bc4_1
msys2-conda-epoch 20160418 1
multipledispatch 0.6.0 py37_0
nbconvert 5.4.0 py37_1
nbformat 4.4.0 py37_0
networkx 2.2 py37_1
nltk 3.4 py37_1
nose 1.3.7 py37_2
notebook 5.7.4 py37_0
numba 0.41.0 py37hf9181ef_0
numexpr 2.6.8 py37hdce8814_0
numpy 1.15.4 py37h19fb1c0_0
numpy-base 1.15.4 py37hc3f5095_0
numpydoc 0.8.0 py37_0
odo 0.5.1 py37_0
olefile 0.46 py37_0
openpyxl 2.5.12 py37_0
openssl 1.1.1a he774522_0
packaging 18.0 py37_0
pandas 0.23.4 py37h830ac7b_0
pandoc 1.19.2.1 hb2460c7_1
pandocfilters 1.4.2 py37_1
parso 0.3.1 py37_0
partd 0.3.9 py37_0
path.py 11.5.0 py37_0
pathlib2 2.3.3 py37_0
patsy 0.5.1 py37_0
pep8 1.7.1 py37_0
pickleshare 0.7.5 py37_0
pillow 5.3.0 py37hdc69c19_0
pip 18.1 py37_0
pkginfo 1.4.2 py37_1
pluggy 0.8.0 py37_0
ply 3.11 py37_0
prometheus_client 0.5.0 py37_0
prompt_toolkit 2.0.7 py37_0
psutil 5.4.8 py37he774522_0
py 1.7.0 py37_0
pycodestyle 2.4.0 py37_0
pycosat 0.6.3 py37hfa6e2cd_0
pycparser 2.19 py37_0
pycrypto 2.6.1 py37hfa6e2cd_9
pycurl 7.43.0.2 py37h7a1dbc1_0
pyflakes 2.0.0 py37_0
pygments 2.3.1 py37_0
pylint 2.2.2 py37_0
pyodbc 4.0.25 py37ha925a31_0
pyopenssl 18.0.0 py37_0
pyparsing 2.3.0 py37_0
pyqt 5.9.2 py37h6538335_2
pysocks 1.6.8 py37_0
pytables 3.4.4 py37he6f6034_0
pytest 4.0.2 py37_0
pytest-arraydiff 0.3 py37h39e3cac_0
pytest-astropy 0.5.0 py37_0
pytest-doctestplus 0.2.0 py37_0
pytest-openfiles 0.3.1 py37_0
pytest-remotedata 0.3.1 py37_0
python 3.7.1 h8c8aaf0_6
python-dateutil 2.7.5 py37_0
python-libarchive-c 2.8 py37_6
pytz 2018.7 py37_0
pywavelets 1.0.1 py37h8c2d366_0
pywin32 223 py37hfa6e2cd_1
pywinpty 0.5.5 py37_1000
pyyaml 3.13 py37hfa6e2cd_0
pyzmq 17.1.2 py37hfa6e2cd_0
qt 5.9.7 vc14h73c81de_0
qtawesome 0.5.3 py37_0
qtconsole 4.4.3 py37_0
qtpy 1.5.2 py37_0
requests 2.21.0 py37_0
rope 0.11.0 py37_0
ruamel_yaml 0.15.46 py37hfa6e2cd_0
scikit-image 0.14.1 py37ha925a31_0
scikit-learn 0.20.1 py37h343c172_0
scipy 1.1.0 py37h29ff71c_2
seaborn 0.9.0 py37_0
send2trash 1.5.0 py37_0
setuptools 40.6.3 py37_0
simplegeneric 0.8.1 py37_2
singledispatch 3.4.0.3 py37_0
sip 4.19.8 py37h6538335_0
six 1.12.0 py37_0
snappy 1.1.7 h777316e_3
snowballstemmer 1.2.1 py37_0
sortedcollections 1.0.1 py37_0
sortedcontainers 2.1.0 py37_0
sphinx 1.8.2 py37_0
sphinxcontrib 1.0 py37_1
sphinxcontrib-websupport 1.1.0 py37_1
spyder 3.3.2 py37_0
spyder-kernels 0.3.0 py37_0
sqlalchemy 1.2.15 py37he774522_0
sqlite 3.26.0 he774522_0
statsmodels 0.9.0 py37h452e1ab_0
sympy 1.3 py37_0
tblib 1.3.2 py37_0
terminado 0.8.1 py37_1
testpath 0.4.2 py37_0
tk 8.6.8 hfa6e2cd_0
toolz 0.9.0 py37_0
tornado 5.1.1 py37hfa6e2cd_0
tqdm 4.28.1 py37h28b3542_0
traitlets 4.3.2 py37_0
unicodecsv 0.14.1 py37_0
urllib3 1.24.1 py37_0
vc 14.1 h0510ff6_4
vs2015_runtime 14.15.26706 h3a45250_0
wcwidth 0.1.7 py37_0
webencodings 0.5.1 py37_1
werkzeug 0.14.1 py37_0
wheel 0.32.3 py37_0
widgetsnbextension 3.4.2 py37_0
win_inet_pton 1.0.1 py37_1
win_unicode_console 0.5 py37_0
wincertstore 0.2 py37_0
winpty 0.4.3 4
wrapt 1.10.11 py37hfa6e2cd_2
xlrd 1.2.0 py37_0
xlsxwriter 1.1.2 py37_0
xlwings 0.15.1 py37_0
xlwt 1.3.0 py37_0
xz 5.2.4 h2fa13f4_4
yaml 0.1.7 hc54c509_2
zeromq 4.2.5 he025d50_1
zict 0.1.3 py37_0
zlib 1.2.11 h62dcd97_3
zstd 1.3.7 h508b16e_0
I have answerd this here, to my understanding this error is caused by the missing/misplacement of libcrypto file in anaconda3/DLLs folder:
From anaconda3\Library\bin copy below files and paste them in anaconda3/DLLs:
- libcrypto-1_1-x64.dll
- libssl-1_1-x64.dll
This can be fixed by manually adding in the PATH variables in PyCharm to the console.
Open your Anaconda cmd
Activate your Conda environment
Get the full PATH value by typing echo %PATH%
if you are on Windows 7/can't copy the output, cd the cmd to your desktop and type echo %PATH% > path_val.txt
this will create a txt file so you can copy/paste the values easier.
In PyCharm, go to Settings -> Build, Execution, Deployment -> Console -> Python Console -> click the folder on the right of Environment variables.
Click the plus button to add a new Environment Variable
The name should be PATH
The value is the entire output from the echo %PATH% command above.
Click OK, then apply.
You can try to set system environment variables, like "C:\Python\Anaconda3;C:\Python\Anaconda3\Scripts;
C:\Python\Anaconda3\Library\bin;".
Only "C:\Python\Anaconda3\Scripts;" is not enough, and it works to me.
I had the same issue on Windows 10 Pro x64 and PyCharm CE 2018.3
A solution that worked for me: Re-install Anaconda3 for all users (with Admin priveledges) and choose "add python.exe to Path" (even though it is nor recommended). Then PyCharm connected successfully without this error.
A work around that also worked: Use another environment (pyenv or conda). It worked for me when I faced the same issue on
Some other suggestions I saw on Internet:
run Anaconda3\Scripts>conda update --all
but this threw the HTTP connection error rises, which might be related.
if running under a virtual env (i.e. your have issued the command: conda activate <env_name>)
The files (libcrypto-1_1-x64.dll and libssl-1_1-x64.dll) to copy are found under:
C:\USER<username>.conda\envs<env_name>\Library\bin
They have to be copied into:
C:\USER<username>.conda\envs<env_name>\DLLs
None of the above worked for me, but instead it was a problem with OpenSSL on Windows 10 Enterprise. I fixed it by downloading and installing OpenSSL from here: https://slproweb.com/products/Win32OpenSSL.html
Thanks for the answer!
This works definately.
I have answerd this here, to my understanding this error is caused by the missing/misplacement of libcrypto file in anaconda3/DLLs folder:
From anaconda3\Library\bin copy below files and paste them in anaconda3/DLLs:
libcrypto-1_1-x64.dll
libssl-1_1-x64.dll
In my case the error was a result of spinning pipenv from non-base conda env. Setting variable in PyCharm or moving files in base anaconda structure didn't help, because PyCharm looked for these DLLs in my conda env in the middle.
So I had to move files from base conda structure:
C:\Users<username>\AppData\Local\Continuum\anaconda3\Library\bin
To my conda env from which I spinned the final Pipenv I use in PyCharm:
C:\Users<username>\AppData\Local\Continuum\anaconda3\envs<conda_env_name>\DLLs
The files I had to move to have IPython work correctly:
libcrypto-1_1-x64.dll
libssl-1_1-x64.dll
sqlite3.dll
sqlite3.exe
The reason is that not all libraries/binaries are propagated to the virtual envs. Virtual envs are able to find these files in "root Python". Venvs stay lighter thanks to not copying them.
However, PyCharm gets confused in a setup such as mine, and treats the conda env in the middle as a "root Python", and can't find the libraries/bins. This problem can potentially make itself apparent with other dlls or bins.
It may be worth to move all missing files this way.
another solution: edit python3.7's site.py(in Lib dir):
from os.path import join, abspath, dirname
base_path = dirname(dirname(abspath(__file__)))
os.environ['PATH'] = '%s%s' % (
os.environ['PATH'],
join(base_path, 'Library', 'bin'),
)
I had the same issue and my main IDE is Sublime Text 4.
Environment: Windows 10 with Anaconda
What I needed: from cryptography.fernet import Fernet
The marked solution was unachievable for me and the following answer solved it right away:
pip install --upgrade pip
pip uninstall pyopenssl cryptography
pip install pyopenssl cryptography
Validate with:
python -v -c "from OpenSSL import SSL"
Source: https://groups.google.com/g/qatrack/c/24RB6hesOko?pli=1
Incase someone is searching for Eclipse PyDev, It's same as Rob's answer.
No. 4 can be found at Window -> Preferences -> PyDev -> Interpreters -> Python Interpreters -> Environment -> Click New and add PATH
I have updated to Python 3.8 (with miniconda) and it did not work with Pycharm generating "anaconda-environment-import-ssl-dll-load-fail-error". I have tried several proposals here and tweaked env variables in different way w/o much success.
Finally I have updated to Pycharm 2020.2.3 and it worked like a charm, no tweaks ..
I hope this post brings you well. I have environment issues that I would love to get assistance on.
I installed geopandas with command:
conda install -c conda-forge geopandas
Then typed jupyter-notebook and got a nasty error, here is what it reads:
Traceback (most recent call last):
File "/home/mathlizard/anaconda3/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 10, in
import sqlite3
File "/home/mathlizard/anaconda3/lib/python3.6/sqlite3/init.py", line 23, in
from sqlite3.dbapi2 import *
File "/home/mathlizard/anaconda3/lib/python3.6/sqlite3/dbapi2.py", line 27, in
from _sqlite3 import *
ImportError: libncurses.so.6: cannot open shared object file: No such file or directory
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mathlizard/anaconda3/bin/jupyter-notebook", line 7, in
from notebook.notebookapp import main
File "/home/mathlizard/anaconda3/lib/python3.6/site-packages/notebook/notebookapp.py", line 84, in
from .services.sessions.sessionmanager import SessionManager
File "/home/mathlizard/anaconda3/lib/python3.6/site-packages/notebook/services/sessions/sessionmanager.py", line 13, in
from pysqlite2 import dbapi2 as sqlite3
ModuleNotFoundError: No module named 'pysqlite2'
My attempts
wrote the following commands:
pip install pysqlite2
pip install libsqlite3-dev
sudo apt-get install libsqlite3-dev
pip install pysqlite
conda install pysqlite2
conda install pysqlite
created and edited .condarc file in root directory with following lines of code:
channels:
- conda-forge
- defaults
UPDATES
installed pysqlite with conda install -c kbchoi pysqlite and now the error after typing jupyter-notebook is the following:
Traceback (most recent call last):
File "/home/mathlizard/anaconda3/bin/jupyter-notebook", line 7, in
from notebook.notebookapp import main
File "/home/mathlizard/anaconda3/lib/python2.7/site-packages/notebook/notebookapp.py", line 85, in
from .services.sessions.sessionmanager import SessionManager
File "/home/mathlizard/anaconda3/lib/python2.7/site-packages/notebook/services/sessions/sessionmanager.py", line 13, in
from pysqlite2 import dbapi2 as sqlite3
File "/home/mathlizard/anaconda3/lib/python2.7/site-packages/pysqlite2/dbapi2.py", line 28, in
from pysqlite2._sqlite import *
ImportError: libncurses.so.6: cannot open shared object file: No such file or directory
Background info:
which python outputs this: /home/mathlizard/anaconda3/bin/python
which jupyter outputs this: /home/mathlizard/anaconda3/bin/jupyter
my .bashrc file contains these lines of code: export PATH="/home/mathlizard/anaconda3/bin:$PATH"
and export PATH="/home/mathlizard/anaconda2/bin:$PATH"
conda list generates:
Name Version Build Channel
_ipyw_jlab_nb_ext_conf 0.1.0 py27h08a7f0c_0
alabaster 0.7.10 py27he5a193a_0
anaconda 5.2.0 py27_3
anaconda-client 1.6.14 py27_0
anaconda-navigator 1.8.7 py27_0
anaconda-project 0.8.2 py27h236b58a_0
asn1crypto 0.24.0 py27_0
astroid 1.6.3 py27_0
astropy 2.0.6 py27h3010b51_1
attrs 18.1.0 py27_0
babel 2.5.3 py27_0
backports 1.0 py27h63c9359_1
backports.functools_lru_cache 1.5 py27_1
backports.shutil_get_terminal_size 1.0.0 py27h5bc021e_2
backports_abc 0.5 py27h7b3c97b_0
beautifulsoup4 4.6.0 py27h3f86ba9_1
bitarray 0.8.1 py27h14c3975_1
bkcharts 0.2 py27h241ae91_0
blas 1.0 mkl
blaze 0.11.3 py27h5f341da_0
bleach 2.1.3 py27_0
blosc 1.14.3 hdbcaa40_0
bokeh 0.12.16 py27_0
boto 2.48.0 py27h9556ac2_1
bottleneck 1.2.1 py27h21b16a3_0
bzip2 1.0.6 h14c3975_5
ca-certificates 2018.03.07 0
cairo 1.14.12 h7636065_2
cdecimal 2.3 py27h14c3975_3
certifi 2018.4.16 py27_0
cffi 1.11.5 py27h9745a5d_0
chardet 3.0.4 py27hfa10054_1
click 6.7 py27h4225b90_0
cloudpickle 0.5.3 py27_0
clyent 1.2.2 py27h7276e6c_1
colorama 0.3.9 py27h5cde069_0
conda 4.5.11 py27_0
conda-build 3.10.5 py27_0
conda-env 2.6.0 h36134e3_1
conda-verify 2.0.0 py27hf052a9d_0
configparser 3.5.0 py27h5117587_0
contextlib2 0.5.5 py27hbf4c468_0
cryptography 2.2.2 py27h14c3975_0
curl 7.60.0 h84994c4_0
cycler 0.10.0 py27hc7354d3_0
cython 0.28.2 py27h14c3975_0
cytoolz 0.9.0.1 py27h14c3975_0
dask 0.17.5 py27_0
dask-core 0.17.5 py27_0
datashape 0.5.4 py27hf507385_0
dbus 1.13.2 h714fa37_1
decorator 4.3.0 py27_0
distributed 1.21.8 py27_0
docutils 0.14 py27hae222c1_0
entrypoints 0.2.3 py27h502b47d_2
enum34 1.1.6 py27h99a27e9_1
et_xmlfile 1.0.1 py27h75840f5_0
expat 2.2.5 he0dffb1_0
fastcache 1.0.2 py27h14c3975_2
filelock 3.0.4 py27_0
flask 1.0.2 py27_1
flask-cors 3.0.4 py27_0
fontconfig 2.12.6 h49f89f6_0
freetype 2.8 hab7d2ae_1
funcsigs 1.0.2 py27h83f16ab_0
functools32 3.2.3.2 py27h4ead58f_1
futures 3.2.0 py27h7b459c0_0
get_terminal_size 1.0.0 haa9412d_0
gevent 1.3.0 py27h14c3975_0
glib 2.56.1 h000015b_0
glob2 0.6 py27hcea9cbd_0
gmp 6.1.2 h6c8ec71_1
gmpy2 2.0.8 py27h4cf3fa8_2
graphite2 1.3.11 h16798f4_2
greenlet 0.4.13 py27h14c3975_0
grin 1.2.1 py27_4
gst-plugins-base 1.14.0 hbbd80ab_1
gstreamer 1.14.0 hb453b48_1
h5py 2.7.1 py27ha1f6525_2
harfbuzz 1.7.6 h5f0a787_1
hdf5 1.10.2 hba1933b_1
heapdict 1.0.0 py27_2
html5lib 1.0.1 py27h5233db4_0
icu 58.2 h9c2bf20_1
idna 2.6 py27h5722d68_1
imageio 2.3.0 py27_0
imagesize 1.0.0 py27_0
intel-openmp 2018.0.0 8
ipaddress 1.0.22 py27_0
ipykernel 4.8.2 py27_0
ipython 5.7.0 py27_0
ipython_genutils 0.2.0 py27h89fb69b_0
ipywidgets 7.2.1 py27_0
isort 4.3.4 py27_0
itsdangerous 0.24 py27hb8295c1_1
jbig 2.1 hdba287a_0
jdcal 1.4 py27_0
jedi 0.12.0 py27_1
jinja2 2.10 py27h4114e70_0
jpeg 9b h024ee3a_2
jsonschema 2.6.0 py27h7ed5aa4_0
jupyter 1.0.0 py27_4
jupyter_client 5.2.3 py27_0
jupyter_console 5.2.0 py27hc6bee7e_1
jupyter_core 4.4.0 py27h345911c_0
jupyterlab 0.32.1 py27_0
jupyterlab_launcher 0.10.5 py27_0
kiwisolver 1.0.1 py27hc15e7b5_0
lazy-object-proxy 1.3.1 py27h682c727_0
libcurl 7.60.0 h1ad7b7a_0
libedit 3.1.20170329 h6b74fdf_2
libffi 3.2.1 hd88cf55_4
libgcc-ng 7.2.0 hdf63c60_3
libgfortran-ng 7.2.0 hdf63c60_3
libpng 1.6.34 hb9fc6fc_0
libsodium 1.0.16 h1bed415_0
libssh2 1.8.0 h9cfc8f7_4
libstdcxx-ng 7.2.0 hdf63c60_3
libtiff 4.0.9 he85c1e1_1
libtool 2.4.6 h544aabb_3
libxcb 1.13 h1bed415_1
libxml2 2.9.8 h26e45fe_1
libxslt 1.1.32 h1312cb7_0
llvmlite 0.23.1 py27hdbcaa40_0
locket 0.2.0 py27h73929a2_1
lxml 4.2.1 py27h23eabaa_0
lzo 2.10 h49e0be7_2
markupsafe 1.0 py27h97b2822_1
matplotlib 2.2.2 py27h0e671d2_1
mccabe 0.6.1 py27h0e7c7be_1
mistune 0.8.3 py27h14c3975_1
mkl 2018.0.2 1
mkl-service 1.1.2 py27hb2d42c5_4
mkl_fft 1.0.1 py27h3010b51_0
mkl_random 1.0.1 py27h629b387_0
more-itertools 4.1.0 py27_0
mpc 1.0.3 hec55b23_5
mpfr 3.1.5 h11a74b3_2
mpmath 1.0.0 py27h9669132_2
msgpack-python 0.5.6 py27h6bb024c_0
multipledispatch 0.5.0 py27_0
navigator-updater 0.2.1 py27_0
nbconvert 5.3.1 py27he041f76_0
nbformat 4.4.0 py27hed7f2b2_0
ncurses 6.1 hf484d3e_0
networkx 2.1 py27_0
nltk 3.3.0 py27_0
nose 1.3.7 py27heec2199_2
notebook 5.5.0 py27_0
numba 0.38.0 py27h637b7d7_0
numexpr 2.6.5 py27h7bf3b9c_0
numpy 1.14.3 py27hcd700cb_1
numpy-base 1.14.3 py27h9be14a7_1
numpydoc 0.8.0 py27_0
odo 0.5.1 py27h9170de3_0
olefile 0.45.1 py27_0
openpyxl 2.5.3 py27_0
openssl 1.0.2o h20670df_0
packaging 17.1 py27_0
pandas 0.23.0 py27h637b7d7_0
pandoc 1.19.2.1 hea2e7c5_1
pandocfilters 1.4.2 py27h428e1e5_1
pango 1.41.0 hd475d92_0
parso 0.2.0 py27_0
partd 0.3.8 py27h4e55004_0
patchelf 0.9 hf79760b_2
path.py 11.0.1 py27_0
pathlib2 2.3.2 py27_0
patsy 0.5.0 py27_0
pcre 8.42 h439df22_0
pep8 1.7.1 py27_0
pexpect 4.5.0 py27_0
pickleshare 0.7.4 py27h09770e1_0
pillow 5.1.0 py27h3deb7b8_0
pip 10.0.1 py27_0
pixman 0.34.0 hceecf20_3
pkginfo 1.4.2 py27_1
pluggy 0.6.0 py27h1f4f128_0
ply 3.11 py27_0
prompt_toolkit 1.0.15 py27h1b593e1_0
psutil 5.4.5 py27h14c3975_0
ptyprocess 0.5.2 py27h4ccb14c_0
py 1.5.3 py27_0
pycairo 1.15.4 py27h1b9232e_1
pycodestyle 2.4.0 py27_0
pycosat 0.6.3 py27ha4109ae_0
pycparser 2.18 py27hefa08c5_1
pycrypto 2.6.1 py27h14c3975_8
pycurl 7.43.0.1 py27hb7f436b_0
pyflakes 1.6.0 py27h904a57d_0
pygments 2.2.0 py27h4a8b6f5_0
pylint 1.8.4 py27_0
pyodbc 4.0.23 py27hf484d3e_0
pyopenssl 18.0.0 py27_0
pyparsing 2.2.0 py27hf1513f8_1
pyqt 5.9.2 py27h751905a_0
pysocks 1.6.8 py27_0
pytables 3.4.3 py27h02b9ad4_2
pytest 3.5.1 py27_0
python 2.7.15 h1571d57_0
python-dateutil 2.7.3 py27_0
pytz 2018.4 py27_0
pywavelets 0.5.2 py27hecda097_0
pyyaml 3.12 py27h2d70dd7_1
pyzmq 17.0.0 py27h14c3975_1
qt 5.9.5 h7e424d6_0
qtawesome 0.4.4 py27hd7914c3_0
qtconsole 4.3.1 py27hc444b0d_0
qtpy 1.4.1 py27_0
readline 7.0 ha6073c6_4
requests 2.18.4 py27hc5b0589_1
rope 0.10.7 py27hfe459b0_0
ruamel_yaml 0.15.35 py27h14c3975_1
scandir 1.7 py27h14c3975_0
scikit-image 0.13.1 py27h14c3975_1
scikit-learn 0.19.1 py27h445a80a_0
scipy 1.1.0 py27hfc37229_0
seaborn 0.8.1 py27h633ea1e_0
send2trash 1.5.0 py27_0
setuptools 39.1.0 py27_0
simplegeneric 0.8.1 py27_2
singledispatch 3.4.0.3 py27h9bcb476_0
sip 4.19.8 py27hf484d3e_0
six 1.11.0 py27h5f960f1_1
snappy 1.1.7 hbae5bb6_3
snowballstemmer 1.2.1 py27h44e2768_0
sortedcollections 0.6.1 py27_0
sortedcontainers 1.5.10 py27_0
sphinx 1.7.4 py27_0
sphinxcontrib 1.0 py27h1512b58_1
sphinxcontrib-websupport 1.0.1 py27hf906f22_1
spyder 3.2.8 py27_0
sqlalchemy 1.2.7 py27h6b74fdf_0
sqlite 3.23.1 he433501_0
ssl_match_hostname 3.5.0.1 py27h4ec10b9_2
statsmodels 0.9.0 py27h3010b51_0
subprocess32 3.5.0 py27h14c3975_0
sympy 1.1.1 py27hc28188a_0
tblib 1.3.2 py27h51fe5ba_0
terminado 0.8.1 py27_1
testpath 0.3.1 py27hc38d2c4_0
tk 8.6.7 hc745277_3
toolz 0.9.0 py27_0
tornado 5.0.2 py27_0
traitlets 4.3.2 py27hd6ce930_0
typing 3.6.4 py27_0
unicodecsv 0.14.1 py27h5062da9_0
unixodbc 2.3.6 h1bed415_0
urllib3 1.22 py27ha55213b_0
wcwidth 0.1.7 py27h9e3e1ab_0
webencodings 0.5.1 py27hff10b21_1
werkzeug 0.14.1 py27_0
wheel 0.31.1 py27_0
widgetsnbextension 3.2.1 py27_0
wrapt 1.10.11 py27h04f6869_0
xlrd 1.1.0 py27ha77178f_1
xlsxwriter 1.0.4 py27_0
xlwt 1.3.0 py27h3d85d97_0
xz 5.2.4 h14c3975_4
yaml 0.1.7 had09818_2
zeromq 4.2.5 h439df22_0
zict 0.1.3 py27h12c336c_0
zlib 1.2.11 ha838bed_2
This steps works for me:
Jupyter Instance
!source activate python3( or other env. , not root )
!conda config --remove channels defaults
!conda config --add channels conda-forge
!conda install "your library"
!conda list
import "your library"