We Keep Coding

Miniconda in Mac Book - x array issues

I installed the last version of Miniconda and created a new environment.
I have pip installed the following repos:
pip install git+https://git.ufz.de/FINAM/finam.git
pip install git+https://git.ufz.de/FINAM/finam-netcdf.git
pip install git+https://git.ufz.de/FINAM/finam-plot.git
pip install git+https://git.ufz.de/FINAM/finam-graph.git
Finally, I cloned this repo
git clone https://git.ufz.de/FINAM/finam-examples.git
In that repo I ran two files component.py and pet_calculator.py located at:
~/finam-examples/01_Hargreaves-Samani/src/
I first ran component.py, and that was ok.
The problem came when I ran pet_calculator.py.
This is my error:
(test) sh-3.2$ /Users/lealroja/miniconda3/envs/test/bin/python /Users/lealroja/Documents/UFZ/finam_project/finam-examples/01_Hargreaves-Samani/src/pet_calculator.py
2023-02-19 23:09:44,109 INFO: init composition - FINAM
Traceback (most recent call last):
File "/Users/lealroja/Documents/UFZ/finam_project/finam-examples/01_Hargreaves-Samani/src/pet_calculator.py", line 49, in <module>
composition.initialize()
File "/Users/lealroja/miniconda3/envs/test/lib/python3.11/site-packages/finam/schedule.py", line 148, in initialize
mod.initialize()
File "/Users/lealroja/miniconda3/envs/test/lib/python3.11/site-packages/finam/sdk/component.py", line 65, in initialize
self._initialize()
File "/Users/lealroja/miniconda3/envs/test/lib/python3.11/site-packages/finam_netcdf/reader.py", line 191, in _initialize
self.dataset = xr.open_dataset(self.path)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/lealroja/miniconda3/envs/test/lib/python3.11/site-packages/xarray/backends/api.py", line 510, in open_dataset
engine = plugins.guess_engine(filename_or_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/lealroja/miniconda3/envs/test/lib/python3.11/site-packages/xarray/backends/plugins.py", line 177, in guess_engine
raise ValueError(error_msg)
ValueError: did not find a match in any of xarray's currently installed IO backends ['netcdf4', 'scipy']. Consider explicitly selecting one of the installed engines via the ``engine`` parameter, or installing additional IO dependencies, see:
https://docs.xarray.dev/en/stable/getting-started-guide/installing.html
https://docs.xarray.dev/en/stable/user-guide/io.html
I have that error when using a Mac book Pro. However, when I do the exact same procedure and run it in Ubuntu I have no issues.
I am not an expert, and therefore I have no idea what to do. Could you please help me?
I have tried answers from this links:
IO backend error in the xarray for netcdf file
ValueError: found the following matches with the input file in xarray's IO backends: ['netcdf4', 'h5netcdf']
Thank you in advance.

Numpy not importing

When i try to import numpy to my python file it says:
Traceback (most recent call last):
File "c:\src\Python\Raycast_Test_1\test.py", line 1, in <module>
import numpy as np
ModuleNotFoundError: No module named 'numpy'
(i just found some things in the pip help thingy so i did them and this is what i got)
and when i do pip search numpy this shows up:
ERROR: Exception:
Traceback (most recent call last):
File "c:\users\marti\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\cli\base_command.py", line 224, in _main
status = self.run(options, args)
File "c:\users\marti\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\commands\search.py", line 62, in run
pypi_hits = self.search(query, options)
File "c:\users\marti\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\commands\search.py", line 82, in search
hits = pypi.search({'name': query, 'summary': query}, 'or')
File "c:\users\marti\appdata\local\programs\python\python38\lib\xmlrpc\client.py", line 1109, in __call__
return self.__send(self.__name, args)
File "c:\users\marti\appdata\local\programs\python\python38\lib\xmlrpc\client.py", line 1450, in __request
response = self.__transport.request(
File "c:\users\marti\appdata\local\programs\python\python38\lib\site-packages\pip\_internal\network\xmlrpc.py", line 46, in request
return self.parse_response(response.raw)
File "c:\users\marti\appdata\local\programs\python\python38\lib\xmlrpc\client.py", line 1341, in parse_response
return u.close()
File "c:\users\marti\appdata\local\programs\python\python38\lib\xmlrpc\client.py", line 655, in close
raise Fault(**self._stack[0])
xmlrpc.client.Fault: <Fault -32500: "RuntimeError: PyPI's XMLRPC API has been temporarily disabled due to unmanageable load and will be deprecated in the near future. See https://status.python.org/ for more information.">
and when i do pip check numpy this shows up:
No broken requirements found.
i have tried uninstaling the installing numpy but that didnt fix it.
How do i fix this?
(note i am using the vscode terminal for this)

pip install numpy --upgrade
or
pip install numpy --upgrade --ignore-installed

Just to make sure you are running correct pip, try running "pip -V" or "pip --version", it gives you which version of python it refers to. Its also possible that you might have pip for say python 2.7, pip3 for python 3.6, and pip3.7 for python 3.7, if you have multiple versions of python installed. For simplicity you can set the most frequently used pip version as pip by setting an alias in ~/.bashrc. This is done by adding the following line in bashrc:
alias pip=pip3.6.
After this you can try and install the packages and import it swiftly.

Conda skeleton pypi fails with pmdarima -- AttributeError numpy disutils

I'm trying to install pmdarima package in a conda env as it requires statsmodels <0.12 and I would also like to use statmodels latest release for this project. My understanding is limited, but I first tried
conda skeleton pypi pmdarima
This failed with the same AttributeError related to numpy disutils described below.
Similarly to the questioner in this question: Conda skeleton pypi: ModuleNotFoundError: No module named 'numpy' I was unable to get this to work either.
conda install -n _build numpy
I then tried the advice laid out in the above-linked SO question and ran:
conda skeleton pypi --extra-specs numpy pmdarima
This results in pretty long output but the key piece as far as I can tell is:
Applying patch: '/var/folders/7f/8d7rz34522gcq4z774n9v91r0000gn/T/tmp8tv1gdnxconda_skeleton_pmdarima-1.7.1.tar.gz/pypi-distutils.patch'
patching file core.py
Hunk #1 succeeded at 168 with fuzz 2 (offset 1 line).
Partial import of pmdarima during the build process.
Requirements: ['joblib>=0.11', 'Cython>=0.29,<0.29.18', 'numpy>=1.17.3', 'pandas>=0.19', 'scikit-learn>=0.22', 'scipy>=1.3.2', 'statsmodels>=0.11,<0.12', 'urllib3', 'setuptools<50.0.0']
Setting up with numpy.distutils.core
Traceback (most recent call last):
File "setup.py", line 324, in <module>
do_setup()
File "setup.py", line 320, in do_setup
setup(**metadata)
File "/opt/anaconda3/conda-bld/skeleton_1599410605346/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.7/site-packages/numpy/distutils/core.py", line 130, in setup
if dist.help or not _command_line_ok():
AttributeError: 'NoneType' object has no attribute 'help'
$PYTHONPATH = /var/folders/7f/8d7rz34522gcq4z774n9v91r0000gn/T/tmp8tv1gdnxconda_skeleton_pmdarima-1.7.1.tar.gz/pmdarima-1.7.1
Leaving build/test directories:
Work:
/opt/anaconda3/conda-bld/skeleton_1599410605346/work
Test:
/opt/anaconda3/conda-bld/skeleton_1599410605346/test_tmp
Leaving build/test environments:
Test:
source activate /opt/anaconda3/conda-bld/skeleton_1599410605346/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place
Build:
source activate /opt/anaconda3/conda-bld/skeleton_1599410605346/_build_env
Error: command failed: /opt/anaconda3/conda-bld/skeleton_1599410605346/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python setup.py install
Any ideas? Thank you so much in advance, very much appreciated!

Not sure what the proper resolution is for the NumPy-based build, but the setup.py does check for an sdist argument, which branches to use setuptools for the build instead of numpy. Hence, the following seems to avoid the code causing the issue:
conda skeleton pypi --setup-options sdist pmdarima
I suspect the downside to this workaround is that there is no Cython compilation done, but not totally sure.
You may want to file an issue on the pmdarima GitHub to see if any of the devs have ideas. Otherwise, it might just be more efficient to pip install the package in the environment you want it (as long as that environment is not base).

#merv thank you for your help! Interestingly, this works for the skeleton, but fails when running conda-build with the following (truncated until the errors) output:
0 location(s) to search for versions of sdist:
Given no hashes to check 0 links for project 'sdist': discarding no candidates
ERROR: Could not find a version that satisfies the requirement sdist (from versions: none)
ERROR: No matching distribution found for sdist
Exception information:
Traceback (most recent call last):
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/cli/base_command.py", line 216, in _main
status = self.run(options, args)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/cli/req_command.py", line 182, in wrapper
return func(self, options, args)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/commands/install.py", line 325, in run
reqs, check_supported_wheels=not options.target_dir
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 183, in resolve
discovered_reqs.extend(self._resolve_one(requirement_set, req))
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 388, in _resolve_one
abstract_dist = self._get_abstract_dist_for(req_to_install)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 339, in _get_abstract_dist_for
self._populate_link(req)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 305, in _populate_link
req.link = self._find_requirement_link(req)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/resolution/legacy/resolver.py", line 270, in _find_requirement_link
best_candidate = self.finder.find_requirement(req, upgrade)
File "$PREFIX/lib/python3.7/site-packages/pip/_internal/index/package_finder.py", line 928, in find_requirement
req)
pip._internal.exceptions.DistributionNotFound: No matching distribution found for sdist
Removed build tracker: '/private/tmp/pip-req-tracker-umxriyi2'
Traceback (most recent call last):
File "/opt/anaconda3/bin/conda-build", line 11, in <module>
sys.exit(main())
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 474, in main
execute(sys.argv[1:])
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/cli/main_build.py", line 465, in execute
verify=args.verify, variants=args.variants)
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/api.py", line 210, in build
notest=notest, variants=variants)
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 3036, in build_tree
notest=notest,
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/build.py", line 2155, in build
cwd=src_dir, stats=build_stats)
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/utils.py", line 407, in check_call_env
return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
File "/opt/anaconda3/lib/python3.7/site-packages/conda_build/utils.py", line 387, in _func_defaulting_env_to_os_environ
raise subprocess.CalledProcessError(proc.returncode, _args)
subprocess.CalledProcessError: Command '['/bin/bash', '-o', 'errexit', '/opt/anaconda3/conda-bld/pmdarima_1599500058832/work/conda_build.sh']' returned non-zero exit status 1.
I'll confess we're well above my head at this point, (do I need to edit the meta.yaml file somehow? Is that crazy?) but will try opening an issue as you suggested, once again really appreciate it.

`No module named nose_parameterized` during theano installation

At the moment I am trying to install theano on my ubuntu 16.04 system with cuda support. The installation process itself went well until I tried to test the installation:
python -c "import theano; theano.test()"
The last few lines of the output were:
======================================================================
ERROR: Failure: ImportError (No module named nose_parameterized)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/myUser/anaconda2/lib/python2.7/site-packages/theano/tests/test_rop.py", line 16, in <module>
from theano.tests import unittest_tools as utt
File "/home/myUser/anaconda2/lib/python2.7/site-packages/theano/tests/unittest_tools.py", line 7, in <module>
from nose_parameterized import parameterized
ImportError: No module named nose_parameterized
----------------------------------------------------------------------
Ran 408 tests in 13.085s
FAILED (SKIP=26, errors=80)
These were preceded by approximately 20 blocks with what appears to be different tests failing on exactly the same top 3 lines in the stacktrace. The code where the final occurs looks like this (with added line numbers):
411. # FIXME: to support module.name names,
412. # do what resolve-name does and keep trying to
413. # import, popping tail of module into addr.call,
414. # until we either get an import or run out of
415. # module parts
416. try:
417. module = self.importer.importFromPath(
418. addr.filename, addr.module)
419. finally:
420. self.config.plugins.afterImport(
421. addr.filename, addr.module)
While searching for a solution I came across:
Error running nosetests
I tried the solution proposed there:
sudo pip install nose-parameterized
However this had no effect. Does anyone know what else could be causing this error?
edit: so I reinstalled theano to make the problem more reproducible. I installed theano using the instructions found here, so I used:
sudo apt-get install python-numpy python-scipy python-dev python-pip python-nose g++ libopenblas-dev git
sudo pip install Theano
Then I used:
nosetests theano
The last few lines of the output of this are almost the same as earlier:
======================================================================
ERROR: Failure: ImportError (No module named nose_parameterized)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/loader.py", line 418, in loadTestsFromName
addr.filename, addr.module)
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/importer.py", line 47, in importFromPath
return self.importFromDir(dir_path, fqname)
File "/home/myUser/anaconda2/lib/python2.7/site-packages/nose/importer.py", line 94, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File "/home/myUser/.local/lib/python2.7/site-packages/theano/tests/test_rop.py", line 16, in <module>
from theano.tests import unittest_tools as utt
File "/home/myUser/.local/lib/python2.7/site-packages/theano/tests/unittest_tools.py", line 7, in <module>
from nose_parameterized import parameterized
ImportError: No module named nose_parameterized
----------------------------------------------------------------------
Ran 457 tests in 160.608s
FAILED (SKIP=24, errors=95)
edit 2: I thought it might be a python-version issue. When I try:
python -V
I get:
Python 2.7.12 :: Anaconda 4.2.0 (64-bit)
And the following folders exist:
/usr/local/lib/python2.7/dist-packages/nose_parameterized
So, that shouldn't be a problem as far as I can see. However when I try:
python2.7 -c "import nose_parameterized"
I get:
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named nose_parameterized

Found the problem, anaconda was looking in its own folder. So to solve, I did:
conda install nose-parameterized

The following works for me:
pip install nose-parameterized

Initializing H2O in Python

When trying to initialize H2O in Python by running the lines:
import h2o
h2o.init()
I getting the exceptions:
Traceback (most recent call last):
File "C:/workspace/DataScience/univariate_analysis/prudential_RF.py", line 4, in
h2o.init()
File "C:\Anaconda\lib\site-packages\h2o\h2o.py", line 668, in init
H2OConnection(ip=ip, port=port,start_h2o=start_h2o,enable_assertions=enable_assertions,license=license,max_mem_size_GB=max_mem_size_GB,min_mem_size_GB=min_mem_size_GB,ice_root=ice_root,strict_version_check=strict_version_check)
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 101, in __init__
raise EnvironmentError, message
EnvironmentError: Version mismatch. H2O is version 3.2.0.9, but the python package is version 3.6.0.8.
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
File "C:\Anaconda\lib\atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 565, in end_session
H2OConnection.delete(url_suffix="InitID")
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 393, in delete
return __H2OCONN__._do_raw_rest(url_suffix, "DELETE", None, **kwargs)
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 483, in _do_raw_rest
.format(http_result.status_code,http_result.reason,method,url,detailed_error_msgs))
EnvironmentError: h2o-py got an unexpected HTTP status code:
404 Not Found (method = DELETE; url = http://localhost:54321/3/InitID).
detailed error messages: DELETE /3/InitID not found
Error in sys.exitfunc:
Traceback (most recent call last):
File "C:\Anaconda\lib\atexit.py", line 24, in _run_exitfuncs
func(*targs, **kargs)
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 565, in end_session
H2OConnection.delete(url_suffix="InitID")
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 393, in delete
return __H2OCONN__._do_raw_rest(url_suffix, "DELETE", None, **kwargs)
File "C:\Anaconda\lib\site-packages\h2o\connection.py", line 483, in _do_raw_rest
.format(http_result.status_code,http_result.reason,method,url,detailed_error_msgs))
EnvironmentError: h2o-py got an unexpected HTTP status code:
404 Not Found (method = DELETE; url = http://localhost:54321/3/InitID).
detailed error messages: DELETE /3/InitID not found
Please note I have followed the installation steps in this link Use H2O directly from Python

You have a version mismatch between your h2o python code and your h2o java code.
If you download a fresh version of h2o, make sure you also do a pip install with the same version number. Don't use a generic pip install... rather specify the url and version explicitly.
Download the java package and start the java jar:
But then also click the install python link to get the corresponding pip install command:
Notice that this line needs to match the java version number that you just installed:
pip install http://h2o-release.s3.amazonaws.com/h2o/reltibshirani/8/Python/h2o-3.6.0.8-py2.py3-none-any.whl
Then try again:
import h2o
h2o.init()
Hope this helps!

EnvironmentError: Version mismatch. H2O is version 3.2.0.9, but the python package is version 3.6.0.8.
It looks like you have an instance of 3.2.0.9 already running and the Python package, which is 3.6.0.8, is complaining due to incompatibility.
I recommend you kill the current running instance of H2O (the 3.2.0.9 version that you see in the error message) and run the same commands (import h2o, h2o.init) to drive H2O from Python cleanly.