We Keep Coding

No module found error for a downloaded package (sksparse.cholmod) and how to download a package from the internet to conda

I need to use the sksparse.chomod package however my pycharm does not let me install it as it can't seem to find it.
I found the sksparse package on github and downloaded it but I do not know how to add a package downloaded from the internet into a conda environment. So, my first question would be can you download a package from github and add it to your conda environment, and how do you do this?
As I did not know how to do the above I instead saved the package within my project and thought I could simply import sksparse.cholmod. However, the line in my code that says import sksparse.cholmod as sks has no errors with it, so I assumed that meant this was ok, but when I try to run my file I get this error:
import sksparse.cholmod as sks
ModuleNotFoundError: No module named 'sksparse.cholmod'
If I have downloaded the package into my project why can't it be found, yet there are no errors when importing?
The cholmod file is a pyx file which I've been told should not be a problem.
Please could anyone help, I am reasonably new to python and I am looking for a straight forward solution that won't be time consuming.

It was an issue with windows, I was able to fix this using the instructions on this link
https://github.com/EmJay276/scikit-sparse
We must follow these steps precisely:
(This was tested with a Anaconda 3 installation and Python 3.7)
Install these requirements in order:
'''
conda install -c conda-forge numpy - tested with v1.19.1
conda install -c anaconda scipy - tested with v1.5.0
conda install -c conda-forge cython - tested with v0.29.21
conda install -c conda-forge suitesparse - tested with v5.4.0
'''
Download Microsoft Build Tools for C++ from https://visualstudio.microsoft.com/de/visual-cpp-build-tools/ (tested with 2019, should work with 2015 or newer)
Install Visual Studio Build Tools
Choose Workloads
Check "C++ Buildtools"
Keep standard settings
Run ''' pip install git+https://github.com/EmJay276/scikit-sparse '''
Test ''' from sksparse.cholmod import cholesky '''
Use all the versions stated for numpy etc, however with scipy I installed the latest version and it worked fine.

How to add kaleido package to poetry.lock file?

When attempting to install "kaleido" via Poetry, I receive the following error message:
~ poetry add kaleido
Using version ^0.2.1 for kaleido
Updating dependencies
Resolving dependencies... (3.1s)
Package operations: 1 install, 0 updates, 0 removals
• Installing kaleido (0.2.1.post1): Failed
RuntimeError
Unable to find installation candidates for kaleido (0.2.1.post1)
at ~/.poetry/lib/poetry/installation/chooser.py:72 in choose_for
68│
69│ links.append(link)
70│
71│ if not links:
→ 72│ raise RuntimeError(
73│ "Unable to find installation candidates for {}".format(package)
74│ )
75│
76│ # Get the best link
However, the "kaleido" appears in the poetry.lock file:
[[package]]
name = "kaleido"
version = "0.2.1.post1"
description = "Static image export for web-based visualization libraries with zero dependencies"
category = "main"
optional = false
python-versions = "*"
If I try to export an image, I unsurprisingly receive the following error message:
ValueError:
Image export using the "kaleido" engine requires the kaleido package,
which can be installed using pip:
$ pip install -U kaleido
Does anyone know how to install this package via poetry (or amend the .lock file to do it manually)?

Firstly try to use a master version of poetry as advised in Github issue
or upgrade it to the latest version
pip3 install --upgrade poetry
Then try to install with kaleido with locked version:
poetry add kaleido==0.2.1
That worked in my case.

Heroku: slug size too large after installing Pytorch

I've been getting the slug size too large warning (Compiled slug size: 789.8M is too large (max is 500M)) from Heroku and I can't figure out why, as my model size (cnn.pth below) is fairly small and my file directory is only 1.1mb in total: screenshot of directory.
It seems like the size increase is caused by running pipenv install torch, as the slug size was 89.1mb before installing torch and 798.8mb after.
My Pipfile currently has these packages installed:
[packages]
flask = "*"
flask-sqlalchemy = "*"
psycopg2 = "*"
psycopg2-binary = "*"
requests = "*"
numpy = "*"
gunicorn = "*"
pillow = "*"
torch = "*"
Is there any workaround for this?
Edit: I'm running Mac OSX 10.10.5, using Flask and pipenv.

The pytorch package that you're installing comes with both cpu and gpu support, thus has a large size. It seems you're using the free version of heroku, and only require the cpu support. The solution is to install the pytorch package for cpu only i.e.
In requirements.txt, write the wheel file path corresponding to the version of pytorch (cpu) that you're interested in. You can find the list of wheel files, which can be installed with pip. For example, for PyTorch 1.3.1, torchvision 0.4.2, Python 3.7, Linux, you can write the following for pytorch and torchvision respectively:
https://download.pytorch.org/whl/cpu/torch-1.3.1%2Bcpu-cp37-cp37m-linux_x86_64.whl
https://download.pytorch.org/whl/cpu/torchvision-0.4.2%2Bcpu-cp37-cp37m-linux_x86_64.whl
The above will download torch-1.3.1+cpu-cp37-cp37m-linux_x86_64.whl (107MB) torchvision-0.4.2+cpu-cp37-cp37m-linux_x86_64.whl (13MB) respectively.

Go to PyTorch Get Started page. Choose Stable version, Linux, pip, python, cpu and you can see:
pip install torch==1.8.1+cpu torchvision==0.9.1+cpu torchaudio==0.8.1 -f https://download.pytorch.org/whl/torch_stable.html
Copy versions to your requirements.txt.
torch==1.8.1+cpu
torchvision==0.9.1+cpu
And add to the beginning of the requirements.txt this:
-f https://download.pytorch.org/whl/torch_stable.html
After that, all packages will be installed on Heroku.
For reference:
torch==1.8.1+cpu (169.1 MB);
torchvision==0.9.1+cpu (13.3 MB)

By today, March 28, 2022, these are the working versions:
torch==1.11.0+cpu
torchvision==0.12.0+cpu

(Aug, 2, 2022) the only solution I found was leaving the requirements.txt like this:
--find-links https://download.pytorch.org/whl/torch_stable.html
torch==1.11.0+cpu
--find-links https://download.pytorch.org/whl/torch_stable.html
torchvision==0.12.0+cpu

XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?

I am facing this problem while moving the python-package directory of XGBoost.
Traceback (most recent call last): File "setup.py", line 19, in LIB_PATH = libpath'find_lib_path' File "xgboost/libpath.py", line 46, in find_lib_path 'List of candidates:\n' + ('\n'.join(dll_path))) builtin.XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
Could anyone explain to me how to fix it?
thanks in advance.

You get that message when trying to install the xgboost Python package without the xgboost binaries present. The proper way to install the xgboost Python package from source is the following (assuming you have a compiler such as gcc installed):
git clone --recursive https://github.com/dmlc/xgboost.git
cd xgboost
./build.sh
cd python-package
python setup.py install
I prefer to do it inside a virtual environment. Note that the option --recursive when cloning the repo is essential since it will also clone folders from different repos such as dmlc-core which are necessary for building xgboost.

The other answers didn't work for me so I installed xgboost through Conda commands as listed here.
Just run conda install -c conda-forge xgboost

The first answer's suggestions did not work for me, and left me with the same error as the original question.
If I'm assuming correctly that your full error message is something like this:
C:\Users\Matt\xgboost\python-package>python setup.py install
Traceback (most recent call last):
File "setup.py", line 19, in <module>
LIB_PATH = [os.path.relpath(libfile, CURRENT_DIR) for libfile in libpath['find_lib_path']()]
File "xgboost/libpath.py", line 49, in find_lib_path
'List of candidates:\n' + ('\n'.join(dll_path)))
XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
List of candidates:
C:\Users\Matt\xgboost\python-package\xgboost\xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\../../lib/xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\./lib/xgboost.dll
C:\Users\Matt\AppData\Local\Programs\Python\Python35\xgboost\xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\../../windows/x64/Release/xgboost.dll
C:\Users\Matt\xgboost\python-package\xgboost\./windows/x64/Release/xgboost.dll
then the solution is to
1) Get/find/download the library that setup.py is looking for. Search the xgboost folder for .dll files. See if you can find something like xgboost.dll it might be called libxgboost.dll. If you can, move to step 2. If you cannot find it download it here
2) Copy the .dll file into the xgboost/python-package/xgboost folder. If that .dll is not called xgboost.dll (i.e. if it's called libxgboost.dll) then change the name to xgboost.dll
3) Run the commands as outlined in Gustavo answer. Note these are meant to be run from Git Bash.
If you want a more soup-to-nuts tutorial, this was the best one I found.

thanks to Joe Nyland that found the a good answer for this problem in here.
As he said (and also worked for me) you need to run following commands:
$ brew install gcc#5
$ pip install xgboost

Try this it worked for me:
brew install gcc-5
brew install cmake
pip install xgboost

In my case (Ubuntu 16.04, with CUDA 9.0) the Python library build was evidently broken by adding R library package to the cmake options list:
1) python lib installation works OK:
cmake .. -DUSE_CUDA=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-${CUDA_VER}
cd /tmp/xgboost/python-package && python3 setup.py install
2) adding '-DR_LIB=ON' switch to cmake breaks subsequent python library installation attempt:
cmake .. -DUSE_CUDA=ON -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-${CUDA_VER} -DR_LIB=ON
cd /tmp/xgboost/python-package && python3 setup.py install
> "XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root
> path?"

I had the same issues. I downloaded the file (xgboost.dll) from
(https://picnet.com.au/blog/xgboost-windows-x64-binaries-for-download/)
into the already existing xgboost folder
(C:\Users\Naganandini\AppData\Roaming\Python\Python37\site-packages\xgboost).
It's perfectly working now :)

I faced the same error while installing xgboost using anaconda prompt because the installation of xgboost was interfering with the versions of some other pre-installed libraries. Installing xgboost by creating a virtual environment solved the issue.
On macOS and Linux:
python3 -m pip install --user virtualenv #Install virtualenv module
python3 -m venv env #Create a virtual environment 'env'
source env/bin/activate #Activate virtual environment
pip install xgboost #Install xgboost
On Windows:
py -m pip install --user virtualenv #Install virtualenv module
py -m venv env #Create a virtual environment 'env'
.\env\Scripts\activate #Activate virtual environment
pip install xgboost #Install xgboost
Refer https://packaging.python.org/guides/installing-using-pip-and-virtual-environments for help regarding virtual environment.

Search for xgboost.dll file under your C:\Users<your-user-name> directory.
If you have installed xgboost, somewhere you should find a copy of the xgboost.dll file.
Once you find the xgboost.dll file, just copy it to one of the paths mentioned in the error message,
e.g. C:\Users<your-user-name>\AppData\Roaming\Python\Python37\site-packages\xgboost\ . Import should work now.

1.I started off with the conda installation from anaconda prompt:
(base) C:\Users\abhi.b>conda install -c anaconda py-xgboost
This lead to the following error :
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.|
failed
UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:
Specifications:
- py-xgboost -> python[version='>=2.7,<2.8.0a0|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0']
Your python: python=3.8
If python is on the left-most side of the chain, that's the version you've asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.
Decided not to followup this path, as exploring python downgrade was not an option for me.
2.There after i followed "The final attempt that works " on the below link :
https://medium.com/analytics-vidhya/this-is-what-you-should-do-if-you-fail-to-install-the-latest-or-an-older-version-of-xgboost-on-7942a7641eee
However i got the same error as other users faced ::
>>> import xgboost as xgb
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\__init__.py", line 11, in <module>
from .core import DMatrix, Booster
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\core.py", line 161, in <module>
_LIB = _load_lib()
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\core.py", line 123, in _load_lib
lib_paths = find_lib_path()
File "C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\libpath.py", line 45, in find_lib_path
raise XGBoostLibraryNotFound(
xgboost.libpath.XGBoostLibraryNotFound: Cannot find XGBoost Library in the candidate path, did you install compilers and run build.sh in root path?
List of candidates:
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\../../lib/xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\./lib/xgboost.dll
C:\Users\abhi.b\Anaconda3\xgboost\xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\../../windows/x64/Release/xgboost.dll
C:\Users\abhi.b\Anaconda3\lib\site-packages\xgboost\./windows/x64/Release/xgboost.dll
>>> exit()
BTW the link to download xgboost.dll as posted above -
https://picnet.com.au/blog/xgboost-windows-x64-binaries-for-download/
was not clear to me ,so i decided to not follow this thread.
3.Finally ,the solution was very simple for me,Open ANACONDA PROMPT and Type
(base) C:\Users\abhi.b>pip install xgboost
This lead to
Collecting xgboost
Downloading xgboost-1.2.1-py3-none-win_amd64.whl (86.5 MB)
|████████████████████████████████| 86.5 MB 1.7 MB/s
Requirement already satisfied: scipy in c:\users\abhi.b\anaconda3\lib\site-packages (from xgboost) (1.5.0)
Requirement already satisfied: numpy in c:\users\abhi.b\anaconda3\lib\site-packages (from xgboost) (1.18.5)
Installing collected packages: xgboost
Successfully installed xgboost-1.2.1
Then goto python prompt
(base) C:\Users\abhi.b>python
>>>import xgboost as xgb
Or you can type the same command on IDE ie Spyder etc
Voila!!

Anaconda pip offline installation including dependencies (tensorflow)

I want to install tensorflow in an anaconda environment on a system without internet connection, and where I have not got root access (i.e. I want to install it to my local user only)
I have downloaded the .whl files of tensorflow and the required dependencies and copied them to the machine I want to use. Once I've entered my anaconda environment I started installing the packages using
pip install -b working_directory/build -t working_directory/target package.whl
But when I want to install a package which depends on an earlier installed package, it cannot find it.
So I wonder, how can you tell pip where to look for dependencies? Can I install tensorflow in an easier way, still offline and without root?

I am using PyCharm for the development with anaconda. I was also facing issue while installing tensorflow using conda, I also installed the python 3.6 and used the steps given on the tensorflow website. But finally I solved this using below steps and made it work on pyCharm:
Step 1: I downloaded binaries (.whl) file of tensorflow (The links of binaries are given on the git page https://github.com/tensorflow/tensorflow)
Step 2: Then I installed the tensorflow offline using below command:
pip.exe install --upgrade --no-deps C:\Important_Software\tensorflow-1.3.0rc0-cp36-cp36m-win_amd64.whl
Step 3: Then Tensorflow files have been created at below location:
C:\Program Files\Python36\Lib\site-packages
I copied these files and pasted in the Anaconda site-packages (Anaconda3\Lib\site-packages).
Step 4: Tensorflow is installed but below error has come when running the basic program:
File "C:\ProgramData\Anaconda3\lib\site-packages\tensorflow\core\framework\graph_pb2.py", line 6, in <module>
from google.protobuf import descriptor as _descriptor
ModuleNotFoundError: No module named 'google'
Step 5: I have solved this error using pip installation of protocol buffer
pip.exe install --upgrade --no-deps "C:\TarFile_location\protobuf-3.3.0.tar.gz"
Step 6: After this 3 files “protobuf-3.3.0-py3.6-nspkg.pth”, “protobuf-3.3.0-py3.6.egg-info” and “google” are created at the below location:
C:\Program Files\Python36\Lib\site-packages
These three file should be pasted at the Anaconda site-packages. (Anaconda3\Lib\site-packages)
Step 6: The I ran the below program and It worked:
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))
If there will be still some Errors then all dependencies have to be downloaded and installed similar as step 2 or 5 from https://pypi.python.org/pypi/tensorflow.
Important note: I was using the Windows command prompt with admin Access.