I am trying to create a conda package which should be compatible with all python versions greater than or equal to 3.7.
I specified this requirement in my conda.recipe/meta.yaml:
requirements:
host:
- python >=3.7
- pip
run:
- python >=3.7
- importlib-resources >=1.4.0
- ...
For my build command, I first activated a Python 3.7 conda environment, then I specified the build command should use the same environment, and I also specified that the build command should use Python 3.7 just to be safe:
$ CONDA_ENV=/path/to/py3.7/conda/env
$ conda create --yes -p $CONDA_ENV python=3.7 conda-build conda-verify importlib-resources>=1.4.0 # ...remaining reqs
$ conda activate $CONDA_ENV
$ conda build --python=3.7 -p $CONDA_ENV /path/to/package/dir
The build step in conda.recipy/meta.yaml too uses this same environment:
build:
script: bash -c 'source ~/.bashrc && conda activate /path/to/py3.7/conda/env && python -m pip install --no-deps --ignore-installed -vv /path/to/package/dir'
The problem
Running the above commands creates this file:
my-package-1.1.0-py310_0.tar.bz2
I don't get why py310 is in the package name, I did everything I can think of to make it compatible with 3.7 as well.
Here is what happens when I try to create an environment with both python 3.7 and my created package (after uploading the tar.bz2 file to my conda repo):
$ conda create -p ~/temp/conda python=3.7 my-package
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
Package python conflicts for:
python=3.7
my-package -> python[version='>=3.10,<3.11.0a0']
my-package -> importlib-resources[version='>=1.4.0'] -> python[version='2.7.*|3.5.*|3.6.*|>=2.7,<2.8.0a0|>=3.8,<3.9.0a0|>=3|>=3.6|>=3.7|>=3.6,<3.7.0a0|>=3.7,<3.8.0a0|>=3.5,<3.6.0a0|3.4.*|>=3.9,<3.10.0a0|3.10.*']
The following specifications were found to be incompatible with your system:
- feature:/linux-64::__glibc==2.27=0
- feature:|#/linux-64::__glibc==2.27=0
Your installed version is: 2.27
But if I don't specify 3.7 when creating the environment it works fine, but creates an environment with 3.10.
$ conda create -p ~/temp/conda python my-package
Question
How can I make my-package compatible with Python 3.7?
After much unproductive research an unending amounts of trial and error, I found that I should have made my package not specific to any python version by adding noarch: python to my meta.yaml:
build:
noarch: python
script: bash -c 'source ~/.bashrc && conda activate /path/to/py3.7/conda/env && python -m pip install --no-deps --ignore-installed -vv /path/to/package/dir'
Related
When I try to generate proto files using Python's grpc_tools.protoc library:
python -m grpc_tools.protoc -I protos --python_out=. --gprc_python_out=. protos/test.proto
it fails with the error
protoc-gen-gprc_python: program not found or is not executable
--gprc_python_out: protoc-gen-gprc_python: Plugin failed with status code 1.
I've already installed the following packages in my conda environment:
conda install -c conda-forge grpclib
conda install grpcio grpcio-tools
$ python -m grpc_tools.protoc --version
libprotoc 3.6.1
What is my conda environment missing that is prevent me from running my above command? Thank you
Typo.
You want --grpc_python_out=.
You have --gprc_python_out=.
I'm trying to use make build which has a docker build but every time it reaches a layer in which it tries to install "conda install -c conda-forge r-base" it keeps failing:
Step 12/25 : RUN conda install -c conda-forge r-base
---> Running in 828e7f234ed3
Collecting package metadata (current_repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
And then it starts examining conflicts until it reaches this error:
The command '/bin/sh -c conda install -c conda-forge r-base' returned a non-zero code: 1
make: *** [Makefile:2: build] Error 1
Does somebody have any idea what I could do to fix this? So that it continues building with no errors? I didn't code the Dockerfile so I'm a little lost on what to do.
Dockerfile
FROM continuumio/anaconda3
ENV INSTALL_PATH /environment_name
RUN mkdir -p $INSTALL_PATH
WORKDIR $INSTALL_PATH
#COPY environment.yml environment.yml
#RUN conda env create -f environment.yml
RUN conda create -n environment_name
RUN /opt/conda/bin/activate environment_name
#RUN conda config --add channels conda-forge
#RUN conda config --set channel_priority strict
#RUN conda install r-cairodevice
RUN conda install -n environment_name bokeh h5py numpy scipy pandas scikit-learn sphinx spyder jupyter statsmodels
RUN conda install -n environment_name -c bioconda pymzml
RUN conda install -n environment_name -c bioconda netcdf4
RUN conda install -c conda-forge fonts-conda-forge
RUN conda install -c conda-forge r-base
RUN conda install -c bioconda bioconductor-ropls
RUN pip install flask werkzeug gunicorn
RUN conda install seaborn
RUN echo "source activate environment_name" > ~/.bashrc
ENV PATH /opt/conda/envs/environment_name/bin:$PATH
COPY . .
RUN chmod +x run_server.sh
RUN chmod +x start.sh
ADD start.sh /
CMD /start.sh
RUN conda install -c conda-forge libjpeg-turbo
RUN git clone https://bitbucket.org/iAnalytica/mshub_process.git
RUN conda install -n environment_name -c bioconda pyteomics
#CMD gunicorn -b 0.0.0.0:8000 --access-logfile - "api.app:app"
Makefile
build:
docker build -t environment_name .
This means that the dependencies you're listing within your conda file are causing some sort of error during the build. It could be many things, but if you were able to build that conda environment in your local environment outside of docker, then a possible culprit would be that the difference in operating system is creating a sufficient change in the dependencies that's causing the conda build to fail. Without knowing more about the error itself or the traceback, I would try commenting out half of the dependencies in your conda file and rerunning. If it succeeds, then try with the other half commented out. If it fails, uncomment half of the previously commented dependencies and continue this binary-like search until you find the offending dependency(ies).
For what it's worth, using docker removes much of the benefit of using an environment manager like conda. While I don't have visibility into the rest of your application, it might be worth considering what you're really getting out of conda in a containerized environment and determining if other means of setting up your environment better suit your needs. I say this especially because the build times for conda can be quite slow and add to the bloat of the container.
I tried to install psi4 to do a quantum chemistry calculation in the jupyter/minimal-notebook container.
However, as shown in the code block below, I could not install it due to UnsatisfiableError.
(base) jovyan#17285cef33b0:~/work$ conda create -n psi4 python=3.6
Collecting package metadata (current_repodata.json): done
Solving environment: done
....
(ellipsis)
(base) jovyan#17285cef33b0:~/work$ conda activate psi4
(psi4) jovyan#17285cef33b0:~/work$ conda install psi4 -c psi4
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: \
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed
UnsatisfiableError: The following specifications were found to be incompatible with each other:
Output in format: Requested package -> Available versions
(psi4) jovyan#17285cef33b0:~/work$
It did not show what was not satisfied.
I don't want to use anaconda because I need to pay for commercial use of anaconda, and I want to use conda-forge as the environment to run psi4. I also want to use docker because I want to have the same conditions to run the program.
Do you have any good ideas to solve this error?
Or is there any environment that can fulfill my wishes?
My environment is Windows 10 and I am using Docker Desktop WSL 2 backend.
The previous answer was deleted because I just pasted the my blog post link. I extract the main steps of how to build from the my blog post and describe it.
I built and installed Psi4 on a docker container by myself. I wrote a post on my blog about how to do that. Please see the following blog post for more details.
https://solkul.hatenablog.com/entry/2021/04/23/002237
Here are the main steps on how to build it.
Basically, you can build it according to this Psi4 documentation.
1. Clone Psi4 repository
Clone the Psi4 repository.
(base) root$ git clone https://github.com/psi4/psi4.git
(base) root$ cd psi4
2. Create new conda environment
Start bash as root. Psi4 only supports until python 3.7, so I should create a conda environment for python 3.7.
(base) root$ conda create -n quantum python=3.7
(base) root$ conda activate quantum
(quantum) root$
3. Install the packages needed to build
Type the following command to install the packages required for the build. Psi4 documentation lists tools and dependencies required for the build.
(quantum) root$ apt-get update
(quantum) root$ apt-get install -y build-essential cmake clang libssl-dev
(quantum) root$ apt-get install -y libopenblas-base libopenblas-dev
(quantum) root$ conda install -y numpy networkx pint pydantic
(quantum) root$ apt-get install -y libmpfr-dev libeigen3-dev
4. Buid and install
It's time to build! Type the following command to Configure and Generate to build.
(quantum) root /psi4$ mkdir build
(quantum) root /psi4$ cd build
(quantum) root /psi4/build$ cmake ..
Let's build!
(quantum) root /psi4/build$ make -j`getconf _NPROCESSORS_ONLN`
This build takes several hours.
After build is complete, type the following command to install it.
(quantum) root /psi4/build$ make install
5. Append lib of install directry to the python import path
In the default configuration, the binaries, includes, libraries, etc. should be installed in /usr/local/psi4. In order to use Psi4 as python module, it is needed to append /usr/local/psi4/lib to the python import path. You should also use Psi4 in the quantum environment, which I just created as the python 3.7 environment. You can do this by appending the directry to sys.path each time as follows.
import sys
sys.path.append("usr/local/psi4/lib")
From now, you can import Psi4.
import psi4
I followed the directions to install etetoolkit for mac.
# Install Minconda (you can ignore this step if you already have Anaconda/Miniconda)
curl -L 'http://repo.continuum.io/miniconda/Miniconda3-latest-MacOSX-x86_64.sh' -o Miniconda3-latest-MacOSX-x86_64.sh
bash Miniconda3-latest-MacOSX-x86_64.sh -b -p ~/anaconda_ete/
export PATH=~/anaconda_ete/bin:$PATH;
# Install ETE and external tools
conda install -c etetoolkit ete3 ete_toolchain
# Check installation
ete3 build check
I get an error when I run conda install -c etetoolkit ete3 ete_toolchain
Error:
UnsatisfiableError: The following specifications were found to be
incompatible with the existing python installation in your
environment:
ete3 -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0']
ete_toolchain -> python[version='>=2.7,<2.8.0a0,>=3.5,<3.6.0a0,>=3.6,<3.7.0a0'
I ran the [package installer][2] for anaconda 3.7. (I don't think I actually had to do this.)
From terminal run: conda install python=2.7
Then conda install -c etetoolkit ete3 ete_toolchain
Then I verified this worked by running ete3 build check. In the results clustalo: MISSING displayed, but that's a new problem to solve.
UPDATE
The final problem was solved by running conda install -c etetoolkit ete3_external_apps
There is a type-o in the instructions given by ete3. Use this instead.
I'm trying to create a python virtualenv using anaconda python3.6 in ubuntu 16.04. Following https://docs.python.org/3/library/venv.html , I've tried
deploy#server:~/miniconda3/bin$ python3 -m venv ~/test
Error: Command '['/home/deploy/test/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.
operating from the miniconda directory at ~/miniconda3/bin (screenshot). How can I this working?
edit:
deploy#server:~/miniconda3/bin$ /home/deploy/test/bin/python3 -Im ensurepip --upgrade --default-pip
/home/deploy/test/bin/python3: No module named ensurepip
If you are using anaconda, you should be using conda environments.
conda create --name test
For more information, see Managing Environments.
EDIT In response to OP wanting to use virtualenvs.
The error is with python not being able to find pip. You can get around this by installing it manually.
python3 -m venv test --without-pip
cd test
source bin/activate
curl https://bootstrap.pypa.io/get-pip.py | python3
At this point you will have a basic virtualenv with pip installed.