I'm looking for a way to build a conda package which would use a specific python version.
According to the issue submitted a while ago conda treats python in a special way, so simply putting
run:
- python ==2.7.13 <build>
into meta.yaml won't work: conda will create a dependency on the latest release of 2.7 (>=2.7,<2.8) ignoring the minor version and build.
Why?
The aforementioned issue suggests that pinning the python version is the wrong thing to do, but I really like the idea of my builds and deployments being reproducible. Running conda create -n prod_env my_application=1.0.0 today should produce exactly the same environment as it did yesterday, including all dependencies and python version.
Related
I have downloaded the anaconda3 a complete newer version from official site.. I tried to create an environment but the 'solving environment' keeps on running..
I tried turning off windows defender but it didn't work.. someone plz help.. I am using windows 11 pro, and I have downloaded and installed Anaconda3-2022.10-Windows-x86_64 on 23/12/2022
The actively available Anaconda builds for Python 3.6 all use 3.6.10, so you would have a possibly easier solve with:
conda create -n ssd_env python=3.6 anaconda
However, there is the custom build, anaconda=custom=py36_1 that should be generally compatible with python>=3.6,<3.7. That is what I get when simulating the solve from OP:
CONDA_SUBDIR=win-64 mamba create -n ssd_env --override-channels -c anaconda python=3.6.8 anaconda
which solves almost instantly with Mamba.
Using conda also takes unreasonably long for me, even when explicitly identifying the anaconda=custom=py36_1. The reason this takes so long is that this anaconda package has no version constraints on the 100+ packages it specifies, which means a huge search space. This is aggravated by the fact that the solvers work from latest to oldest package versions, and the versions that are expected to be identified are ~3 years down the stack.
I recommend:
Use Mamba as the solver.
Don't use the anaconda package unless absolutely needed. Most users do not ever need all those packages - instead just specify the packages actually required.
Background
Currently, I have Python 2.7.17 and Python 3.8.1 installed on my Mac (v 10.14.6). Both Python versions were downloaded directly from Python.org using a "macOS 64-bit installer" .pkg file, in late 2019. Both were installed here:
python2: /Library/Frameworks/Python.framework/Versions/2.7/bin/python2
python3: /Library/Frameworks/Python.framework/Versions/3.8/bin/python3
Currently, I use only pip (either pip2 or pip3, depending) to install python packages, which are saved here:
python2: /Library/Frameworks/Python.framework/Versions/X.X/lib/python2.7/site-packages
python3: /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages
I am about to begin an online course which requires that I install miniconda in order to install the packages that I need for the course. But after having a very bad experience having Anaconda installed on my machine years ago, and it taking me months to remove it and re-create a clean Python install, I am nervous. I don't want my machine to become a mess again, rife with error messages that it can't find this or that python version or this or that python library.
Question(s)
Assuming there will now be multiple installations of Python on my machine (in addition to the versions that I already have, since I believe yet another version of python is installed along with conda), and there will be new places where python packages will be saved, what can I expect to change about how I write python after I install miniconda?
Is there anything I will need to do differently from what I am doing now when I write python code?
How can I ensure that everything remains working and compatible and I avoid the fate I suffered many years ago?
Happy to edit/clarify this question per user suggestions. Thanks!
Use conda environments and install Python into those environments. This ignores, by design, the versions of Python that are installed at the system level. Also, what's installed into environments (other than the one you're in) is ignored. As somebody who uses conda for all Python development, I don't even know what I have installed in /usr/local/bin/ or other places. Everything is in /path/to/miniconda3/envs/env_name_n. You don't have to uninstall other versions of Python you have elsewhere; the point of environments is to keep everything isolated.
What can I expect to change about how I write python after I install miniconda?
Is there anything I will need to do differently from what I am doing now when I write python code?
Use conda environments and use conda as your package manager. For almost every major project, it nearly a drop-in replacement for pip. Outside of managing dependencies, not much changes.
I am experiencing difficulty with installing Python correctly on my Mac (Fresh Install, I wiped my PC clean). I installed firstly
Home Brew
Xcode through App Store
Command line tools using command: "xcode-select --install"
On typing the following command:
which python3
Output is as follows:
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % which python
/usr/bin/python
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % which python3
/usr/bin/python3
upon running the following command which I was following off a blog from this site:
https://www.pyimagesearch.com/2016/12/05/macos-install-opencv-3-and-python-3-5/
brew install eigen tbb
Home brew installed python 3.9 dependencies so no when I type the following commands:
brew list python3
I get the following output:
gaurangsmacbookpro#Gaurangs-New-MacBook-Pro ~ % brew list python3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/2to3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/2to3-3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/easy_install-3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/idle3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/idle3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pip3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pip3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pydoc3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/pydoc3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3-config
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3.9
/usr/local/Cellar/python#3.9/3.9.1_8/bin/python3.9-config
/usr/local/Cellar/python#3.9/3.9.1_8/bin/wheel3
/usr/local/Cellar/python#3.9/3.9.1_8/Frameworks/Python.framework/ (3019 files)
/usr/local/Cellar/python#3.9/3.9.1_8/IDLE 3.app/Contents/ (8 files)
/usr/local/Cellar/python#3.9/3.9.1_8/lib/pkgconfig/ (4 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/bin/ (7 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/pip/ (480 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/setuptools/ (334 files)
/usr/local/Cellar/python#3.9/3.9.1_8/libexec/wheel/ (44 files)
/usr/local/Cellar/python#3.9/3.9.1_8/Python Launcher 3.app/Contents/ (16 files)
/usr/local/Cellar/python#3.9/3.9.1_8/share/man/ (2 files)
The contents of my ~./bash_profile file are as follows:
# Add Homebrew's executable directory to the front of the PATH
export PATH=/usr/local/bin:$PATH
export PATH="/usr/local/sbin:$PATH"
export PATH="/usr/local/opt/python#3.8/bin:$PATH"
What am I doing wrong? I just want a fresh clean install of Python 3.8 not 3.9 as there is no support for TF Lite on 3.9 yet. Can someone help me fix this mess? I will be very grateful!
I'm a bit late at the game, but I hope this will be useful to somebody. The best solution here is to use a virtual environment, but for the sake of argument and because it is a little bit more involved, I won't discuss that as an option here and will assume you just want a specific version installed on your machine where you can just add all the dependencies you need.
From looking at the page OP was using, the easiest solution here is probably to use a distribution like Anaconda if you are particularly interested in OpenCV or that type of libraries. Then it's just a case of setting up your path like export PATH=$HOME/anaconda/bin:$PATH or whatever path you used to install Anaconda with your shell of choice.
But let's imagine that Anaconda doesn't help you and you still need a specific version of Python, and you figure 3.8.2 is good enough for your needs and a newer version wouldn't work. Using the Python version that comes with the Xcode command line tools is probably a bad idea, just like using the system installed Python is. That version is really meant to support the Apple toolchain and while I've never run into any issues with it, I would not recommend it.
Next option is is to use homebrew. Great option since most people will already be using it, and you might already have python installed as part of a dependency but maybe like OP it's not the version you were hoping for. You can still make it work by installing an older version like so brew install python#3.8 however, it comes with a few caveats. Whenever one of the python dependencies is updated, you might need to unlink and relink it so that you can continue to use the correct version. Believe me, it's no way to live.
What I've found works (in combination with virtual environments is to use pyenv. Install it through homebrew, set your shell as described in the install doc and use it to install the python version you want (very easy) and then set the global interpreter to be whatever version you want and your system will stick to it until you set it to something else. It also works great in combination with tools like pipenv which allow you to manage virtual environments.
I'm getting familiar with nix by using it instead of brew on my Mac. I like it pretty much so far, but there is an issue with python environment. I can't figure out how to configure it properly.
I'm using powerline status, which is a python package itself, but also available as derivation (nixpkgs.python37Packages.powerline). So, I installed it together with python and some other packages:
# file: python.nix
with import <nixpkgs> {};
python3.withPackages (ps: with ps; [ pip pipenv powerline ipython ])
This is working as expected.
However, I'm also using some additional segments for powerline (ex. powerline-gitstatus), which are not available as derivations. So, I just installed them via pip install --user .... And then set PYTHONPATH=$HOME/.local/lib/python3.7/site-packages.
This is also working.
Apart from that I need to have awscli, which is a python package and available as standalone derivation (not nixpkgs.python37Packages.awscli), so I can't install it as a part of my custom python derivation, as far as I understand.
I tried to get it with nix-env -i awscli, but it didn't work together with set PYTHONPATH.
I tried to get it with pip install --user awscli, but it didn't work also.
In both cases it crashed because of lack of dependencies or version incompatibilities.
So, my question is: how to setup global python environment properly with nix?
I guess following well-documented very convenient approach could help in your case also.
How to consume python modules using pip in a virtualenv like I am used to on other Operating Systems ?
10.09.2020 Update:
Developing Python with Poetry & Poetry2nix: Reproducible flexible Python environments describes an other working with python on NixOS approach.
Some time ago when i wanted to install a package using Conda in Anaconda python distribution i saw that Conda wants to update the python package from 2.7.10-0 to 2.7.10-1. It's the same python version (2.7.10 in this case).
Checking the channel's content I see there are multiple packages for the same python version:
python-2.7.10-0.tar.bz2 18.3M
python-2.7.10-1.tar.bz2 16.7M
python-2.7.10-3.tar.bz2 16.7M
...
So what is the difference between these builds and how can i prevent them to be updated?
What you're seeing there are build numbers.
They're usually used to fix a build of the same version of a package.
For example, imagine you have built this python version accidentally as a pydebug build. However, that's not what you want since it will lead to crashes in users of this package if they're not away that this is a pydebug build.
In this case you should rebuild the package (correctly this time), increment the build number and re-upload it.
So what is the difference between these builds?
You can't easily know the difference, unless Continuum provides a changelog for each build of python they provide (which I sincerely doubt).
To install a package with a specific build number you could do: conda install "python=2.7.10 0". The 0 means the build number.
I don't know if this syntax is officially supported, however it worked the last time I used it.
how can i prevent them to be updated?
First I would have to know what is your workflow.
If you're asking about the command-line, I don't think that is possible.
If you're asking about using environment.yml files you can pin a package to a specific version (including the build number) using a similar syntax of conda install.