I am following the tutorial on MLFlow website. I was able to run the train.py and mlflow ui worked fine. Packaging the project tries to use env variable MLFLOW_CONDA_HOME but can't find conda.
I have tried setting the variable to the path of anaconda3/condabin but it doesn't seem to find my executable. This is the error I get:
ERROR mlflow.cli: === Could not find Conda executable at /anaconda3/condabin\bin/conda. Ensure Conda is installed as per the inst
ructions at https://conda.io/docs/user-guide/install/index.html. You can also configure MLflow to look for a specific Conda executable by setting the MLFLOW_CONDA_HOME environment variable
to the path of the Conda executable ===
Adding \bin/conda at the end of my path seems to be the problem, I am not sure why mlflow is doing it. I even tried setting it to my python.exe in my conda env, but no luck. I can't find bin/conda folder in my Anaconda folder anywhere.
I resolved this by running it from Anaconda Prompt. Make sure mlflow is installed in anaconda first as well, nothing else. But the problem then is that it's not well compatible on windows, you would need to split into two steps, activate the conda environment and then run with --no-conda as mentioned here https://github.com/mlflow/mlflow/issues/2674
MLflow 1.5 was just released today.
It doesn't specifically mention it in the github notes, but I had the same issue, where it affixed \bin/conda, and now it doesn't do that anymore.
If you don't have conda environment then you can execute the following command from your terminal
mlflow run <enter your local directory name> --no-conda -P alpha=0.5
This should solve the issues with the environment variable.
I solved the issue by removing the MLFLOW_CONDA_HOME environment variable alltogether. Make sure you have added the path to the conda executable to your PATH variable.
Here is one possible solutions (the fastest one, in my opinion).
Key points:
The project virtual environment should be created with conda.
Use pip to install MLFlow.
Follow the steps for Windows:
Install miniconda (in my case, version 3)
Set conda bat file (installation path + condabin dir + conda.bat) in PATH
Create your project without virtual environment (in my case, I set in PyCharm conda instead of venv and it did not create any virtual environment, just added some external libraries), at least not in the project directory.
Create conda virtual environment manually in the project directory. In your project directory, execute conda create -n venv and follow the instructions (I used default for all the questions there).
Open a terminal and activate conda virtual environment. If you use PyCharm, you will be positioned properly, otherwise just prompt yourself in the project directory. Execute conda activate venv where venv is my virtual environment created at point 4.
Execute python -m pip install mlflow
If you want to test it, you can try one of the tests from MLFlow. E.g., you can use mlflow run https://github.com/mlflow/mlflow-example.git -P alpha=5.0
In my case, it worked.
If you're using mlflow.pyfunc.spark_udf and get an error saying Could not find Conda executable conda then try to define the environment variable MLFLOW_CONDA_HOME in spark-env.sh as Spark doesn't recognize variables defined elsewhere. Also make sure to use the absolute path for the Conda executable.
I faced this issue within a kubernetes deployment with miniconda3 as the base image. Fixed this by setting the MLFLOW_CONDA_HOME env variable to "/opt/conda/"
Related
I am having trouble adding conda to my environment variables on windows. I installed anaconda 3 though I didn't installed python, so neither pip or pip3 is working in my prompt. I viewed a few post online but I didn't find anything regarding how to add conda to my environment variables.
I tried to create a PYTHONPATH variable which contained every single folder in Anaconda 3 though it didn't worked.
My anaconda prompt isn't working too. :(
so...How do I add conda and pip to my environment variables or path ?
You can access to conda and pip directly in their directory :
On Windows : %ANACONDA_INSTALL_DIR%\Scripts, you will find conda.exe and pip.exe
On Linux : $ANACONDA_INSTALL_DIR/bin, there are conda and pip executable files.
Where $ANACONDA_INSTALL_DIR is the directory of your Anaconda installation.
To add Anaconda to your path, you can add this folder to your PATH.
For Linux, you can add this line to yout .bashrc file
export PATH=$ANACONDA_INSTALL_DIR/bin:$PATH
As said #BcK, reinstalling Anaconda and choosing to add Anaconda to your path is also a way to do this. This option makes a backup of your .bashrc file and add the export line.
EDIT
To open a terminal with the conda environment activated on Windows, I usually create a desktop shortcut with the following target :
%windir%\System32\cmd.exe "/K" %ANACONDA_INSTALL_DIR%\Scripts\activate.bat %ANACONDA_INSTALL_DIR%
Thanks guys for helping me out. I solved the problem reinstalling anaconda (several times :[ ), cleaning every log and resetting the path variables via set path= in the windows power shell (since I got some problems reinstalling anaconda adding the folder to PATH[specifically "unable to load menus" or something like that])
I have created a virtual enviroment using virtualenv command with a standard procedure. Now, when I activate it or deactivate, the output of which pip or which python gives me same output /usr/local/bin/pip. However there is copies of python and pip commands in my vitrual enviroment directory - I have found them there. What might be the problem?
Did you move the virtual environment folder to a different name? The original path to the virtual environment is written into the generated activate script so if you move the environment activate will set your path to the old path of the virtual environment.
To fix this run virtualenv --relocatable $YOUR_VIRTUALENV_PATH then modify the VIRTUAL_ENV variable in the activate script to point to your new location.
I think it happen when I moved the environment folder to a different location.
I solved it by reinstalling virtualenv and creating a new environment
Conda env is activated using source activate env_name.
How can I activate the environment in pycharm ?
open
pycharm/preferences/project/Project Interpreter
And check existing interpreter. Conda environments may already be listed there.
If not exists, you can create a new conda environment with "Create Conda Env" button
If you are looking for a specific conda environment you can use 'add local'. When you click 'add local' you will input conda environment path + /bin/python
You can list all conda environment in your system with following commnad.
>>conda info --env
# conda environments:
#
tensorflow * /Users/username/miniconda3/envs/tensorflow
you can chose the approach best fits your needs.
The best PyCharm specific answer is this one by wasabi (below).
In general though, if you want to use an interpreter from within a Conda environment then you can change the location of the interpreter to point to the particular environment that you want to use e.g. /home/username/miniconda/envs/bunnies as mentioned in this comment.
However, as mentioned in this answer by Mark Turner, it is possible to have a shell script executed when activating an environment. This method will not run that shell script, but you can follow his workaround if you need that shell script run:
open a conda prompt
activate the environment
run pycharm from the conda prompt
How about environment.yml
Pycharm can create a new conda environment indeed. Unfortunately, until this issue is fixed, it won't offer environment.yml support, which means it won't install the dependencies declared there.
When working on a project based on such a file, you need to create / update the dedicated env manually on your machine:
conda env create -n <my-project>
Then remember to update each time environment.yml changes (from you or upstream).
conda env update -n <my-project>
Not ideal
As mentioned in one of the comments above, activating an environment can run scripts that perform other actions such as setting environment variables. I have worked in one environment that did this. What worked in this scenario was to:
open a conda prompt
activate the environment
run pycharm from the conda prompt
Pycharm then had access to the environment variables that were set by activating the environment.
I had the same problem i am on windows 10 professional 64 bit
my solution was to start Pycharm as adminstrator and it worked
Go to settings at the top right corner of the PyCharm IDE.
Go to Project:{Your Project Name}->Python Interpreter
Go to the settings inside here and click add:
In Add Python Interpreter select conda env
Select existing environment and click on your required conda environment path from the dropdown menu OR add the path of the python.exe file in your conda environment. As a reference, I am adding the path for my windows10 system: C:\Users\maria\AppData\Local\Continuum\anaconda3\envs<mycondaenv>\python.exe It can vary for your system based on installation configs.
It seems important to me to know, that setting project interpreter as described in wasabi's comment does not actually activate the conda environment.
I had issue with running xgboost (that I installed with conda) inside PyCharm and it turned out that it also need some folders added to PATH. In the end I had to make do with an ugly workaround:
Find out what are the additional folders in PATH for given environment (with echo %PATH% in cmd)
In the file I wish to run put to the top before anything else:
import os
os.environ["PATH"] += os.pathsep + os.pathsep.join(my_extra_folders_list)
I know this is not at all proper solution, but i was unable to find any other beside what Mark Turner mentioned in his comment.
To use Conda environment as PyCharm interpreter
activate Conda environment from Conda navigator
open PyCharm from the navigator tool list
in Conda Add interpreter section choose existing Conda environment and it automatically recognises the path of that environment's python.exe file
First , select Interpreter setting ... in right bottom of Pycharm.
Then choose python.exe from your desired conda environment.
My environment path is : C:\Users\javadsh\anaconda3\envs\tf-gpu\python.exe
Go to Pycharm -> Preferences -> Project Interpreter. At the top left of the packages table there is a plus sign, minus sign, a green circle and an eye; uncheck the green sign; that will let you have access to the packages while using conda environment.
I am new to Anaconda Python and I am setting up a project in Sublime Text 3. I have installed Anaconda and created a virtual environment using:
conda create -n python27 python=2.7 anaconda
conda create -n python35 python=3.5 anaconda
I am having trouble setting up the Virtualenvs plugin for SublimeText 3.
When I try, it asks me for a virtualenvs path which I give:
~/users/../anaconda/envs/python27
Then it asks for what I'm assuming is a path to a python distribution because it lists file paths for the system versions of python -- but not the anaconda install.
I have no real desire to use the plug in, I just want to be able to use both versions of python. Could I use a project settings file to set the version of python instead?
If you activate the environment you're interested in, you can find that answer in the environment variables.
on MacOS/Linux:
source activate python35
echo $CONDA_PREFIX
on Windows:
conda activate python35
echo %CONDA_PREFIX%
You can also run conda info --envs, and that will show the paths to all your environments.
To get the path to the instance of python being used by a particular environment, do the following:
on MacOS/Linux:
source activate python35
which python
on Windows:
conda activate python35
where python
That should return the path you're looking for.
You can run the command conda info.
This will output something like this:
envs directories : C:\Users\Geo\.local\Miniconda3\envs
C:\Users\Geo\.conda\envs
C:\Users\Geo\AppData\Local\conda\conda\envs
I have installed conda at C:\Users\Geo\.local\Miniconda3.
Then with the command conda info -e you get the location of each environment.
(base) C:\Users\Geo>conda info -e
# conda environments:
#
miniconda2 C:\Users\Geo\.conda\envs\miniconda2
base * C:\Users\Geo\.local\Miniconda3
anaconda3 C:\Users\Geo\.local\Miniconda3\envs\anaconda3
ml C:\Users\Geo\.local\Miniconda3\envs\ml
Your environments are located in Anaconda3\envs\<yourEnv_directory>\
To answer your question the folder for your python binaries and packages for the environment are located in ~Anaconda_installation_folder~/envs/python35.
But I cannot really say if that solves your problem. Normally you just switch to your environment source activate python35 and then type python. This will automatically give you the "right" python executable. So if you have a package you could use:
source activate python35
python setup.py install
# Now it is installed in your python35 environment
source activate python27
python setup.py install
# Now it is also installed in your python27 environment
Just change python setup.py install to what you want to do in the environment. I don't have any experience using Sublime Text and what you mean with build system. But you can always use something like tox which automates a lot of these manual builds.
None of the other windows solutions worked for me so I'm providing my own. Activate the environment inside anaconda prompt, then issue the command 'where python' and you'll likely see multiple results but one of them, most likely the top one, is the one you're after. For me, my environments were located in AppData\Local... which is not what anyone else had mentioned but the best solution is to use 'where python' which should result in an answer regardless of how you've installed Anaconda.
For me, with default anaconda settings and Windows 10, the path that displays after activating the environment is C:\Users\usrname>, but it does not contain an Anaconda3 folder. However, it contains a .conda folder that contains an environments.txt file that lists all conda environments and their locations. By default, the environment folders were stored in:
C:\Users\usrname\AppData\Local\conda\conda\envs\EnvName
None of the above worked. In the end, I found mine at:
c:\Users\myusername\venvs\test1-V7fphpR9\
Open your c directory
Go to Users and then Open your naming folder (in my case Suman Biswas)
Fiend Anaconda3 folder
Fiend the folder envs in Anaconda3
Now you can see your virtual environment
enter image description here
enter image description here
enter image description here
On Windows 10 x64 and Anaconda3, the python interpreter for a newly created environment "my_env" would appear here:
C:\ProgramData\Anaconda3\envs\my_env\python.exe
Or here:
C:\Users\[username]\AppData\Local\conda\conda\envs\my_env
Check both places.
Update 2020-07-17
If Anaconda is installed as Administrator, then the default is one location.
If Anaconda is not installed as Administrator, then it appears in a different location.
I've installed Anaconda and created two extra environments: py3k (which holds Python 3.3) and py34 (which holds Python 3.4). Besides those, I have a default environment named 'root' which the Anaconda installer created by default and which holds Python 2.7. This last one is the default, whenever I launch 'ipython' from the terminal it gives me version 2.7. In order to work with Python 3.4, I need to issue the commands (in the shell)
source activate py34
ipython
which change the default environment to Python 3.4. This works fine, but it's annoying since most of the time I work on Python 3.4, instead of Python 2.7 (which I hold for teaching purposes, it's a rather long story). Anyway, I'll like to know how to change the default environment to Python 3.4, bearing in mind that I don't want to reinstall everything from scratch.
If you just want to temporarily change to another environment, use
source activate environment-name
ETA: This may be deprecated. I believe the current correct command is:
source conda activate environment-name
(you can create environment-name with conda create)
To change permanently, there is no method except creating a startup script that runs the above code.
Typically it's best to just create new environments. However, if you really want to change the Python version in the default environment, you can do so as follows:
First, make sure you have the latest version of conda by running
conda update conda
Then run
conda install python=3.5
This will attempt to update all your packages in your root environment to Python 3 versions. If it is not possible (e.g., because some package is not built for Python 3.5), it will give you an error message indicating which package(s) caused the issue.
If you installed packages with pip, you'll have to reinstall them.
Overview
Some people have multiple Conda environments with different versions of Python for compatibility reasons. In this case, you should activate the desired default environment in the shell initialization file (e.g., .bashrc, .zshrc). With this method, you can preserve the versions of Python you use in your environments.
The following assumes environment_name is the name of your environment
Mac / Linux:
Edit your bash profile so that the last line is conda activate environment_name. In Mac OSX this is ~/.bash_profile, in other environments this may be ~/.bashrc
Example:
Here's how I did it on Mac OSX
Open Terminal and type:
nano ~/.bash_profile
Go to end of file and type the following, where "p3.5" is my environment:
conda activate p3.5
Exit File. Start a new terminal window.
Type the following to see what environment is active
conda info -e
The result shows that I'm using my p3.5 environment by default.
For Windows:
Create a command file (.cmd) with activate environment_name and follow these instructions to have it execute whenever you open a command prompt
Create a batch file command, e.g. "my_conda.cmd", put it in the Application Data folder.
Configure it to be started automatically whenever you open cmd. This setting is in Registry:
key: HKCU\SOFTWARE\Microsoft\Command Processor
value: AutoRun
type: REG_EXPAND_SZ
data: "%AppData%\my_conda.cmd"
from this answer: https://superuser.com/a/302553/143794
Under Linux there is an easier way to set the default environment by modifying ~/.bashrc or ~/.bash_profile
At the end you'll find something like
# added by Anaconda 2.1.0 installer
export PATH="~/anaconda/bin:$PATH"
Replace it with
# set python3 as default
export PATH="~/anaconda/envs/python3/bin:$PATH"
and thats all there is to it.
For windows Anaconda comes with Anaconda Prompt which is a shortcut to cmd and can be used run conda commands without adding anaconda in PATH variable.
Find the location of it, copy and rename the copy (say myenv_prompt). Right click myenv_prompt and select properties in the context menu.
The Target form of Properties window should already be filled with text, something like %windir%\system32\cmd.exe "/K" C:\Users\xxx\AppData\Local\Continuum\Miniconda3\Scripts\activate.bat C:\Users\xxx\AppData\Local\Continuum\Miniconda3\
There are three parts of this command 1)start ...\cmd.exe 2)run ...\acitvate.bat with environment 3)...\Miniconda3\
Change 3rd part to path of the environment (say myenv) you want as default i.e. fill the Target form something like %windir%\system32\cmd.exe "/K" C:\Users\xxx\AppData\Local\Continuum\Miniconda3\Scripts\activate.bat C:\Users\xxx\AppData\Local\Continuum\Miniconda3\envs\myenv
Now myenv_prompt will act as shortcut to start cmd with myenv as the default environment for python. This shortcut you can keep in start menu or pinned in taskbar.
One advantage of this method is that you can create a few shortcuts each having different environment as default environment. Also you can set the default folder by filling Start in form of the Properties window
Hope this helps
PS:It is not required to find Anaconda Prompt and can be done by changing target of any shortcut. But you will require to know path of cmd.exe and activate.bat
Just activate your py34 environment when you load your terminal/shell.
If you use Bash, put the line:
conda activate py34
in your .bash_profile (or .bashrc):
$ echo 'conda activate py34' >> ~/.bash_profile
Every time you run a new terminal, conda environment py34 will be loaded.
The correct answer (as of Dec 2018) is... you can't. Upgrading conda install python=3.6 may work, but it might not if you have packages that are necessary, but cannot be uninstalled.
Anaconda uses a default environment named base and you cannot create a new (e.g. python 3.6) environment with the same name. This is intentional. If you want your base Anaconda to be python 3.6, the right way to do this is to install Anaconda for python 3.6. As a package manager, the goal of Anaconda is to make different environments encapsulated, hence why you must source activate into them and why you can't just quietly switch the base package at will as this could lead to many issues on production systems.
Change permanent
conda install python={version}
Change Temporarily
View your environments
run conda info --envs on your terminal window or an Anconda Prompt
If It doesn't show environment that you want to install
run conda create -n py36 python=3.6 anaconda for python 3.6 change version as your prefer
Activating an environment (use Anaconda prompt)
run activate envnme envnme you can find by this commandconda info --envs as a example when you run conda info --envs it show
base * C:\Users\DulangaHeshan\Anaconda3
py36 C:\Users\DulangaHeshan\Anaconda3\envs\py36
then run activate py36
to check run python --version
In Windows, it is good practice to deactivate one environment before activating another.
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html?highlight=deactivate%20environment
If you want Anaconda Navigator to default to Virtual Env you created, go to file > Preference and select default conda env in drop down lint:
If you want Anaconda command automatically opens to virtual env without having to type activate envName, do this:
Right click on conda shortcut > go to properties and change the Target to something like this:
%windir%\System32\cmd.exe "/K" C:\Anaconda\Scripts\activate.bat C:\Anaconda\envs\p37
Optionally you can set your default working dir as well, like I did in snapshop below:
gl
On Windows, create a batch file with the following line in it:
start cmd /k "C:\Anaconda3\Scripts\activate.bat C:\Anaconda3 & activate env"
The first path contained in quotes is the path to the activate.bat file in the Anaconda installation. The path on your system might be different. The name following the activate command of course should be your desired environment name.
Then run the batch file when you need to open an Anaconda prompt.
Here is the solution I found for autoactivating my preferred environment on a Windows 10 system:
Open anaconda prompt & use 'conda env list' to find the location of the environment you wish to use.
Go to the start menu, right-click 'Anaconda Prompt' and go to file location.
Create a copy of this shortcut file
Open its properties & change the target to the location of your preferred environment.
Now every time you open anaconda prompt through this shortcut it will automatically load your chosen environment.
activate.py is hardcoded to emit conda activate base\n into your shell profile when you evaluate the shell hook produced by conda shell.zsh hook.
you can suppress this hardcoded "auto-activate base" via:
conda config --set auto_activate_base false
then, in ~/.zshrc, ~/.bashrc or wherever you source your shell profile from, you can append the following (after the conda shell hook) to explicitly activate the environment of your choosing:
conda activate py34
I wasn't satisfied with any of the answers presented here, since activating an environment takes a few seconds on my platform (for whatever reason)
I modified my path variable so that the environment I want as default has priority over the actual default.
In my case I used the following commands to accomplish that for the environment "py35":
setx PATH "%userprofile%\Anaconda3\envs\py35\;%PATH%"
setx PATH "%userprofile%\Anaconda3\envs\py35\Scripts;%PATH%"
to find out where your environment is stored, activate it and enter where python.
I'm not sure yet if this approach has any downsides. Since it also changes the default path of the conda executable. If that should be the case, please comment.
For Jupyter and Windows users, you can change the Target path in your Jupyter Notebook (anaconda3) shortcut from C:\Users\<YourUserName>\anaconda3 to C:\Users\<YourUserName>\anaconda3\envs\<YourEnvironmentName>
you could do the same thing for the Anaconda Prompt..etc.
After changing the path you can check your active environment by opening a terminal in Jupyter and run conda info --envs.
I got this when installing a library using anaconda. My version went from Python 3.* to 2.7 and a lot of my stuff stopped working.
The best solution I found was to first see the most recent version available:
conda search python
Then update to the version you want:
conda install python=3.*.*
Source: http://chris35wills.github.io/conda_python_version/
Other helpful commands:
conda info
python --version
Create a shortcut of anaconda prompt onto desktop or taskbar, and then in the properties of that shortcut make sure u modify the last path in "Target:" to the path of ur environment:
C:\Users\BenBouali\Anaconda3\ WILL CHANGE INTO
C:\Users\BenBouali\Anaconda3\envs\tensorflow-gpu
preview
and this way u can use that shortcut to open a certain environment when clicking it, you can add it to ur path too and now you'll be able to run it from windows run box by just typing in the name of the shortcut.
Tried both source activate default_3_9 and source conda activate default_3_9
but worked conda activate default_3_9
I'm trying to update Anaconda in order to use Python 3.10.4 and then Spyder 5.3.2. Actually, I wanted to set the Python interpreter used by Pycharm inside the Spyder console but it required the newest Spyder version. I didn't try all the possible solutions (it's pending for me to use the window batch and modifying path solutions given here) but:
Since I couldn't update the Anaconda base due to the well-known error on the "Solving environment". Then Python and Spyder remain the same.
Creating a new env allows to get the last Python and then his newest Spyder version but it doesn't actualize the Anaconda shortcuts and even the Anaconda navigator if you set it to this new env still has some inconsistencies like keeping the older Spyder version in his menu.
Besides, on point 2, changing the shortcuts target path doesn't work for me.
Finally, I create a new shortcut of the Spyder file from the Scripts folder inside the environment directory ( C:\Users<userName>>\Anaconda3\envs<EnvName>\Scripts )
I couldn't use the default Anaconda shortcuts but I have what I wanted and quick access.