I am creating a .SH script for setting up automatically my dev environment in Azure ML, according to this:
https://learn.microsoft.com/en-gb/azure/machine-learning/how-to-customize-compute-instance
The script looks like this:
#!/bin/bash
set -e
# https://pypi.org/project/azure-ai-ml/
# Requires: Python <4.0, >=3.7
# This script creates a custom conda environment and kernel based on a sample yml file.
conda env create python=3.10
#conda env create -f env.yml
echo "Activating new conda environment"
conda activate envname
conda install -y ipykernel
echo "Installing kernel"
sudo -u azureuser -i <<'EOF'
conda activate envname
python -m ipykernel install --user --name envname --display-name "mykernelp310v2"
echo "Conda environment setup successfully."
pip install azure-ai-ml
EOF
my env looks like this:
name: p310v2
dependencies:
- python=3.10
- numpy
- matplotlib
- pandas
- scikit-learn
- pip:
-kaggle==1.5
When I check this document:
https://carpentries-incubator.github.io/introduction-to-conda-for-data-scientists/04-sharing-environments/index.html
I am confused between the dependencies section and the pip section. For example scikit-learn I could put in dependencies but also on the pip section, so whats the deal here?
dependencies are conda dependencies; listing them is equivalent to run conda install <deps>. For example conda installs numpy from https://anaconda.org/conda-forge/numpy .
pip means pip dependencies for pip install command; pip installs them from PyPI.org. For example, https://pypi.org/project/numpy/
The same for scikit-learn: https://anaconda.org/search?q=scikit-learn and https://pypi.org/search/?q=scikit-learn
See also https://stackoverflow.com/search?q=%5Bconda%5D+%5Bpip%5D+difference
I want to install a package which is just used in one project.
So I create a virtual env and install it
python -m venv .venv --system-site-packages
python -m pip install my_package
Then I want to debug it with jupyter.
The problem is It seems I have to either install another jupyter in my virtual env ( quite large):
pip install jupyter
jupyter notebook
or register this env as a kernel to user:
python -m ipykernel install --user --name my_test_env
in this case, this env will list in my global kernel list but I won't use if for any other project. And delete the project folder will leave a broken kernel in global jupyter.
Is there any way to register a kernel just for current project?
I have installed my needed packages (dependencies) in the global system environment instead of my virtual environment (virtualenv) because i have used the command pip install <package-name> outside of the virtual environment.
So i want to know how can i make a list out of them and install them in any of my virtualenvs?
This is useful in a case that anyhow you have installed some packages (dependencies) in your global system environment instead of your virtualenv, by mistake.
For example by using the command "pip install", not "pipenv install" (outside of virtual environment).
So the solution is:
In the global system environment (outside of any virtualenv), create a "requirements.txt" file from all of your installed packages:
$ pip freeze > requirements.txt
Import installed dependencies from the created "requirements.txt" file to the Pipfile by running the below command in the root that the above created file "requirements.txt" exist; but first check:
a) If Pipfile not exist:
$ pipenv install
b) If Pipfile do exist (i.e. already created virtualenv):
$ pipenv install -r requirements.txt
Then your package listing files "Pipfile" & "Pipfile.lock" would be updated and locked.
But I personally recommend that for avoiding this problem to happen, always use the command
$ pipenv install
instead of $ pip install.
You can create file in your global system environment the format of this file is like following:
my_backages.txt
$ pip install -U Flask-SQLAlchemy
$ pip install --upgrade
$ pip install flask
then you can use pipfile as next :
$ pip install -r my_backages.txt
I was creating a new virtual environment on Ubuntu 20.04:
$ virtualenv my_env
But it gave an error:
ModuleNotFoundError: No module named 'virtualenv.seed.embed.via_app_data'
Other info:
$ virtualenv --version
virtualenv 20.0.17 from /usr/lib/python3/dist-packages/virtualenv/__init__.py
#yushulx
I also ran into the same issue. I installed both via pip3 and via sudo apt install python3-virtualenv and it gave me an error but after I ran pip3 uninstall virtualenv I could create a virtualenv without issue
Try to create the virtual environment using directly venv module
python3 -m venv my_env
To fix this on Ubuntu 20.04, I had to uninstall virtualenv from the system: apt remove python3-virtualenv, and reinstall it using pip: pip install --user virtualenv --force-reinstall. I had errors about dependencies conflicts, I fixed them by calling pip install --user ${package} --force-reinstall for every package involved.
virtualenv is installed by default with python itself and when you install virtualenv via pip3 and try to create virtual environment using pipenv you will get this error:
ModuleNotFoundError: No module named 'virtualenv.seed.embed.via_app_data
check the version of installed virtualenv using apt list --installed
mine is:
python3-virtualenv/focal,focal,now 20.0.17-1 all [installed,automatic]
with the installed virtualenv by pip3
min is :
virtualenv 20.4.0
default installation of virtualenv is different with pip3 installed virtualenv
so when you try to create a virtual environment using pipenv for example installing django in a directory home/user/djano with pipenv install django~=3.1.5 you will get that error
the solution is remove installed virtualenv using pip3 uninstall virtualenv and use the default installation of virtualenv this time when you create virtual environment with pipenv it will create it successfully.
I want to have virtualenvwrapper. On Debian 10 testing I did:
apt remove python3-virtualenvwrapper # not purge, I want no changes in ~/.virtualenvs/
apt purge python3-virtualenv
/usr/bin/python3.8 -m pip install --force-reinstall virtualenvwrapper
/usr/bin/python3.8 -m pip install --force-reinstall virtualenv==20.0.23
.24 no longer works. I hope it will be solved sometimes...
EDIT 2021.01: I have changed my stack to: pyenv + pyenv-virtualenvwrapper + poetry. Ie. I use no apt or pip installation of virtualenv or virtualenvwrapper, and instead I install pyenv's plugin pyenv-virtualenvwrapper. This is easier way.
If someone encounters this problem inside existing env (when for example using pyenv) you can also use command below (found on GitHub when tried to fix poetry virtual env installation):
pip install --force-reinstall virtualenv
When I installed virtualenv via pip3, it failed to run virtualenv command. Then I changed the installation via:
sudo apt install python3-virtualenv
The virtualenv command can normally work.
I too had this issue. What I found is it is a permissions issue. For some unknown reason ownership of my home directory was off. I did a chown -R for the directory I was using for my project making myself the owner of my own directory and now everything works as normal.
I also had same issue, seems installed version has different user level so I followed their doc and below one work for me:
python3 -m virtualenv --help
To create new environment:
python3 -m virtualenv my_env
I also faced the same issue but after removing virtualenv which was installed with pip3, I could get rid of this error. Uninstall virtualenv with below command (don't forget to use sudo)
sudo pip3 uninstall virtualenv
After this, virtualenv command works totally fine.
It means that there are two virtualenv in your system.
One is "pip install" by sudo or root, the other may be installed by apt(if you are using ubuntu os)
Just uninstall one of them and the error should be fixed.
I fixed this error by removing all virtualenv and virtualenvwrapper related packages on system and reinstall the virtualenv and virtualenvwrapper with pip with below command(as i use ubuntu, so below only show apt)
remove all packages shown in below result
apt list --installed | grep virtualenvwrapper
apt list --installed | grep virtualenvwrapper
install virtualenv virtualenvwrapper with pip
pip install virtualenvwrapper virtualenvwrapper
set ~/.zshrc
export WORKON_HOME=$HOME/.virtualenvs
export PROJECT_HOME=$HOME/amd
export VIRTUALENVWRAPPER_SCRIPT=/home/robot/.local/bin/virtualenvwrapper.sh
export VIRTUALENVWRAPPER_PYTHON=$(which python3)
source /home/robot/.local/bin/virtualenvwrapper.sh
When we use pip3 or python3 to install virtualenv then I got that error too. I had to run each time to create virtualenv (my_env is virtual environment name)
python3 -m virtualenv my_env
But if I install it using
sudo apt install virtualenv
Then virtualenv command works fine.
virtualenv my_env
I currently use a Mac. I recently created a new python virtual environment and installed jupyter. When I activate jupyter notebook within the virtual environment, it says it cannot find any python kernels. I have another virtual environment that also has jupyter installed and it works perfectly fine. Can anyone help? Also, I'm not sure where the Kernels are even located on my machine. Library/Jupyter only has a runtime folder.
I need to do 2 things to get this to work:
conda install -y ipykernel
python -m ipykernel install --user --name myenv
Kind of frustrating and I never see it mentioned much in the docs. Jupyter set up remains a pretty painful process in my opinion. This is from beginning to end, if it helps anyone:
conda create -n myenv python=3.7
conda activate myenv
conda install -y -c conda-forge jupyterlab
conda install -y -c anaconda jupyter
conda install -y ipykernel
python -m ipykernel install --user --name myenv
Try installing nb_conda in your environment, by going to your command line conda activate your environment and conda install nb_conda. Make sure you also have ipykernel installed in your environment, then deactivate and reactivate your environment and try again.