I am using miniconda to create several environments, all of which with Python 3.7. I found that for each env, the installed packages go to .local path, which means for different envs of Python 3.7, they share the same collections of packages.
For example:
Python 3.7.7 (default, May 7 2020, 21:25:33) [GCC 7.3.0] :: Anaconda, Inc. on linux Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__file__ '/home/my_name/.local/lib/python3.7/site-packages/numpy/__init__.py'
I suppose the installed packages should be in miniconda's directory, not my local .local directory.
How are you installing the packages? Are you using ‘conda env create’ for each new environment? You may need to check that the ‘base’ conda environment is running on your terminal, maybe you are using the default python interpreter instead of miniconda.
Also, if you are don’t have a specific reason to use conda, and just want it to manage virtual environments and packages, I would suggest that you try ‘pipenv’ instead. I found it to be simpler than conda and isolates pretty well the virtual environments.
Related
After installing miniconda, my python modules stopped working, throwing ModuleNotFoundError. From what I can tell, miniconda changed my default environment settings. I checked both .bash_profile and .bashrc and updated the files to give conda the lowest priority. This fixed my default python version but didn't fix any of the broken modules.
Next I checked my PYTHONPATH with python3 -c "import sys;print(sys.path)". I discovered that the PYTHONPATH consisted entirely of conda python paths instead of the python version I had called. For reference, my default python version should be 3.8 (now set in .bashrc), and the conda version is 3.9.
['', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python39.zip', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/lib-dynload']
I then manually changed my PYTHONPATH in the .bashrc file to include the appropriate library paths. After reloading .bashrc:
['', '/Users/Ghoti/venv/3.8/lib/python3.8/site-packages', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.8/lib/python3.8/site-packages', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python39.zip', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9', '/Library/Developer/CommandLineTools/Library/Frameworks/Python3.framework/Versions/3.9/lib/python3.9/lib-dynload']
My modules now work! However, I haven't been able to figure out how to stop the conda 3.9 libraries from being appended to my PYTHONPATH. In addition, my printed python version is wrong.
Ghoti$ python --version
Python 3.9.6
Ghoti$ which python
/Users/Ghoti/venv/3.8/bin/python
I was able to "fix" my ModuleNotFoundError problem. However, the solution is only temporary. If I ever need to switch python version/environment, I'll have to go through the process again. I'd like to figure out what is overriding my PYTHONPATH, causing it to call conda 3.9 libraries, and fix the python version irregularity. I've considered that there might be a script/process running in the background, but I haven't found any related to conda/miniconda. I've also been looking for a python setting/config file. No luck. Any suggestions on where I should look?
Edit - Did some more digging. It looks like my version 3.8 python executable was entirely overwritten, and the only existing python installation that is version 3.9.6 is in my "/usr/bin". The two conda environments have versions "3.9.12" and "3.8.13". I feel more confident the issue isn't due to conda, but unsure what could have caused the problem.
Final Edit
I don't think the problem was miniconda. I did start having problems within a few days of using miniconda and I original assumed that it just took me a while to notice the issues. However, I now think that my virtual environment was created using a shared python. Problems were noticed on the same day that I connected to network. The shared python version changed, and that broke my environment. I don't have a solution to salvage the broken environment, but rebuilding it from scratch shouldn't take too long.
Sounds like you only want to use conda when you explicitly need it, in other words, the default Python is the system Python.
If that's the case, you should disable the auto-activation of the base environment:
conda config --set auto_activate_base false
<restart shell>
Now you'll need to explicitly activate the conda environment before you can use the conda Python:
$ python
Python 3.10.6 (main, Aug 11 2022, 13:49:25) [Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
$ conda activate base
(base) $ python
Python 3.9.12 (main, Jun 1 2022, 06:36:29)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
I input python in cmd and it shows a warning:
C:\Users\sky>python
Python 3.8.8 (default, Apr 13 2021, 15:08:03) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Warning:
This Python interpreter is in a conda environment, but the environment has
not been activated. Libraries may fail to load. To activate this environment
please see https://conda.io/activation
Type "help", "copyright", "credits" or "license" for more information.
>>>
I cannot use pip install any packages, I thought maybe that's the problem. So I checked the information at the link:https://conda.io/activation
and followed the instruction:
c:\Anaconda3\Scripts\activate base
It said 'system cannot find the path'. I searched for the files in that path and found two of them are named 'activate'. So I input
c:\Anaconda3\Scripts\activate
but still,'system cannot find the path'.
How to activate the conda environment? I really need some help. Thanks
In a clean Anaconda installation you should have condabin only, e.g.
C:\Anaconda3\condabin
on your system path. To activate the base environment from the command prompt you use
C:\> conda activate
(base) C:\>
This will add the following folders to your system path too:
C:\Anaconda3;
C:\Anaconda3\Library\mingw-w64\bin;
C:\Anaconda3\Library\usr\bin;
C:\Anaconda3\Library\bin;
C:\Anaconda3\Scripts;
Only now you you are ready to run python and import all your packages properly.
The same thing typically happens when you select Anaconda prompt from your programs menu.
I have installed Anaconda but still unable to use packages such as pandas and requests when running code on python.
When I input python on Terminal, it shows:
Python 3.6.1 |Anaconda 4.4.0 (x86_64)| (default, May 11 2017, 13:04:09)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
And I also created a virtual environment and activate before opening the python shell:
conda create -n my-env python=3.6 anaconda
However, when reopening and running codes on Python, it kept saying "No module named 'pandas' or "No module named 'requests'" Does anyone know what the problem with this code is? Besides, I am not if version of Python has anything to do with this error? Is it because I am using 3.6.2 Python instead of 3.6.1?
Probably Anaconda became the default Python installation on your system. Specially when you are using a new environment, you have to install the modules you want in this new environment.
Try conda install pandas and, after that, import pandas
I had python version 2.7.3 and i wanted to learn django so i installed django version 1.8.2 on my ubuntu 12.0.4 .
invivtus#invictus:~/bin$ python
Python 2.7.3 (default, Sep 26 2013, 20:08:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 8, 2, 'final', 0)
Then i read that best way yo work with django is to work on python version 3.3 so i installed python version 3.3.6 on my system where py is symbolic link pointing to /opt/python3.3/bin/python3.3
invictus#invictus:~/bin$ py
Python 3.3.6 (default, Jun 21 2015, 16:13:35)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
when I try and import django here i get error
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'django
I see django got installed my python 2.7 directory.
>>> import django
>>> django
<module 'django' from '/usr/local/lib/python2.7/dist-packages/django/__init__.pyc'>
How can i use this django with my 3.3.6 version. My default python version is 2.7.3
What are possible workout here?
Yes, each Python version has its own folder with installed packages. You'll have to install Django separately for Python 3.3. The same is true for every package that is not available by default.
(If you're using Python 3, why not go for the latest and greatest, 3.4?)
As some of the comments said, you should be using a virtualenv to isolate your environments. You would do it like this:
1) Ensure you have virtualenv installed. On Ubuntu for instance, that would be package virtualenv.
2) Create a new, empty environment. You choose which python version it will be like this:
virtualenv -p /usr/bin/python3.4 env
3) That created an env folder. Activate the newly created environment:
. env/bin/activate
This updates your paths so now, when you run python or pip from this shell, they will execute in the context of your virtualenv.
4) Update the virtualenv (optional)
pip install -U pip
5) Install whatever packages you need. The recommended way is to have a requirements.txt file at the root of your project. You would pull them this way:
pip install -r myproject/requirements.txt
That's it. Use the pip command as usual. As long as you're working with the virtualenv active, your python command will only see the modules you explicitly install in it.
6) Don't forget to re-run . env/bin/activate in every new shell. If you think you'll probably forget, you can add this to your manage.py:
import sys
if __name__ == "__main__":
if not hasattr(sys, 'real_prefix'):
sys.stderr.write('Running outside of any virtualenv - did you forget to activate one?\n')
What are the benefits?
You have an isolated environment for every project (no conflicts).
You may use different versions of the same module in different projects.
System updates will not break your project.
You are not polluting your system with unmanaged files.
You never run stuff as root, which means both added isolation, and the possibility of running your project without having root access to the system.
As long as you keep your requirements.txt up to date (using pip freeze), you can rebuild the virtualenv on another system and it will work.
[edit: using requirements.txt]
That's just a file that has pip install specifications, one by line. It allows to rebuild the virtualenv from scratch easily. You can generate it from your current virtualenv using:
pip freeze > requirements.txt
So the idea is just to remember to re-run this command everytime you change your environment (installing, removing or upgrading some package).
I have build my virtual env with this command:
virtualenv env --distribute --no-site-packages
And then I have installed several modules (django etc) into env with pip, the problem is that when I wanted to run the code on the second machine it would not work, here is what I have done:
visgean#rewitaqia:~/scripty/project_name$ source ./env/bin/activate
(env)visgean#rewitaqia:~/scripty/project_name$ python
Python 2.7.1+ (r271:86832, Sep 27 2012, 21:12:17)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.__file__
'/home/visgean/scripty/project_name/env/lib/python2.7/site-packages/django/__init__.pyc'
but when I want to run it on the second machine:
(env)user#debian:~/project_name$ python
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named django
>>>
I wild error appears! The first machine is ubuntu, the second one is ubuntu. There seems to be some broken links in /home/user/project_name/env/lib/python2.7 , is that the problem? and if so, how should I prevent it/repair it?
If you are just copying the virtualenv to the second machine you may encounter some issues. From the virtualenv site:
Normally environments are tied to a specific path. That means that you
cannot move an environment around or copy it to another computer. You
can fix up an environment to make it relocatable with the command:
$ virtualenv --relocatable ENV
This will make some of the files
created by setuptools or distribute use relative paths, and will
change all the scripts to use activate_this.py instead of using the
location of the Python interpreter to select the environment.
Note: you must run this after you've installed any packages into the
environment. If you make an environment relocatable, then install a
new package, you must run virtualenv --relocatable again.
I have just noticed that I did have a wrong version of python on the second machine - debian does not have python2.7, installing 2.7 and pip for is the solution