We Keep Coding

Use Python in powershell with conda environment activated

I have a conda environment named ml, and I have activated this environment in powershell using following commands:
>> conda init powershell
>> conda activate ml
(ml) >>
When I open a notebook with jupyter notebook command, it works fine and I have access to the modules inside ml environment:
import torch # No errors inside a jupyter notebook cell
But when I try to test my modules inside the powershell and Python prompt, it doesn't work:
(ml) >> py
Python 3.8.6 (tags/v3.8.6:db45529, Sep 23 2020, 15:52:53) [MSC v.1927 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'torch'
I'm newbie to the conda environments. Is there any way to use conda environments within Python prompt of windows powershell?

Make sure pytorch is installed in your environment using:
conda install -c pytorch pytorch

I think there must be some issue with the conda initialization in the powershell try restarting the powershell once you have done init
conda init powershell
and then activate environment then it should work

ModuleNotFoundError: No module named 'gensim'

My goal is to import gensim in Python 3 on Windows.
I am using Python 3.7.2 (checked by running python -V in Windows command prompt). I installed gensim by running pip install gensim. I checked the installation by running pip freeze, and saw the line gensim==3.7.3.
Then, I ran the command py to enter the interactive python mode (still in Windows command prompt). I ran the line import gensim and got the following output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gensim'
I also tried from gensim import test and got the following output:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ModuleNotFoundError: No module named 'gensim'
Any suggestions? How do I install gensim on Windows with Python 3? How do I test gensim?

In Mac, open anaconda navigator, Click on 'Open Terminal option'. If you are using Windows, Run anaconda prompt as administrator and run the following command:
conda install -c conda-forge gensim

I think you have installed it using normal cmd, so it may have installed it on python2.x. Install it with anaconda prompt.
Let me know if it worked for you.

I got the same error after installing gensim in Anaconda. It worked only after I re-started the Anaconda: by exiting it, and re-opening it via the command prompt. I wanted to share this experience since someone else may meet the same issue.

To understand why this happens, you must know how Windows finds executables to run, and how the Python software is installed.
When running a command, Windows searches for an executable in the environment variable PATH. It executes the first one found.
python.exe is installed in <PYTHON_INSTALL_DIR> (e.g. C:\Python\3.7).
pip.exe and other Python tools (e.g. pylint, virtualenv, pycrust, etc.) are installed in <PYTHON_INSTALL_DIR>\Scripts.
py.exe is installed in your Windows system directory (e.g. C:\Windows).
python and pip commands use the modules found in the directory their installed in, they do not look at PATH.
So, let's say you have the following Python versions:
C:\Python\2.7
C:\Python\3.6
C:\Python\3.7
and your PATH environment contains the following directories:
C:\Python\2.7
C:\Python\3.6\Scripts
then, see the following output:
C:\>python -V
Python 2.7.16
C:\>pip -V
pip 19.1.1 from c:\python\3.6\lib\site-packages\pip (python 3.6)
C:\>py -V
Python 3.7.3
So, when running pip, it is possible that the packages are installed in another Python version then the version you'll get when running python.
To see which versions are (correctly) installed on your system, run py -0p. Example output:
C:\>py -0p
Installed Pythons found by py Launcher for Windows
-3.7-64 C:\Python\3.7-64\python.exe *
-3.7-32 C:\Python\3.7-32\python.exe
-3.6-64 C:\Python\3.6-64\python.exe
-2.7-64 C:\Python\2.7-64\python.exe
-2.7-32 C:\Python\2.7-32\python.exe
General solution (for Windows)
The best thing is not to rely on your system PATH. Use the py launcher to select the version you want. To run the pip module corresponding to the Python version you want to use, start pip as a module instead of executable.
So instead of:
pip install <package>
run:
py -3.6 -m pip install <package>

Here's My hypothesis towards your situation, since your OS is able to recognize both python and py commands in commandline, this may mean that you have two separate versions of python installed.
Since, as you mentioned that python -V shows gensim as an installed module. Try opening python interactive interpreter via command python instead of py, and import gensim module in it.
C:\Users> Python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 21:26:53) [MSC v.1916 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import gensim
see if that works.
P.S.
I would not recommend having two different compiler versions on a single OS as it creates a lot of commotion, and create incompatibility issues with program's made on one compiler with the other. And makes problems (like you mentioned) a lot more prevalent.

Most probably you have > 1 python installed in your machine. To install gensim (or any package) inside python command line, you can run below:
type "python" then enter
type "import subprocess" then enter
type
"subprocess.check_call(["python", '-m', 'pip', 'install', 'gensim'])"
then enter
Sample below:
>>> import subprocess
>>> subprocess.check_call(["python", '-m', 'pip', 'install', 'gensim'])
Collecting gensim
..
...
Installing collected packages: smart-open, gensim
Successfully installed gensim-3.7.3 smart-open-1.8.3
0

Anaconda installed but No module named 'conda' after installing flake8

While trying to install atom-lint package in Atom editor I somehow corrupted my conda installation.
I did the following things that might have caused the issue:
Installed a python dependency Flake8 using conda install.
Messed around with Atom Init Script (I can provide more info if needed)
After I did these things I encountered the following problem:
If I run conda I get
$ conda
Traceback (most recent call last):
File "/Users/me/miniconda3/bin/conda", line 12, in <module>
from conda.cli import main
ModuleNotFoundError: No module named 'conda'
Strangely enough if I run python in my command line anaconda still seems to be installed an working.
$ python
Python 3.7.1 (default, Oct 23 2018, 14:07:42)
[Clang 4.0.1 (tags/RELEASE_401/final)] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
I have also noticed that the Python3 bin has disappeared from the conda environment folder ~/miniconda3/envs/my_env/bin/
This is very strange and I thought might be related.
Any help would be much appreciated.

I ended up making a backup copy of the miniconda3/envs folder, reinstalling miniconda and copying the environment back in. It works now, not sure what caused the issue.

Do I need to install django separately for different python version?

I had python version 2.7.3 and i wanted to learn django so i installed django version 1.8.2 on my ubuntu 12.0.4 .
invivtus#invictus:~/bin$ python
Python 2.7.3 (default, Sep 26 2013, 20:08:41)
[GCC 4.6.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.VERSION
(1, 8, 2, 'final', 0)
Then i read that best way yo work with django is to work on python version 3.3 so i installed python version 3.3.6 on my system where py is symbolic link pointing to /opt/python3.3/bin/python3.3
invictus#invictus:~/bin$ py
Python 3.3.6 (default, Jun 21 2015, 16:13:35)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.
when I try and import django here i get error
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'django
I see django got installed my python 2.7 directory.
>>> import django
>>> django
<module 'django' from '/usr/local/lib/python2.7/dist-packages/django/__init__.pyc'>
How can i use this django with my 3.3.6 version. My default python version is 2.7.3
What are possible workout here?

Yes, each Python version has its own folder with installed packages. You'll have to install Django separately for Python 3.3. The same is true for every package that is not available by default.
(If you're using Python 3, why not go for the latest and greatest, 3.4?)

As some of the comments said, you should be using a virtualenv to isolate your environments. You would do it like this:
1) Ensure you have virtualenv installed. On Ubuntu for instance, that would be package virtualenv.
2) Create a new, empty environment. You choose which python version it will be like this:
virtualenv -p /usr/bin/python3.4 env
3) That created an env folder. Activate the newly created environment:
. env/bin/activate
This updates your paths so now, when you run python or pip from this shell, they will execute in the context of your virtualenv.
4) Update the virtualenv (optional)
pip install -U pip
5) Install whatever packages you need. The recommended way is to have a requirements.txt file at the root of your project. You would pull them this way:
pip install -r myproject/requirements.txt
That's it. Use the pip command as usual. As long as you're working with the virtualenv active, your python command will only see the modules you explicitly install in it.
6) Don't forget to re-run . env/bin/activate in every new shell. If you think you'll probably forget, you can add this to your manage.py:
import sys
if __name__ == "__main__":
if not hasattr(sys, 'real_prefix'):
sys.stderr.write('Running outside of any virtualenv - did you forget to activate one?\n')
What are the benefits?
You have an isolated environment for every project (no conflicts).
You may use different versions of the same module in different projects.
System updates will not break your project.
You are not polluting your system with unmanaged files.
You never run stuff as root, which means both added isolation, and the possibility of running your project without having root access to the system.
As long as you keep your requirements.txt up to date (using pip freeze), you can rebuild the virtualenv on another system and it will work.
[edit: using requirements.txt]
That's just a file that has pip install specifications, one by line. It allows to rebuild the virtualenv from scratch easily. You can generate it from your current virtualenv using:
pip freeze > requirements.txt
So the idea is just to remember to re-run this command everytime you change your environment (installing, removing or upgrading some package).

WARNING: IPython History requires SQLite, your history will not be saved

Hi I'm using Ubuntu release 12.10 (quantal) 32-bit with Linux Kernel 3.5.0-21-generic. I'm trying to get IPython's History to work. I've set it up using pythonbrew and a virtual environment. In there I use pip to install IPython. Currently, when I start up IPython in a terminal I get:
WARNING: IPython History requires SQLite, your history will not be saved
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
Searching on the warning in the first line, I found this issue report, so I went back and installed the following:
sudo apt-get install libsqlite0 libsqlite0-dev libsqlite3-0 libsqlite3-dev
and then removed and reinstalled pysqlite using pip
pip uninstall pysqlite
pip install pysqlite
After that I thought I would check the installation by importing the module:
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: No module named _sqlite3
So now it seems the file _sqlite3.so can't be found. That's when I found this SO question. Either it doesn't exist or it's not in my PYTHONPATH environment variable. Searching for the file, I get:
$ locate _sqlite3.so
/home/me/Desktop/.dropbox-dist/_sqlite3.so
/home/me/epd/lib/python2.7/lib-dynload/_sqlite3.so
/usr/lib/python2.7/lib-dynload/_sqlite3.so
So the file is there, but when I looked in my python path:
import sys
for p in sys.path:
print p
none of the above paths that contain _sqlite3.so were contained in my PYTHONPATH. For giggles, I added the path /usr/lib/python2.7/lib-dynload to my PYTHONPATH in a terminal and then tried to import sqlite3 again:
Python 2.7.3 (default, Nov 8 2012, 18:25:10)
[GCC 4.7.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.append("/usr/lib/python2.7/lib-dynload")
>>> import sqlite3
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/__init__.py", line 24, in <module>
from dbapi2 import *
File "/home/me/.pythonbrew/pythons/Python-2.7.3/lib/python2.7/sqlite3/dbapi2.py", line 27, in <module>
from _sqlite3 import *
ImportError: /usr/lib/python2.7/lib-dynload/_sqlite3.so: undefined symbol: PyUnicodeUCS4_DecodeUTF8
Uh oh. Now I'm completely stuck. Can anyone help me out? I've also read in a few places that I may have to rebuild Python. I have no idea how to do this in pythonbrew. Can anyone point me in the right direction?

I've also read in a few places that I may have to rebuild Python.
This is correct. SQLite is part of the standard library,
and is built when you compile Python. There are a few 'optional' parts
of the standard library, which Python will simply skip (with minimal warning, unfortunately)
if the dependencies are missing at build time, and sqlite is one of these.
You should be able to just install libsqlite3-dev,
then rebuild Python and you should be set.
Keep an eye on the build messages,
as they do report which modules they are skipping due to missing dependencies.

Thanks to minrk for pointing me in the right direction. All I had to do was rebuild python. I've outlined the steps below for those that are using pythonbrew. Notice that I already installed the libsqlite3-dev package up in the question section.
First, with the proper version of python and virtual environment loaded up run the command:
$ pip freeze -l > requirements.txt
This gives us a text file list of all of the pip packages that have been installed in the virtual environment for this particular python release in pythonbrew. Then we remove the version of python from pythonbrew and reinstall it (this is the "rebuild python" step):
$ pythonbrew uninstall 2.7.3
$ pythonbrew install 2.7.3
After that, we switch over to the newly installed python version 2.7.3 and create a new virtual environment (which I've called "sci"):
$ pythonbrew switch 2.7.3
$ pythonbrew venv create sci
$ pythonbrew venv use sci
Ideally you should be able to run the command:
$ pip install -r requirements.txt
and according to this pip should reinstall all the modules that you had in the virtual environment before we clobbered that version of python (2.7.3). It didn't work for me for whatever reason so I manually installed all of the modules using pip individuality.
$ ipython --pylab
Python 2.7.3 (default, Jan 5 2013, 18:48:27)
Type "copyright", "credits" or "license" for more information.
IPython 0.13.1 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
and IPython history works!

What worked for me (using osx + homebrew + brewed python):
# Reinstall Python 2.7 with sqlite
brew remove python
brew install readline sqlite gdbm --universal
brew install python --universal --framework
# Reinstall iPython with correct bindings
pip uninstall ipython
pip install ipython
And you should be good to go.

You should rebuild your python with sqlite support
sudo apt-get install libsqlite3-dev
wget https://www.python.org/ftp/python/2.7.15/Python-2.7.15.tgz
tar -xvf Python-2.7.15.tgz
cd Python-2.7.15
./configure
make
sudo make install
Recreate your virtual environment and you should be good to go
rmvirtualenv venv
mkvirtualenv -p python2 venv
workon venv
pip install -r requirements.txt
# or
pip install ipython

This warning appears on macOS when python is installed with pyenv. By default it installs python without sqlite. These commands reinstall python with sqlite support:
pyenv uninstall 3.7
CFLAGS="-I$(xcrun --show-sdk-path)/usr/include" pyenv install 3.7