We Keep Coding

Resolving errors in python environment on Windows

I just started having some problems with python, and would like to fix this the proper way, as a last resort, reinstalling if I have to.
Before Python 3, I had Python 2.7 installed. I then installed Python 3.7 along side it. I don't remember, if the path environment was automatically set, or if i manually set it.
My system variables path contains :
C:\Python27\;C:\Python27\Scripts;
So I believe this was set automatically.
My user variables path however contains :
C:\Python37\;C:\Python37\Scripts;
Would the python installer use the user environment? I'm not sure, so I don't know if I set that myself.
However, the user variables path also contains :
%PYTHON_DIR%\Python37\Scripts\;%PYTHON_DIR%\Python37\;%PYTHON_DIR%\Python36\Scripts\;%PYTHON_DIR%\Python36\
Again, I don't know if I set this myself, and I have no recollection of setting this in my user variables :
PYTHON_DIR = C:\Users\pcuser\AppData\Local\Programs\Python
...but it's there, and I have two folders in that path.
I also have this in my user variables path :
PYTHON = os.path.expanduser(os.getenv('PYTHON', 'C:\\Python37\\python.exe'))
I know I created a python environment. I just have to look for it, as I don't remember where to find it right now.
Here is the problem I am having.
I also am using Python in MSYS, and get this error from Python in C:\msys64\usr\bin :
Could not find platform independent libraries <prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000000800018040 (most recent call first):
0 [main] python 1489 cygwin_exception::open_stackdumpfile: Dumping stack t
race to python.exe.stackdump
I set PYTHONHOME to C:\msys64\usr\bin. That doesn't solve the problem.
I also removed PYTHON_DIR to see if there was a conflict there, but that didn't solve the problem.
Rather, I got another error :
Fatal Python error: initfsencoding: Unable to get the locale encoding
ModuleNotFoundError: No module named 'encodings'
Current thread 0x0000000800018040 (most recent call first):
0 [main] python 143 cygwin_exception::open_stackdumpfile: Dumping stack tr
ace to python.exe.stackdump
So it seems to me my Python environment is messed up, and I want to learn how to set it up correctly, hopefully without having to start over.
How can I get rid of these errors, and get a proper python environment?

Personally, I use miniconda.
Miniconda is the small version of anaconda which is an excellent environment manager. Miniconda does not include a gui and I feel it is nicer to work with.
To create an environment with
conda create -n my-test-env python=3.9
You can activate it using
conda activate my-test-env
You can install other packages using
conda install numpy.
However, not all python packages are available in conda. You can any pip packages by installing pip
conda install pip
pip install numpy

Setting PYTHONHOME to a value like C:\msys64\usr\bin is definitely wrong; it should be something like C:\Python37\Lib if that's where Python installed its libraries. But try simply unsetting it.

After a while of troubleshooting, I was able to discover where the problem was.
Some files in C:\msys64\usr\lib\python3.8 somehow got deleted, or python 3.8 was installed, but not properly. So I did a reinstall, and python stopped complaining about missing encodings and modules.
Running a test on all python installations
MINGW64 ~
# python
Python 3.8.2 (default, Feb 27 2020, 05:27:33) [GCC 9.2.0 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
# python setup_build.py install
running install
running build
running build_ext
skipping 'geotools/geotools.c' Cython extension (up-to-date)
running install_lib
copying build/lib.mingw-3.8/geotools-cpython-38.dll -> C:/msys64/mingw64/lib/python3.8/site-packages
running install_egg_info
Writing C:/msys64/mingw64/lib/python3.8/site-packages/geotools-0.1.0-py3.8.egg-info
MINGW32 ~
# python
Python 3.8.2 (default, Feb 27 2020, 06:39:26) [GCC 9.2.0 32 bit] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
# python setup_build.py install
running install
running build
running build_ext
dllwrap: WARNING: dllwrap is deprecated, use gcc -shared or ld -shared instead
skipping 'geotools/geotools.c' Cython extension (up-to-date)
running install_lib
running install_egg_info
Removing C:/msys64/mingw32/lib/python3.8/site-packages/geotools-0.1.0-py3.8.egg-info
Writing C:/msys64/mingw32/lib/python3.8/site-packages/geotools-0.1.0-py3.8.egg-info
C:\WINDOWS\system32>py -3.7 -m pip list
Package Version
----------------------------- ---------
...
C:\WINDOWS\system32>py -2.7 -m pip list
DEPRECATION: Python 2.7 reached the end of its life on January 1st, 2020. Please
upgrade your Python as Python 2.7 is no longer maintained. pip 21.0 will drop s
upport for Python 2.7 in January 2021. More details about Python 2 support in pi
p can be found at https://pip.pypa.io/en/latest/development/release-process/#pyt
hon-2-support pip 21.0 will remove support for this functionality.
Package Version
----------------------------- ----------
...
C:\WINDOWS\system32>python
Python 2.7.14 (v2.7.14:84471935ed, Sep 16 2017, 20:25:58) [MSC v.1500 64 bit (AM
D64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\WINDOWS\\SYSTEM32\\python27.zip', 'C:\\Python27\\DLLs', 'C:\\Python27\
\lib', 'C:\\Python27\\lib\\plat-win', 'C:\\Python27\\lib\\lib-tk', 'C:\\Python27
', 'C:\\Users\\pcUser\\AppData\\Roaming\\Python\\Python27\\site-packages', 'C:\\Py
thon27\\lib\\site-packages', 'C:\\Python27\\lib\\site-packages\\pybind11-2.6.2-p
y2.7.egg']
>>>
C:\WINDOWS\system32>py -3
Python 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit
(AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', 'C:\\Users\\pcUser\\AppData\\Local\\Programs\\Python\\Python37\\python37.zip'
, 'C:\\Users\\pcUser\\AppData\\Local\\Programs\\Python\\Python37\\DLLs', 'C:\\User
s\\pcUser\\AppData\\Local\\Programs\\Python\\Python37\\lib', 'C:\\Users\\pcUser\\App
Data\\Local\\Programs\\Python\\Python37', 'C:\\Users\\pcUser\\AppData\\Roaming\\Py
thon\\Python37\\site-packages', 'C:\\Users\\pcUser\\AppData\\Local\\Programs\\Pyth
on\\Python37\\lib\\site-packages']
>>>
C:\WINDOWS\system32>py -3.6 -m pip list
Package Version
--------------- --------
...
My python environment is good. All working nicely. :)
What I learned...
Using PYTHONHOME and PYTHONPATH is not neccesary, as was suggested by many... although equally suggessted by many.
From my experience - speaking fron a novice perspective - I would not recommend using them.
Maybe they work for others, but I got errors regardless of how I set it up - whether
C:\Python37 or
C:\Python37\lib or
C:\Users\pcUser\AppData\Local\Programs\Python\Python37 or
C:\msys64\usr\lib\python3.8;C:\msys64\usr\lib\python3.8\site-packages;C:\msys64\usr\lib\python3.8\lib-dynload;C:\msys64\usr\lib\python3.8\distutils
They all return a heap of errors on one installment or other.
So it does not work for me. However, probably works for others.
I am happy though that after probably more than 60 hours, my python environment is back in working order. :)

import torch ModuleNotFoundError: No module named 'torch'

Help me understand few things, im new with python and all these depended libraries.
I m trying to run a project which is written in python. Git repository can be found here: https://github.com/ifzhang/FairMOT
On anaconda prompt I have been running these commands:
conda create -n FairMOT
conda activate FairMOT
conda install pytorch==1.2.0 torchvision==0.4.0 cudatoolkit=10.0 -c pytorch
cd ${FAIRMOT_ROOT}
pip install -r requirements.txt
All being successfully installed
I have performed all the steps that were listed in the Readme file
Now there is training step which requires to run shell script.
Assuming the fact that shell script would not be called on anaconda prompt, I switched to git bash, and I run the script (keeping in mind of path)
sh experiments/crowdhuman_dla34.sh
it throws error
Sanam#LAPTOP-NPVR76P7 MINGW64 /f/NTNU/Deep learning/Repositories/FairMOT (master)
$ sh experiments/crowdhuman_dla34.sh
Traceback (most recent call last):
File "train.py", line 10, in <module>
import torch
ModuleNotFoundError: No module named 'torch'
Question is: is there any other console where I need to run this command other than git bash? it would definitely not work on anaconda prompt. what im doing wrong?
coming back to anaconda prompt, when I test import there, it worked
(FairMOT) F:\NTNU\Deep learning\Repositories\FairMOT>python
Python 3.7.9 (default, Aug 31 2020, 17:10:11) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>>
where & how do I run this command? so that it would work?is running on git bash not good?
thanks in advance!!
PS:
I have already tried reinstalling, activation/ deactivation of environment but it does not work

Pls. create a virtual envrionment first, then install all the dependencies there. Use the same venv in anaconda and run the bash script in any terminal with the venv activated.
Also, you can run which python to ensure you are installing and then importing it from same python interpreter

Using twisted on OS X Mavericks

I am trying to use twisted on OS X Mavericks, but I get this error message when I try to import it.
christohersmbp2:~ christopherspears$ python
Python 2.7.6 (default, Mar 8 2014, 09:29:01)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import twisted
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named twisted
My guess is that I am receiving this error because I am not using the default Python. I am using a Python installed with brew. Ideally, I would want to install twisted into a virtual environment to play with, but the docs are seriously lacking in details. Apparently, a dmg exists for Mac OS X 10.5, which is not helpful for me. I can install from the tarball into the virtual environment, but I am not sure how to do this. Any hints?

If you're using virtualenv, it doesn't matter whether you are using the system python or not.
Simply pip install twisted in your virtualenv, like:
$ workon MyTwistedEnv
$ pip install twisted
Currently, due to a bug in Xcode that affects all projects which contain extension modules, you may need to spell this
$ CFLAGS= pip install twisted
instead; hopefully this will be fixed soon. It may not affect brew installed Pythons, however.

Issue with Python Path. "ImportError: No module named setuptools"

I recently updated to Python 2.7 in order to start working with Django "Django requires Python, specifically Python 2.6.5 - 2.7.x."
As a result I needed to install some of the tools I have been using when I went through the excellent LPTHW Exercise 46
pip from http://pypi.python.org/pypi/pip
distribute from http://pypi.python.org/pypi/distribute
nose from http://pypi.python.org/pypi/nose/
virtualenv from http://pypi.python.org/pypi/virtualenv
Should be easy enough, I had done it all before...
When I run python in terminal I get the below. Note: Python 2.7.5
$ python
Python 2.7.5 (v2.7.5:ab05e7dd2788, May 13 2013, 13:18:45)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
When I type "which python" I get
$ which python
/Library/Frameworks/Python.framework/Versions/2.7/bin/python
However when I try to install distribute, nose or virtualenv I get the below error. Note Python/2.6
$ sudo pip install distribute
Exception:
Traceback (most recent call last):
File "/Library/Python/2.6/site-packages/pip-1.3.1-py2.6.egg/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/Library/Python/2.6/site-packages/pip-1.3.1-py2.6.egg/pip/commands/install.py", line 258, in run
import setuptools
ImportError: No module named setuptools
Storing complete log in /Users/brendanspillane/Library/Logs/pip.log
localhost:~ brendanspillane$
I believe the Python/2.6 is the issue here. As a result i tried to set my Python Path (my first time) Using this question as a reference "changing python path on mac?" i typed in
PATH="/Library/Frameworks/Python.framework/Versions/2.7/bin:${PATH}"
export PATH
This has no apparent effect.
However I am still not able to run sudo pip install distribute and keep getting the same error.
Can anyone help me? Is it my Python Path? or am I looking in the wrong direction?
Any help is as always, much appreciated.
Deepend

I'm not really sure how it worked but it did. #Robert Lujo deserves the up votes for his answer at https://stackoverflow.com/a/16511140/1214163
If the answer is also added here i will accept it.

Virtualenv wont work on

I have build my virtual env with this command:
virtualenv env --distribute --no-site-packages
And then I have installed several modules (django etc) into env with pip, the problem is that when I wanted to run the code on the second machine it would not work, here is what I have done:
visgean#rewitaqia:~/scripty/project_name$ source ./env/bin/activate
(env)visgean#rewitaqia:~/scripty/project_name$ python
Python 2.7.1+ (r271:86832, Sep 27 2012, 21:12:17)
[GCC 4.5.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
>>> django.__file__
'/home/visgean/scripty/project_name/env/lib/python2.7/site-packages/django/__init__.pyc'
but when I want to run it on the second machine:
(env)user#debian:~/project_name$ python
Python 2.6.6 (r266:84292, Dec 27 2010, 00:02:40)
[GCC 4.4.5] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import django
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named django
>>>
I wild error appears! The first machine is ubuntu, the second one is ubuntu. There seems to be some broken links in /home/user/project_name/env/lib/python2.7 , is that the problem? and if so, how should I prevent it/repair it?

If you are just copying the virtualenv to the second machine you may encounter some issues. From the virtualenv site:
Normally environments are tied to a specific path. That means that you
cannot move an environment around or copy it to another computer. You
can fix up an environment to make it relocatable with the command:
$ virtualenv --relocatable ENV
This will make some of the files
created by setuptools or distribute use relative paths, and will
change all the scripts to use activate_this.py instead of using the
location of the Python interpreter to select the environment.
Note: you must run this after you've installed any packages into the
environment. If you make an environment relocatable, then install a
new package, you must run virtualenv --relocatable again.

I have just noticed that I did have a wrong version of python on the second machine - debian does not have python2.7, installing 2.7 and pip for is the solution