I have two python installation. One is installed in default location and other is installed in my home directory. I am using home directory python installation. I am facing problem about pythonpath. When I run sys.path on 2nd python installation it gives me paths for libraries of default python installation
eg.
2nd python is installed in /home/python2.6.1
now when i use sys.path it gives me
/lib/python2.6/lib-old, /lib/python2.6/lib-dynload etc.
instead of
/home/python2.6.1/lib/python2.6/lib-old, /home/python2.6.1/lib/python2.6/lib-dynload
how can I change path for my python installation in home directory
This is because your libraries are installed in /lib or the interpreter has been configured to scan /lib for libraries. You need to recompile Python and give the configure script another prefix for the libraries.
But, this isn't what I would do. I'd install virtualenv and start managing my Python versions that way. This is what I do actually ;) Clean and simple!
Related
Good day!
Installed the Python 3.9.1, checked "Add to path", the cmd did not work though.
Added Environment Variable Path, both folder
C:\Users\XXXXX\AppData\Local\Programs\Python\Python39
(file manager opens the path to python.exe just fine)
and script lines:
C:\Users\XXXXX\AppData\Local\Programs\Python\Python39
Still the commands python -version and pip --version do not work from the command line.
Py --version works just fine though.
Anyone might share and idea what might be the reason?
This happens more often than one would think.
When you installed python from python.org and follow the installer, post install, you should check your environment variables, and verify that you have no other python installation (or if you do just name them appropriately).
Often you can find that there is an Environment Variable, that includes a python version on the global Environment Variable Path.
To verify that you are targeting the correct python version put these two directories (replace, user and python version), at the top of your PATH (user's path), and check that there are no conflicts with the rest of the PATH variables.
C:\Users\XXXXX\AppData\Local\Programs\Python\PythonXX
C:\Users\XXXXX\AppData\Local\Programs\Python\PythonXX\Scripts
If you had Python installed in the system before, the new path is added at the end of PATH system variable and when system looks for python.exe it finds first the old version that is available under a different folder.
If you used a command window opened before the new version got installed, it is also possible that system variables did not reload. Close it and use a new one to check.
I wanted to replace Python 3.8 32-bit with the 64-bit version to install the face_recognition module, so I deleted the previous version and tried to re-route the project to the new Python version by going to File > Settings > Project Interpreter > Show all > Show Paths for Selected Interpreter, and adding all the Python files from the new folder and getting rid of the old ones.
However, it's still showing me this error when I try to install the module:
(Will2.0) C:\Users\solei\PycharmProjects\Will>pip install face_recognition
No Python at 'C:\Users\solei\AppData\Local\Programs\Python\Python38-32\python.exe'
I've also tried going to the Windows System Properties and changing everything that says "Python38-32" there, but it's still not working. It does work when I make a new environment, though, so at least I know that Python installed properly. It's just this one environment that is tripping me up (I'd prefer not to make a new project for this, btw. I've already installed a lot of modules in it.).
Your selected interpreter is not the system interpreter you've replaced with the 64-bit version, but your project's virtual environment interpreter. The virtual environment's files weren't changed in that process and need to be updated before you can use that environment again.
The system interpreter is your Python interpreter installed using the installation executable. In your case it is located in C:\Users\solei\AppData\Local\Programs\Python\Python38\. You can have multiple system interpreters installed, such as having Python 2.7, Python 3.7 and Python 3.8 side-by-side.
The virtual environment interpreter is a copy of another interpreter created using the venv package from the Python standard library. You can have many virtual environments interpreters in the system (one or more for every project, for example)
The base interpreter is the interpreter that was used as a template for the venv package. Every virtual environment interpreter has its base interpreter (usually a system interpreter) that it requires to run. Changing or upgrading the base interpreter requires updating the virtual environment.
If we take a quick look at the documentation, a virtual environment is described as
a self-contained directory tree that contains a Python installation for a particular version of Python, plus a number of additional packages.
That means you can setup an individual environment for every project, which will contain its own packages. The environment is a very efficient way of managing project packages, that's why PyCharm suggests a creation of such environment over the system interpreter by default. In short, it allows you to have two different versions of the same package used by two different projects, without the packages conflicting with each other.
This also explains why your virtual environment files weren't affected by your upgrade.
Now, I am unfortunately no Python expert. I had to spend some time examining how Python handles virtual environments on Windows and Ubuntu. It seems the environment always requires the base system interpreter present in the system. If you remove or change the location of the base interpreter, the environment will fail to function.
As I mentioned before editing this answer, you can in theory simply edit the pyenv.cfg file located in the root folder of the virtual environment. In practice, that will only work in simple cases and it is not the intended way of updating virtual environments.
You need to upgrade your virtual environment's files to work with your new system interpreter. That can mean the 64-bit version over the 32-bit version, or even a newer version of Python - such us upgrading from 3.7 to 3.8.
Close PyCharm
Check if the system interpreter you want to upgrade to is on the system Path
You can quickly check by running
python -c "import platform; print(platform.architecture())"
For you, the output should look like this
('64bit', 'WindowsPE')
If your output is different, you'll need to prefix the absolute path to the Python executable in step 4).
Navigate to the virtual environment's directory
The directory you're looking for contains the Include, Lib and Scripts directories and the pyenv.cfg file. From your screenshots, it seems this directory is your project's root directory, so in your case:
cd C:\Users\solei\PycharmProjects\Will2.0\
Upgrade the virtual environment
python -m venv --upgrade .
... or if Python is not on your path
C:\Users\solei\AppData\Local\Programs\Python\Python38\python.exe -m venv --upgrade .
The . in the commands refers to the current directory.
Open PyCharm and verify your environment is working correctly
... or simply try to run pip directly from the command line. Note you need to first activate the virtual environment by running the Scripts\activate.bat batch file.
If the above-mentioned method doesn't work, you might have to create a new virtual environment. You can create one easily without making a new PyCharm project. See this PyCharm documentation for reference. However, you'll still need to redownload all the required packages again.
For the simplicity, I recommend creating the new virtual environment in a .venv folder located in the project's root.
Disclaimer
I tested only the Python's behavior alone on a fresh Windows installation inside the Windows Sandbox. I was able to install the 32-bit Python, create a virtual environment, replace Python with the 64-bit version and upgrade the virtual environment to have it launch correctly again.
I'm a huge beginner so I'm not very informed about how packages really work. I know that you should create a virtualenv in your project folder to avoid version conflicts etc, and you're not supposed to put your actual project files in the virtual env. So if your project files are in your project directory on the same level as the virtualenv, can your project files "access" the things installed in the virtualenv? Can files outside of your directory access packages in your virtual env?
Yes, it all depends on the context. Your virtualenv can exist anywhere, be it in your project directory, or somewhere else.
When you want to use the virtualenv, you just have to call source command on it. Then whatever python command you execute on whichever file, will have access to the virtualenv. For example, if you store your virtualenv in /home/user/project/virtualenv, then you would do
source /home/user/project/virtualenv/bin/activate
Then whatever you with the python, it would be the version installed in virtualenv.
You can double check if you're using the global python or the virtualenv python by doing which python. It will either point to the global python path which is usually under /usr/bin/python or /home/user/project/virtualenv/bin/python.
So normally, you first do the source command, then you can do pip install on whatever packages you need already. It will be installed in the virtualenv and it will not conflict with other projects.
I installed a module compiling the source code. I did that using python3.4 and the module is located at /usr/local/lib/python3.4/site-packages/. If I try to run it, python3.4 cannot find it. If I add the directory to PYTHONPATH, everything works.
But I have two versions of python (I'm running ubuntu). Python2.7 and python3.4. After I add the directory to PYTHONPATH it becomes:
/home/myuser/development/ros/devel/lib/python2.7/dist-packages:/opt/ros/indigo/lib/python2.7/dist-packages:/home/myuser/Qt/5.5/gcc_64/lib/cmake/lib/python2.7/dist-packages:/usr/local/lib/python3.4/site-packages/:/home/myuser/development/lcm
so that there are directories related to python2.7 and python3.4 in the same environment variable, which is totally unsafe and crazy I think. Its very error prone.
I've been looking around, and found that people recommend using virtualenv. Nevertheless, that doesn't change the PYTHONPATH.
What's the standard way to handle this? some thoughts:
I would expect that python3.4 would automatically look for modules in /usr/local/lib/python3.4/site-packages/ and that is not happening. Maybe my python3.4 installation has something off?
Even if I manage to use that module for python3.4 without using PYTHONPATH I still have info in PYTHONPATH related to python2.7 even when I am using python3.4, so I should probably remove that too.
So, how do I set the path for python to look for modules globally (not using sys.path() for each program) but with each python version using it's own path?
EDIT: From a comment, I realized that I could just add a symbolic link to the directory where my modules are. But I found that neither python 2.7 or 3.4 have site-packages in their path, which is wrong.. is this an ubuntu issue? I read somewhere that site-packages should be used for self compiled modules, and dist-packages for modules installed via apt-get.
EDIT 2: python#.#/site-packages is reserved for custom packages. At least in ubuntu, python won't look in there by default. So the behavior I was having was the correct one, nothing was wrong on the installation.
For user-specific packages, the best place to install them is in the per-user site directory. On a unix system, this will be ~/.local by default, but it can be changed by setting the PYTHONUSERBASE environment variable. To see what the current settings are on your system, run the following command:
[user#localhost tmp]$ PYTHONPATH='' python -m site
sys.path = [
'/tmp',
'/usr/lib/python35.zip',
'/usr/lib/python3.5',
'/usr/lib/python3.5/plat-linux',
'/usr/lib/python3.5/lib-dynload',
'/usr/lib/python3.5/site-packages',
]
USER_BASE: '/home/user/.local' (exists)
USER_SITE: '/home/user/.local/lib/python3.5/site-packages' (doesn't exist)
ENABLE_USER_SITE: True
Note that the directories that will eventually appear under .local/lib are versioned, thus avoiding the issues with PYTHONPATH.
So, when you install a package, all you need to do is ensure you specify the right python executable, and point the configure/setup script at your user site directory:
python3.5 setup.py install --prefix /home/user/.local
Or, if you're using pip, the same thing can be achieved with the --user option:
pip3 install --user some-package
I am developing in both Python 3 and Python 2.6, and have both versions installed. With Python 3, however, the path to lots of the good modules (time, math, ...) is not part of my Python path. I can add the directory to the path, but it's tedious.
Is there a way to permanently modify the path for my Python 3 installation without affecting Python 2?
Create virtual environment: http://pypi.python.org/pypi/virtualenv/
Install packages you want in virtual environment.
If you can't import time, math modules that are in stdlib then your installation of Python 3 is broken.
When you run python setup.py install the files are installed in correct place for current python executable be it a system python or python from a virtualenv environment. The same goes for pip.
You don't need to modify any paths, just use an appropriate executables.