Probably a rookie question: I created a project folder, cd'ed to it, created a virtual environment via
python -m venv .venv
and it made the VE in the right place with what looks like the right stuff. But pip-installing modules sometimes (??) fails, with the module going into the global site-packages. This is a vanilla Python 3.9 install on Linux (Raspberry Pi) and being used within Visual Studio Code. To get it right I need to use the "Pip Manager" extension. Why?? To illustrate my confusion, I understand that Python (and Pip?) uses sys.path to resolve packages (see XKCD 1897). So after activating the VE, I looked and got this? I expected to see the .venv site-packages in that path first, right? Well here is what I get:
Where am I misinformed or otherwise confused?
Info added in response to questions below
This system never had Python2 on it. Within the activated .venv there are links:
pi#rpi1:~/Documents/alpyca-device*.venv/bin $ ls -l pyt*
lrwxrwxrwx 1 pi pi 15 Dec 14 14:41 python -> /usr/bin/python
lrwxrwxrwx 1 pi pi 6 Dec 14 14:41 python3 -> python
lrwxrwxrwx 1 pi pi 6 Dec 14 14:41 python3.9 -> python
and this is the usual way it is put into the Python3 virtuals by the venv module, which is recommended now over virtualenv and other schemes.
(.venv) pi#rpi1:~/Documents/alpyca $ which python
/home/pi/Documents/alpyca/.venv/bin/python
and other requests for info:
(.venv) pi#rpi1:~/Documents/alpyca $ grep VIRTUAL_ENV= .venv/bin/activate
VIRTUAL_ENV="/home/pi/Documents/alpyca/.venv"
(.venv) pi#rpi1:~/Documents/alpyca $ .venv/bin/python
Python 3.9.2 (default, Feb 28 2021, 17:03:44)
[GCC 10.2.1 20210110] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> print(sys.path)
['', '/usr/lib/python39.zip', '/usr/lib/python3.9', '/usr/lib/python3.9/lib-dynload', '/home/pi/Documents/alpyca/.venv/lib/python3.9/site-packages']
>>>
I don't know about Python 2, I have never used it.
Related
I'm in the early stages of learning django with python. Have done none of both before.
I'm following a course where the instructor has showed how to start a new project using vagrant, and one of the files in the main directory is called Vagrantfile, no file extension (same for the instructor).
These are the contents of the file:
# -*- mode: ruby -*-
# vi: set ft=ruby :
# All Vagrant configuration is done below. The "2" in Vagrant.configure
# configures the configuration version (we support older styles for
# backwards compatibility). Please don't change it unless you know what
# you're doing.
Vagrant.configure("2") do |config|
# The most common configuration options are documented and commented below.
# For a complete reference, please see the online documentation at
# https://docs.vagrantup.com.
# Every Vagrant development environment requires a box. You can search for
# boxes at https://vagrantcloud.com/search.
config.vm.box = "ubuntu/bionic64"
config.vm.box_version = "~> 20200304.0.0"
config.vm.network "forwarded_port", guest: 8000, host: 8000
config.vm.provision "shell", inline: <<-SHELL
systemctl disable apt-daily.service
systemctl disable apt-daily.timer
sudo apt-get update
sudo apt-get install -y python3-venv zip
touch /home/vagrant/.bash_aliases
if ! grep -q PYTHON_ALIAS_ADDED /home/vagrant/.bash_aliases; then
echo "# PYTHON_ALIAS_ADDED" >> /home/vagrant/.bash_aliases
echo "alias python='python3'" >> /home/vagrant/.bash_aliases
fi
SHELL
end
From my understanding, this line echo "alias python='python3'" >> /home/vagrant/.bash_aliases tells vagrant that when I use the python command, it should regard it as if I wrote python3.
When I write the following command though python -m venv ~/env, I get in response this message:
Command 'python' not found, but can be installed with:
apt install python3
apt install python
apt install python-minimal
Ask your administrator to install one of them.
I call this command after I've ran the vagrant ssh command and I'm already inside the virtual machine, and navigate to the vagrant dir. I know it because in git bash this shows as my current location vagrant#ubuntu-bionic:/vagrant$.
I'm not sure if it's related, or if it's a different issue, but I'll add that even if I run this command instead python3 -m venv ~/env, I get this response:
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt-get install python3-venv
You may need to use sudo with that command. After installing the python3-venv
package, recreate your virtual environment.
Failing command: ['/home/vagrant/env/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']
If I run the install command it recommends (with sudo), I get this in response:
Reading package lists... Done
Building dependency tree
Reading state information... Done
Package python3-venv is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
This blocks my learning, no idea how to move forward with his. Any help is appreciated.
The problem is probably aliasing python3 to python, by default Ubuntu 18 shipped both Python 2 (python) and Python 3 (python3) and changing that carelessly can cause instabilities, ranging from subtle bugs, to your system not booting, you can:
use update-alternatives to change it
use python3 as it is
install the package python-is-python3 (although I'm not sure if this is available in 18)
Anyway, by deleting the alias, everything should go back to normal.
I'm trying to reproduce this on Ubuntu 20.04.
sudo dpkg -i virtualbox-6.1_6.1.14-140239~Ubuntu~eoan_amd64.deb
mkdir vagrant
cd vagrant
vagrant init hashicorp/bionic64
vagrant up
At this point my Vagrantfile contains no alias. See https://github.com/michaelhochleitner/vagrant-test/blob/master/Vagrantfile.
Testing python in vagrant ssh without alias.
vagrant ssh
python
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
python3
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Testing python in vagrant ssh with alias.
At this point my Vagrantfile contains the alias from your post. See https://github.com/michaelhochleitner/vagrant-test/blob/master/Vagrantfile_with_alias.
vagrant reload
vagrant ssh
python
Python 2.7.15+ (default, Nov 27 2018, 23:36:35)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
python3
Python 3.6.8 (default, Jan 14 2019, 11:02:34)
[GCC 8.0.1 20180414 (experimental) [trunk revision 259383]] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
I'm assuming the
Command 'python' not found, but can be installed with:
error is not caused by the alias in the Vagrantfile.
Try to resolve the issue by installing python3-venv.
What is the output of python in a terminal outside vagrant ssh?
What is the output of python3 in a terminal outside vagrant ssh?
Which operating system are you using?
Also see https://askubuntu.com/questions/958303/unable-to-create-virtual-environment-with-python-3-6.
I've been on macOS for 2 years now and in the past year I've begun to work a lot in Python via VScode. But lately I have been running into so many problems because I didn't set up python properly from the start. I have multiple versions and modules installed globally (I know that is bad)... But I was wondering if anyone had advice about how I can clean up the Python set up so that there is the latest version being used and all modules will be installed properly.
I used homebrew as well and that is just adding to the mess. I want to do this right so that I can stop messing with configurations every day and just be able to develop.
I'll include some basic terminal outputs but if there is more that anyone would like to see I would be happy to provide more detail. If starting from scratch is the best thing to do then I'll do it. I don't know my way around all the configuration files and pathing so I'll need some help if that is what I'll have to do.
$ which python
/usr/bin/python
$ which python3
/usr/local/bin/python3
$ python --version
Python 2.7.16
$ python3 --version
Python 3.7.7
$ python3
Python 3.7.7 (default, Mar 10 2020, 15:43:33)
[Clang 11.0.0 (clang-1100.0.33.17)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import tensorflow
>>> tensorflow.__file__
'/usr/local/lib/python3.7/site-packages/tensorflow/__init__.py'
VScode has the following interpreters available (not sure if that helps)
2.7.16 /usr/bin/python
2.7.16 /System/Library/Frameworks....
3.7.3 /usr/bin/python3
3.7.7 /usr/local/bin/python3
3.7.7 /usr/local/opt/python/bin/python3.7
Any help would be awesome!! I'm just tired of fighting with this and wanted to ask for help
Having multiple versions of python is not really a propblem per se.
What I recommend is :
# In $HOME/.bashrc or .zshrc
PATH=/usr/local/bin:$PATH
cd /usr/local/bin
ln -fs python3 python
# Once the first and this step done, when you type [python],
# you'll be using /usr/local/bin/python3
As the first line of your python scripts, put :
#!/usr/bin/env python
This way, you ensure your are always using the version /usr/local/bin/python3
my OS is CentOS Linux release 7.4.1708
First,I install anaconda for python.then I replace the default python in /usr/bin/python.
$ ll /usr/bin/python*
lrwxrwxrwx. 1 root root 7 Aug 15 03:40 /usr/bin/python -> python2
lrwxrwxrwx. 1 root root 9 Aug 9 22:10 /usr/bin/python3 -> python3.6
lrwxrwxrwx. 1 root root 29 Aug 9 22:10 /usr/bin/python2.7 -> /root/anaconda2/bin/python2.7
lrwxrwxrwx. 1 root root 29 Aug 9 21:59 /usr/bin/python3.6 -> /root/anaconda3/bin/python3.6
lrwxrwxrwx. 1 root root 9 Aug 8 23:49 /usr/bin/python2 -> python2.7
Python 2.7.15 |Anaconda, Inc.| (default, May 1 2018, 23:32:55)
[GCC 7.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
so I can't use yum any more.
$ yum
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:
No module named yum
Please install a package which provides this module, or
verify that the module is installed correctly.
It's possible that the above module doesn't match the
current version of Python, which is:
2.7.15 |Anaconda, Inc.| (default, May 1 2018, 23:32:55)
[GCC 7.2.0]
If you cannot solve this problem yourself, please go to
the yum faq at:
http://yum.baseurl.org/wiki/Faq
I'm try to fix vi /usr/bin/yum the first line to any other python path.
but it doesn't work.
also,I'm trying to reinstall python*.rpm like this:
rpm -ivh python-tools-2.7.5-68.el7.x86_64.rpm python-2.7.5-68.el7.x86_64.rpm python-libs-2.7.5-68.el7.x86_64.rpm tkinter-2.7.5-68.el7.x86_64.rpm
and reinstall yum*.rpm (I download a lot of *.rpm today...)
but, still not work.
anyone give me a hand? thanks!
I got this problem on CentOS7 with Yum3.4.3, Python2.7.5 recently,
[root#centos64b build]# yum list There was a problem importing
one of the Python modules required to run yum. The error leading to
this problem was:
No module named yum
Please install a package which provides this module, or verify that
the module is installed correctly.
It's possible that the above module doesn't match the current version
of Python, which is:
2.7.5 (default, Apr 11 2018, 07:36:10) [GCC 4.8.5 20150623 (Red Hat 4.8.5-28)]
If you cannot solve this problem yourself, please go to the yum faq
at: http://yum.baseurl.org/wiki/Faq
While I did not update Python before I met this problem. Finally it is found that python site-packages libpath was not set in sys.path, so the fix here is to append the site-package libpath to sys.path in /usr/bin/yum Python script. Then yum works fine.
[build#centos64b ~]$ more /usr/bin/yum
#!/usr/bin/python
import sys
sys.path.append('/usr/lib/python2.7/site-packages')
sys.path.append('/usr/lib64/python2.7/site-packages')
I found two solution for the problem on Superuser StackExchange
Solution 1
ln -s /etc/yum.conf /etc/yum/yum.conf
Solution 2
Remove newly installed python
rm /usr/bin/python
Link python with the correct version (x.y)
ln -s /usr/bin/pythonx.y /usr/bin/python
Reinstall python with
rpm -ivh --force python-2.7.5-68.el7.x86_64.rpm
Why it works. Generally rpm is tolerant to files of other packages. In your case it sees that it didn't create the link files so it skipped them on install. From man rpm
we can find
--force
Same as using --replacepkgs, --replacefiles, and --oldpackage.
--replacefiles
Install the packages even if they replace files from other, already installed, packages.
With these options rpm does not care about the fact, that old files were created by someone else.
P.S. Some tips: never remove change files in /usr/bin. /bin is better place for your links. Even better add your bin directory to $PATH by adding to your .bash_profile something like this:
$PATH=/root/anaconda2/bin/python2.7:$PATH
So if something breaks, it's just a matter of removing the line from .bash_profile.
And once more: always do backups, especially when working with system files.
I installed python 2.7.5 using make install not altinstall
Previously it was 2.7.2+
now when i run python it gives
ImportError: cannot import name MAXREPEAT
I know there are already entries related to that here. But I dont have virtualenv installed neither did i have before.
If I run python2.7
It runs perfectly.
Now when I am trying to import a library like mysqldb it says the module doesnt exist.
i installed it before installing python2.7.
What is the solution? Should I uninstall the whole python or rather how should I uninstall python2.7.5 only or should i keep that? If I keep that then how can i import those modules?
** Latest UPDATE**
me#me:/usr/local/bin$ dir python2*
python2 python2-config python2.7 python2.7-config
AND:
/usr/local/bin$ ls -l
total 1776
-rwxrwxr-x 1 root root 101 Jul 17 20:17 2to3
-r-xr-xr-x 1 root root 7223 Mar 31 20:47 config_data
-rwxr-xr-x 1 root root 299 Jul 18 20:53 easy_install
-rwxr-xr-x 1 root root 307 Jul 18 20:53 easy_install-2.7
-rwxrwxr-x 1 root root 99 Jul 17 20:17 idle
-r-xr-xr-x 1 root root 3910 Mar 31 20:47 json_pp
-r-xr-xr-x 1 root root 487 Mar 31 20:47 package-stash-conflicts
-rwxr-xr-x 1 root root 281 Jul 17 19:51 pip
-rwxr-xr-x 1 root root 289 Jul 17 19:51 pip-2.7
-rwxrwxr-x 1 root root 84 Jul 17 20:17 pydoc
lrwxrwxrwx 1 root root 7 Jul 18 23:04 python -> python2
lrwxrwxrwx 1 root root 14 Jul 18 23:04 python-config -> python2-config
lrwxrwxrwx 1 root root 9 Jul 18 23:04 python2 -> python2.7
lrwxrwxrwx 1 root root 16 Jul 18 23:04 python2-config -> python2.7-config
-rwxr-xr-x 1 root root 1737376 Jul 18 23:03 python2.7
-rwxr-xr-x 1 root root 1674 Jul 18 23:04 python2.7-config
-rwxrwxr-x 1 root root 18547 Jul 17 20:17 smtpd.py
-rwxr-xr-x 1 root root 316 Jul 18 21:10 virtualenv
-rwxr-xr-x 1 root root 324 Jul 18 21:10 virtualenv-2.7
would reinstalling this solve the problem of getting and running the default 2.7.2+ in ubuntu back?: https://launchpad.net/ubuntu/oneiric/+source/python-defaults/2.7.2-7ubuntu2
* Prior UPDATE*
Now I can run python from bash
Here is the new problem. and it is with every module that I install using pip.
existing modules from lib can be imported but newly installed ones cant be:
me:~$ sudo pip install requests
Downloading/unpacking requests
Downloading requests-1.2.3.tar.gz (348kB): 348kB downloaded
Running setup.py egg_info for package requests
Installing collected packages: requests
Running setup.py install for requests
Successfully installed requests
Cleaning up...
me:~$ python -i
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import requests
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named requests
>>>
After checking version of python it is running locally, for conflict issues:
me:~$ python
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
me:~$ sudo python
[sudo] password for me:
Python 2.7.5 (default, Jul 17 2013, 20:16:12)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> exit()
me:~$ which python
/usr/local/bin/python
me:~$ sudo which python
/usr/local/bin/python
me:~$ sudo which python2.7
/usr/local/bin/python2.7
me:~$ which python2.7
/usr/local/bin/python2.7
I think its running the same. and then..
me:/usr/local/bin$ dir python2.7
python2.7
me:/usr/local/bin$ cd /usr/bin
me:/usr/bin$ dir python2.7
python2.7
me:/bin$ cd /usr/bin
me:/usr/bin$ dir python
python
me:/usr/local/bin$ dir python
python
me:/usr/local/bin$ dir python2.7
python2.7
Also check following:
~$ which pip
/usr/local/bin/pip
Ok now I installed virtualenv
and this is what i did
(foldername)me#me:~/caller$ sudo pip install requests
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/local/lib/python2.7/dist-packages
Cleaning up...
(foldername)me#me:~/caller$ python -i
Python 2.7.2+ (default, Jul 20 2012, 22:15:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
And NOW we can see that 2.7.2+ is back again! So what is happening now??
what is wrong?
Uninstalling older python from linux is not recommended. So how can i run virtualenv with 2.7.5 on it and not 2.7.2+ ?!
When installing python use the following steps
using prefix to specify the installation directory
./configure --prefix=/scratch/bin/
make
sudo make install
Then everytime u run a new Terminal u have specify
export PATH="$PATH:/scratch/bin/"
to tell where is the installation directory of Python
This way u can use any number of pythons
Have you installed it using apt-get or built from sources?
If built from sources, are you sure the installation has finished successfully? Usually in order to build Python from sources one has to do the following steps
./configure
make
sudo make install (sudo might not be required, but the make script will attempt to change files in /usr/ directory.
in your python sources directory. The last command, amongst others, copies python to the /usr/ directory. If you want to have it installed somewhere else you'd have to pass --path=XXX (if i'm not mistaken) to ./configure.
pip is a Python program. When you run pip from the command line, it uses a particular Python executable to run the program. pip will install modules in a location where that Python executable can find them.
When you run sudo pip install some_package, the OS looks at the shebang line to determine what Python executable to run.
Type which pip and look at the first line in the file to see which python executable it is linked to.
To install packages for your new python2.7.5, you could either install a new pip. Or, you could just run /usr/local/bin/python pip ..., but I think the easiest way to manage multiple python installations is to use virtualenv.
If you use virtualenv (which I'd highly recommend), then a new pip is automatically installed since it is one of its dependencies.
If you go this route, I'd recommend installing virtualenvwrapper too.
If you install pip globally, notice the line
python get-pip.py
or
python setup.py install
The python executable you use on this line sets the "link" between that version of pip and that version of python.
How to use checkinstall to remove a custom-built Python:
sudo apt-get install libsqlite3-dev libgdbm-dev liblzma-dev checkinstall
sudo apt-get build-dep python
http://www.python.org/ftp/python/2.7.5/Python-2.7.5.tar.bz2
tar xvjf Python-2.7.5.tar.bz2
cd Python-2.7.5
# inspect the options
./configure --help
./configure
make
make test
sudo checkinstall -D --fstrans=no make install
# Accept the default answers EXCEPT that you MUST change the name to something
# other than python. That way, the name of your custom package does not conflict
# with the name of Ubuntu's "python" package
Should I create a default set of package docs? [y]:
Enter a number to change any of them or press ENTER to continue: 2
Enter new name:
>> python275
Copying files to the temporary directory...OK
# wait, wait, wait...
# python2.7.5 is now installed in /usr/local/bin
# remove it with:
sudo dpkg -r python275
on my computer
~$ python -V
Python 3.2.1
but I get into problems when I run some python programs. my guess is (or at least I want to try this) that there is some backward compatibility issues, and I want to run those python scripts with
python2 2.7.2-2
which is also installed on my system but I do not know how to make it as the (temporary) default python. The python script starts with
#!/usr/bin/env python
and I am using arch linux.
You can use virtualenv
# Use this to create your temporary python "install"
# (Assuming that is the correct path to the python interpreter you want to use.)
virtualenv -p /usr/bin/python2.7 --distribute temp-python
# Type this command when you want to use your temporary python.
# While you are using your temporary python you will also have access to a temporary pip,
# which will keep all packages installed with it separate from your main python install.
# A shorter version of this command would be ". temp-python/bin/activate"
source temp-python/bin/activate
# When you no longer wish to use you temporary python type
deactivate
Enjoy!
mkdir ~/bin
PATH=~/bin:$PATH
ln -s /usr/bin/python2 ~/bin/python
To stop using python2, exit or rm ~/bin/python.
Just call the script using something like python2.7 or python2 instead of just python.
So:
python2 myscript.py
instead of:
python myscript.py
What you could alternatively do is to replace the symbolic link "python" in /usr/bin which currently links to python3 with a link to the required python2/2.x executable. Then you could just call it as you would with python 3.
You don't want a "temporary default Python"
You want the 2.7 scripts to start with
/usr/bin/env python2.7
And you want the 3.2 scripts to begin with
/usr/bin/env python3.2
There's really no use for a "default" Python. And the idea of a "temporary default" is just a road to absolute confusion.
Remember.
Explicit is better than Implicit.
You could use alias python="/usr/bin/python2.7":
bash-3.2$ alias
bash-3.2$ python
Python 2.7.6 (v2.7.6:3a1db0d2747e, Nov 10 2013, 00:42:54)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> ^D
bash-3.2$ alias python="/usr/bin/python3.3"
bash-3.2$ python
Python 3.3.3 (v3.3.3:c3896275c0f6, Nov 16 2013, 23:39:35)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
If you have some problems with virtualenv,
You can use it:
sudo ln -sf python2 /usr/bin/python
and
sudo ln -sf python3 /usr/bin/python
Use python command to launch scripts, not shell directly. E.g.
python2 /usr/bin/command
AFAIK this is the recommended method to workaround scripts with bad env interpreter line.
As an alternative to virtualenv, you can use anaconda.
On Linux, to create an environment with python 2.7:
conda create -n python2p7 python=2.7
source activate python2p7
To deactivate it, you do:
source deactivate
It is possible to install other package inside your environment.
I think it is easier to use update-alternatives:
sudo update-alternatives --install /usr/bin/python python /usr/bin/python2.7 1